2024-03-08T05:50:13.326 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-08T05:50:13.326 DEBUG:teuthology.run:Teuthology command: teuthology --name jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi --owner scheduled_jcollin@teuthology --archive /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417 --verbose --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin} -- /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417/orig.config.yaml 2024-03-08T05:50:13.366 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-08T05:50:13.452 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417 branch: main description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin} email: jcollin@redhat.com first_in_suite: false job_id: '7585417' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: lockdep: true ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - missing required features sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 kclient: syntax: v2 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-jcollin-test_per-client-perf-counters sha1: de7a7e179fbee9e314fa702f8f6c09d121c5e1a7 owner: scheduled_jcollin@teuthology priority: 10 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7313 sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-jcollin-test_per-client-perf-counters suite_path: /home/teuthworker/src/github.com_joscollin_ceph_de7a7e179fbee9e314fa702f8f6c09d121c5e1a7/qa suite_relpath: qa suite_repo: https://github.com/joscollin/ceph.git suite_sha1: de7a7e179fbee9e314fa702f8f6c09d121c5e1a7 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi076.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKhnkFqUvrG7IuDf2BSBP5sxy4mKIsOWPSx8+7GZs+T1PXg7Ik6V+ReoC7jNDuzTxr44jT+2K9nPAyuWL4OFTgk= smithi081.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBRX6/RIgJsEJ5Pga1/gJkuha9CjhD24Ar+FSY6norAtgGSq5IKJS2vFEpHLUJ/WwLDDmn/t3NCnOcTMIfvYZwc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_admin.TestLabeledPerfCounters teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-08_05:43:46 tube: smithi user: jcollin verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-08T05:50:13.452 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_joscollin_ceph_de7a7e179fbee9e314fa702f8f6c09d121c5e1a7/qa; will attempt to use it 2024-03-08T05:50:13.453 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_joscollin_ceph_de7a7e179fbee9e314fa702f8f6c09d121c5e1a7/qa/tasks 2024-03-08T05:50:13.453 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-08T05:50:13.455 INFO:teuthology.task.internal:Checking packages... 2024-03-08T05:50:13.476 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1fb5d10608dccee73edd6ef8a2c7b804a097c324' 2024-03-08T05:50:13.476 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-08T05:50:13.476 INFO:teuthology.packaging:ref: None 2024-03-08T05:50:13.476 INFO:teuthology.packaging:tag: None 2024-03-08T05:50:13.476 INFO:teuthology.packaging:branch: main 2024-03-08T05:50:13.476 INFO:teuthology.packaging:sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-03-08T05:50:13.476 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-03-08T05:50:13.717 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1940.g639d1827 2024-03-08T05:50:13.718 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-08T05:50:13.725 INFO:teuthology.task.internal:no buildpackages task found 2024-03-08T05:50:13.725 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-08T05:50:13.763 INFO:teuthology.task.internal:Saving configuration 2024-03-08T05:50:13.777 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-08T05:50:13.784 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-08T05:50:13.805 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 05:44:35.857957', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKhnkFqUvrG7IuDf2BSBP5sxy4mKIsOWPSx8+7GZs+T1PXg7Ik6V+ReoC7jNDuzTxr44jT+2K9nPAyuWL4OFTgk='} 2024-03-08T05:50:13.826 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 05:44:35.859438', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBRX6/RIgJsEJ5Pga1/gJkuha9CjhD24Ar+FSY6norAtgGSq5IKJS2vFEpHLUJ/WwLDDmn/t3NCnOcTMIfvYZwc='} 2024-03-08T05:50:13.826 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-08T05:50:13.833 INFO:teuthology.task.internal:roles: ubuntu@smithi076.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-08T05:50:13.833 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-08T05:50:13.833 INFO:teuthology.run_tasks:Running task console_log... 2024-03-08T05:50:13.902 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa62a27d0d0>, signals=[15]) 2024-03-08T05:50:13.903 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-08T05:50:13.909 INFO:teuthology.task.internal:Opening connections... 2024-03-08T05:50:13.909 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi076.front.sepia.ceph.com 2024-03-08T05:50:13.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:50:13.999 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi081.front.sepia.ceph.com 2024-03-08T05:50:14.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:50:14.082 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-08T05:50:14.089 DEBUG:teuthology.orchestra.run.smithi076:> uname -m 2024-03-08T05:50:14.104 INFO:teuthology.orchestra.run.smithi076.stdout:x86_64 2024-03-08T05:50:14.105 DEBUG:teuthology.orchestra.run.smithi076:> cat /etc/os-release 2024-03-08T05:50:14.162 INFO:teuthology.orchestra.run.smithi076.stdout:NAME="CentOS Stream" 2024-03-08T05:50:14.162 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION="9" 2024-03-08T05:50:14.162 INFO:teuthology.orchestra.run.smithi076.stdout:ID="centos" 2024-03-08T05:50:14.162 INFO:teuthology.orchestra.run.smithi076.stdout:ID_LIKE="rhel fedora" 2024-03-08T05:50:14.162 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION_ID="9" 2024-03-08T05:50:14.162 INFO:teuthology.orchestra.run.smithi076.stdout:PLATFORM_ID="platform:el9" 2024-03-08T05:50:14.162 INFO:teuthology.orchestra.run.smithi076.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-08T05:50:14.162 INFO:teuthology.orchestra.run.smithi076.stdout:ANSI_COLOR="0;31" 2024-03-08T05:50:14.162 INFO:teuthology.orchestra.run.smithi076.stdout:LOGO="fedora-logo-icon" 2024-03-08T05:50:14.162 INFO:teuthology.orchestra.run.smithi076.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-08T05:50:14.163 INFO:teuthology.orchestra.run.smithi076.stdout:HOME_URL="https://centos.org/" 2024-03-08T05:50:14.163 INFO:teuthology.orchestra.run.smithi076.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-08T05:50:14.163 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-08T05:50:14.163 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-08T05:50:14.163 INFO:teuthology.lock.ops:Updating smithi076.front.sepia.ceph.com on lock server 2024-03-08T05:50:14.185 DEBUG:teuthology.orchestra.run.smithi081:> uname -m 2024-03-08T05:50:14.200 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2024-03-08T05:50:14.200 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release 2024-03-08T05:50:14.254 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="CentOS Stream" 2024-03-08T05:50:14.254 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="9" 2024-03-08T05:50:14.254 INFO:teuthology.orchestra.run.smithi081.stdout:ID="centos" 2024-03-08T05:50:14.254 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="rhel fedora" 2024-03-08T05:50:14.254 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="9" 2024-03-08T05:50:14.254 INFO:teuthology.orchestra.run.smithi081.stdout:PLATFORM_ID="platform:el9" 2024-03-08T05:50:14.254 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-08T05:50:14.254 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2024-03-08T05:50:14.254 INFO:teuthology.orchestra.run.smithi081.stdout:LOGO="fedora-logo-icon" 2024-03-08T05:50:14.254 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-08T05:50:14.254 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://centos.org/" 2024-03-08T05:50:14.255 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-08T05:50:14.255 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-08T05:50:14.255 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-08T05:50:14.255 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2024-03-08T05:50:14.285 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-08T05:50:14.294 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-08T05:50:14.300 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-08T05:50:14.300 DEBUG:teuthology.orchestra.run.smithi076:> test '!' -e /home/ubuntu/cephtest 2024-03-08T05:50:14.302 DEBUG:teuthology.orchestra.run.smithi081:> test '!' -e /home/ubuntu/cephtest 2024-03-08T05:50:14.315 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-08T05:50:14.321 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-08T05:50:14.321 DEBUG:teuthology.orchestra.run.smithi076:> test -z $(ls -A /var/lib/ceph) 2024-03-08T05:50:14.360 DEBUG:teuthology.orchestra.run.smithi081:> test -z $(ls -A /var/lib/ceph) 2024-03-08T05:50:14.373 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-08T05:50:14.424 INFO:teuthology.run_tasks:Running task kernel... 2024-03-08T05:50:14.438 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-08T05:50:14.438 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-03-08T05:50:14.438 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-03-08T05:50:14.439 DEBUG:teuthology.orchestra.run.smithi076:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-08T05:50:14.439 DEBUG:teuthology.orchestra.run.smithi081:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-08T05:50:14.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T05:50:14.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T05:50:14.453 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2024-03-08T05:50:14.453 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-03-08T05:50:14.506 INFO:teuthology.orchestra.run.smithi081.stdout:5.14.0-361.el9.x86_64 2024-03-08T05:50:14.506 INFO:teuthology.task.kernel:Running kernel on smithi081: 5.14.0-361.el9.x86_64 2024-03-08T05:50:14.507 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum install -y kernel 2024-03-08T05:50:14.511 INFO:teuthology.orchestra.run.smithi076.stdout:5.14.0-361.el9.x86_64 2024-03-08T05:50:14.511 INFO:teuthology.task.kernel:Running kernel on smithi076: 5.14.0-361.el9.x86_64 2024-03-08T05:50:14.511 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum install -y kernel 2024-03-08T05:50:14.733 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for process with pid 5567 to finish. 2024-03-08T05:50:15.126 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:25 ago on Fri 08 Mar 2024 05:49:50 AM UTC. 2024-03-08T05:50:15.224 INFO:teuthology.orchestra.run.smithi081.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-08T05:50:15.224 INFO:teuthology.orchestra.run.smithi081.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-08T05:50:15.224 INFO:teuthology.orchestra.run.smithi081.stdout:Package kernel-5.14.0-427.el9.x86_64 is already installed. 2024-03-08T05:50:15.275 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-08T05:50:15.276 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-03-08T05:50:15.276 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-08T05:50:15.321 DEBUG:teuthology.orchestra.run.smithi081:> echo no | sudo yum reinstall kernel || true 2024-03-08T05:50:15.889 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:25 ago on Fri 08 Mar 2024 05:49:50 AM UTC. 2024-03-08T05:50:16.037 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-08T05:50:16.038 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-08T05:50:16.038 INFO:teuthology.orchestra.run.smithi081.stdout: Package Architecture Version Repository Size 2024-03-08T05:50:16.038 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-08T05:50:16.038 INFO:teuthology.orchestra.run.smithi081.stdout:Reinstalling: 2024-03-08T05:50:16.039 INFO:teuthology.orchestra.run.smithi081.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-08T05:50:16.039 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:50:16.039 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-08T05:50:16.039 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-08T05:50:16.039 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:50:16.039 INFO:teuthology.orchestra.run.smithi081.stdout:Total size: 6.0 M 2024-03-08T05:50:16.039 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 0 2024-03-08T05:50:16.039 INFO:teuthology.orchestra.run.smithi081.stderr:Operation aborted. 2024-03-08T05:50:16.068 INFO:teuthology.orchestra.run.smithi081.stdout:Is this ok [y/N]: 2024-03-08T05:50:16.069 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum reinstall -y kernel || true 2024-03-08T05:50:16.634 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:26 ago on Fri 08 Mar 2024 05:49:50 AM UTC. 2024-03-08T05:50:16.782 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-08T05:50:16.783 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-08T05:50:16.784 INFO:teuthology.orchestra.run.smithi081.stdout: Package Architecture Version Repository Size 2024-03-08T05:50:16.784 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-08T05:50:16.784 INFO:teuthology.orchestra.run.smithi081.stdout:Reinstalling: 2024-03-08T05:50:16.784 INFO:teuthology.orchestra.run.smithi081.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-08T05:50:16.784 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:50:16.784 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-08T05:50:16.784 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-08T05:50:16.784 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:50:16.784 INFO:teuthology.orchestra.run.smithi081.stdout:Total size: 6.0 M 2024-03-08T05:50:16.784 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 0 2024-03-08T05:50:16.785 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-08T05:50:16.785 INFO:teuthology.orchestra.run.smithi081.stdout:[SKIPPED] kernel-5.14.0-427.el9.x86_64.rpm: Already downloaded 2024-03-08T05:50:16.852 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-08T05:50:16.852 INFO:teuthology.orchestra.run.smithi081.stdout:Waiting for process with pid 5526 to finish. 2024-03-08T05:50:37.252 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:00:01 ago on Fri 08 Mar 2024 05:50:36 AM UTC. 2024-03-08T05:50:38.433 INFO:teuthology.orchestra.run.smithi076.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-08T05:50:38.433 INFO:teuthology.orchestra.run.smithi076.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-08T05:50:38.592 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-03-08T05:50:38.597 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-03-08T05:50:38.597 INFO:teuthology.orchestra.run.smithi076.stdout: Package Architecture Version Repository Size 2024-03-08T05:50:38.597 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-03-08T05:50:38.597 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-03-08T05:50:38.597 INFO:teuthology.orchestra.run.smithi076.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-08T05:50:38.597 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-03-08T05:50:38.597 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-08T05:50:38.597 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-08T05:50:38.597 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-08T05:50:38.598 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:50:38.598 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-03-08T05:50:38.598 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-03-08T05:50:38.598 INFO:teuthology.orchestra.run.smithi076.stdout:Install 4 Packages 2024-03-08T05:50:38.598 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:50:38.598 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 100 M 2024-03-08T05:50:38.599 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 123 M 2024-03-08T05:50:38.599 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-03-08T05:50:40.005 INFO:teuthology.orchestra.run.smithi076.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 5.9 MB/s | 6.0 MB 00:01 2024-03-08T05:50:40.747 INFO:teuthology.orchestra.run.smithi076.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 12 MB/s | 21 MB 00:01 2024-03-08T05:50:41.538 INFO:teuthology.orchestra.run.smithi076.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 15 MB/s | 39 MB 00:02 2024-03-08T05:50:42.155 INFO:teuthology.orchestra.run.smithi076.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 16 MB/s | 33 MB 00:02 2024-03-08T05:50:42.155 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-03-08T05:50:42.156 INFO:teuthology.orchestra.run.smithi076.stdout:Total 28 MB/s | 100 MB 00:03 2024-03-08T05:50:43.060 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-03-08T05:50:43.459 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-03-08T05:50:43.459 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-03-08T05:50:44.804 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-03-08T05:50:44.804 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-03-08T05:50:45.427 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-08T05:50:45.427 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-08T05:50:45.433 INFO:teuthology.orchestra.run.smithi081.stderr:Error: Cannot open file /var/cache/dnf/baseos-044cae74d71fe9ea/packages/kernel-5.14.0-427.el9.x86_64.rpm: [Errno 2] No such file or directory: '/var/cache/dnf/baseos-044cae74d71fe9ea/packages/kernel-5.14.0-427.el9.x86_64.rpm' 2024-03-08T05:50:45.434 INFO:teuthology.orchestra.run.smithi081.stderr:TypeError: an integer is required (got type NoneType) 2024-03-08T05:50:45.434 INFO:teuthology.orchestra.run.smithi081.stderr:FATAL ERROR: python callback ??? failed, aborting! 2024-03-08T05:50:45.441 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q kernel | sort -rV | head -n 1 2024-03-08T05:50:45.565 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-08T05:50:45.565 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-08T05:50:45.566 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-08T05:50:45.566 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-08T05:50:45.566 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-08T05:50:45.566 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-03-08T05:50:45.566 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi081.front.sepia.ceph.com, path=None, version=distro) 2024-03-08T05:50:45.566 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q kernel | sort -rV | head -n 1 2024-03-08T05:50:45.691 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-08T05:50:45.692 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-08T05:50:45.692 DEBUG:teuthology.orchestra.run.smithi081:> sudo rpm -qi grub2-tools 2024-03-08T05:50:45.722 INFO:teuthology.orchestra.run.smithi081.stdout:Name : grub2-tools 2024-03-08T05:50:45.723 INFO:teuthology.orchestra.run.smithi081.stdout:Epoch : 1 2024-03-08T05:50:45.723 INFO:teuthology.orchestra.run.smithi081.stdout:Version : 2.06 2024-03-08T05:50:45.723 INFO:teuthology.orchestra.run.smithi081.stdout:Release : 68.el9 2024-03-08T05:50:45.723 INFO:teuthology.orchestra.run.smithi081.stdout:Architecture: x86_64 2024-03-08T05:50:45.723 INFO:teuthology.orchestra.run.smithi081.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-08T05:50:45.723 INFO:teuthology.orchestra.run.smithi081.stdout:Group : Unspecified 2024-03-08T05:50:45.723 INFO:teuthology.orchestra.run.smithi081.stdout:Size : 8264254 2024-03-08T05:50:45.723 INFO:teuthology.orchestra.run.smithi081.stdout:License : GPLv3+ 2024-03-08T05:50:45.723 INFO:teuthology.orchestra.run.smithi081.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-08T05:50:45.723 INFO:teuthology.orchestra.run.smithi081.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-08T05:50:45.723 INFO:teuthology.orchestra.run.smithi081.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-08T05:50:45.723 INFO:teuthology.orchestra.run.smithi081.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-08T05:50:45.723 INFO:teuthology.orchestra.run.smithi081.stdout:Packager : builder@centos.org 2024-03-08T05:50:45.723 INFO:teuthology.orchestra.run.smithi081.stdout:Vendor : CentOS 2024-03-08T05:50:45.723 INFO:teuthology.orchestra.run.smithi081.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-08T05:50:45.724 INFO:teuthology.orchestra.run.smithi081.stdout:Summary : Support tools for GRUB. 2024-03-08T05:50:45.724 INFO:teuthology.orchestra.run.smithi081.stdout:Description : 2024-03-08T05:50:45.724 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:50:45.724 INFO:teuthology.orchestra.run.smithi081.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-08T05:50:45.724 INFO:teuthology.orchestra.run.smithi081.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-08T05:50:45.724 INFO:teuthology.orchestra.run.smithi081.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-08T05:50:45.724 INFO:teuthology.orchestra.run.smithi081.stdout:hardware devices. 2024-03-08T05:50:45.724 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:50:45.724 INFO:teuthology.orchestra.run.smithi081.stdout:This subpackage provides tools for support of all platforms. 2024-03-08T05:50:45.725 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-08T05:50:45.725 INFO:teuthology.task.kernel:Updating grub on smithi081 to boot 5.14.0-427.el9.x86_64 2024-03-08T05:50:45.725 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-08T05:50:46.396 INFO:teuthology.orchestra.run.smithi081.stderr:Generating grub configuration file ... 2024-03-08T05:50:47.517 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-03-08T05:50:47.806 INFO:teuthology.orchestra.run.smithi081.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-08T05:50:47.839 INFO:teuthology.orchestra.run.smithi081.stderr:done 2024-03-08T05:50:47.841 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-03-08T05:50:47.905 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.OuNfHene6Z 2024-03-08T05:50:47.906 DEBUG:teuthology.orchestra.run.smithi081:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.OuNfHene6Z 2024-03-08T05:50:47.975 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.OuNfHene6Z 2024-03-08T05:50:48.128 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.OuNfHene6Z is 6KB 2024-03-08T05:50:48.175 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-08T05:50:48.176 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.OuNfHene6Z 2024-03-08T05:50:48.190 DEBUG:teuthology.orchestra.run.smithi081:> sudo /bin/ls /boot/loader/entries || true 2024-03-08T05:50:48.254 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-08T05:50:48.254 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-08T05:50:48.254 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-08T05:50:48.254 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-08T05:50:48.256 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-08T05:50:48.303 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-08T05:50:48.459 DEBUG:teuthology.orchestra.run.smithi081:> sudo shutdown -r now 2024-03-08T05:50:51.169 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-08T05:50:51.360 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-08T05:50:56.154 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-08T05:50:56.289 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-08T05:51:01.068 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-08T05:51:18.462 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-08T05:51:18.463 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-08T05:51:18.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:51:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-08T05:51:30.388 INFO:teuthology.orchestra.run.smithi076.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-08T05:51:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:51:30.473 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-08T05:51:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-08T05:51:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-08T05:51:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-08T05:51:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-08T05:51:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-08T05:51:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:51:31.366 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-03-08T05:51:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-08T05:51:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-08T05:51:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-08T05:51:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-08T05:51:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:51:31.366 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-03-08T05:51:31.890 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q kernel | sort -rV | head -n 1 2024-03-08T05:51:32.033 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-08T05:51:32.033 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-08T05:51:32.033 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-08T05:51:32.033 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-08T05:51:32.034 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-08T05:51:32.034 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-08T05:51:32.034 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi076.front.sepia.ceph.com, path=None, version=distro) 2024-03-08T05:51:32.034 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q kernel | sort -rV | head -n 1 2024-03-08T05:51:32.155 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-08T05:51:32.155 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-08T05:51:32.155 DEBUG:teuthology.orchestra.run.smithi076:> sudo rpm -qi grub2-tools 2024-03-08T05:51:32.189 INFO:teuthology.orchestra.run.smithi076.stdout:Name : grub2-tools 2024-03-08T05:51:32.189 INFO:teuthology.orchestra.run.smithi076.stdout:Epoch : 1 2024-03-08T05:51:32.189 INFO:teuthology.orchestra.run.smithi076.stdout:Version : 2.06 2024-03-08T05:51:32.190 INFO:teuthology.orchestra.run.smithi076.stdout:Release : 68.el9 2024-03-08T05:51:32.190 INFO:teuthology.orchestra.run.smithi076.stdout:Architecture: x86_64 2024-03-08T05:51:32.190 INFO:teuthology.orchestra.run.smithi076.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-08T05:51:32.190 INFO:teuthology.orchestra.run.smithi076.stdout:Group : Unspecified 2024-03-08T05:51:32.190 INFO:teuthology.orchestra.run.smithi076.stdout:Size : 8264254 2024-03-08T05:51:32.190 INFO:teuthology.orchestra.run.smithi076.stdout:License : GPLv3+ 2024-03-08T05:51:32.190 INFO:teuthology.orchestra.run.smithi076.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-08T05:51:32.190 INFO:teuthology.orchestra.run.smithi076.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-08T05:51:32.190 INFO:teuthology.orchestra.run.smithi076.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-08T05:51:32.190 INFO:teuthology.orchestra.run.smithi076.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-08T05:51:32.190 INFO:teuthology.orchestra.run.smithi076.stdout:Packager : builder@centos.org 2024-03-08T05:51:32.190 INFO:teuthology.orchestra.run.smithi076.stdout:Vendor : CentOS 2024-03-08T05:51:32.190 INFO:teuthology.orchestra.run.smithi076.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-08T05:51:32.191 INFO:teuthology.orchestra.run.smithi076.stdout:Summary : Support tools for GRUB. 2024-03-08T05:51:32.191 INFO:teuthology.orchestra.run.smithi076.stdout:Description : 2024-03-08T05:51:32.191 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:51:32.191 INFO:teuthology.orchestra.run.smithi076.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-08T05:51:32.191 INFO:teuthology.orchestra.run.smithi076.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-08T05:51:32.191 INFO:teuthology.orchestra.run.smithi076.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-08T05:51:32.191 INFO:teuthology.orchestra.run.smithi076.stdout:hardware devices. 2024-03-08T05:51:32.191 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:51:32.191 INFO:teuthology.orchestra.run.smithi076.stdout:This subpackage provides tools for support of all platforms. 2024-03-08T05:51:32.192 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-08T05:51:32.192 INFO:teuthology.task.kernel:Updating grub on smithi076 to boot 5.14.0-427.el9.x86_64 2024-03-08T05:51:32.192 DEBUG:teuthology.orchestra.run.smithi076:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-08T05:51:32.891 INFO:teuthology.orchestra.run.smithi076.stderr:Generating grub configuration file ... 2024-03-08T05:51:34.272 INFO:teuthology.orchestra.run.smithi076.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-08T05:51:34.327 INFO:teuthology.orchestra.run.smithi076.stderr:done 2024-03-08T05:51:34.329 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2024-03-08T05:51:34.345 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.Isr4ttrhzw 2024-03-08T05:51:34.345 DEBUG:teuthology.orchestra.run.smithi076:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Isr4ttrhzw 2024-03-08T05:51:34.414 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.Isr4ttrhzw 2024-03-08T05:51:34.550 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.Isr4ttrhzw is 6KB 2024-03-08T05:51:34.571 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.Isr4ttrhzw 2024-03-08T05:51:34.585 DEBUG:teuthology.orchestra.run.smithi076:> sudo /bin/ls /boot/loader/entries || true 2024-03-08T05:51:34.649 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-08T05:51:34.650 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-08T05:51:34.650 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-08T05:51:34.650 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-08T05:51:34.651 DEBUG:teuthology.orchestra.run.smithi076:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-08T05:51:34.841 DEBUG:teuthology.orchestra.run.smithi076:> sudo shutdown -r now 2024-03-08T05:51:36.991 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.81 2024-03-08T05:51:46.000 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-08T05:51:46.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:51:49.055 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.81 2024-03-08T05:52:01.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-08T05:52:01.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:52:04.848 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-08T05:52:04.848 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-03-08T05:52:04.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:52:23.327 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-03-08T05:52:32.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-03-08T05:52:32.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:52:32.904 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-03-08T05:52:33.297 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-08T05:52:33.297 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-427.el9.x86_64"... 2024-03-08T05:52:33.297 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-03-08T05:52:33.310 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-03-08T05:52:33.312 INFO:teuthology.orchestra.run.smithi081.stdout:5.14.0-427.el9.x86_64 2024-03-08T05:52:33.312 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-08T05:52:33.312 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-08T05:52:33.313 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-08T05:52:34.314 DEBUG:teuthology.parallel:result is None 2024-03-08T05:52:45.311 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-03-08T05:52:45.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:52:46.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-03-08T05:53:01.624 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-03-08T05:53:01.625 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:53:33.661 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-03-08T05:53:34.184 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-03-08T05:53:34.184 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-427.el9.x86_64"... 2024-03-08T05:53:34.184 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2024-03-08T05:53:34.210 INFO:teuthology.orchestra.run.smithi076.stdout:5.14.0-427.el9.x86_64 2024-03-08T05:53:34.210 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-08T05:53:34.210 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-08T05:53:34.210 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-08T05:53:35.211 DEBUG:teuthology.parallel:result is None 2024-03-08T05:53:35.211 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-08T05:53:35.221 INFO:teuthology.task.internal:Creating test directory... 2024-03-08T05:53:35.222 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-08T05:53:35.225 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-08T05:53:35.244 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-08T05:53:35.259 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-08T05:53:35.269 INFO:teuthology.task.internal:Creating archive directory... 2024-03-08T05:53:35.269 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-08T05:53:35.283 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-08T05:53:35.317 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-08T05:53:35.324 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-08T05:53:35.324 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-08T05:53:35.349 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-08T05:53:35.389 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T05:53:35.403 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T05:53:35.404 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T05:53:35.463 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T05:53:35.465 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-08T05:53:35.472 INFO:teuthology.task.internal:Configuring sudo... 2024-03-08T05:53:35.473 DEBUG:teuthology.orchestra.run.smithi076:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-08T05:53:35.475 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-08T05:53:35.505 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-08T05:53:35.514 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-08T05:53:35.514 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-08T05:53:35.541 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-08T05:53:35.563 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T05:53:35.636 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T05:53:35.682 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T05:53:35.763 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T05:53:35.804 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T05:53:35.804 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-08T05:53:35.871 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T05:53:35.915 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T05:53:35.969 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T05:53:36.055 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T05:53:36.098 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T05:53:36.099 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-08T05:53:36.168 DEBUG:teuthology.orchestra.run.smithi076:> sudo service rsyslog restart 2024-03-08T05:53:36.170 DEBUG:teuthology.orchestra.run.smithi081:> sudo service rsyslog restart 2024-03-08T05:53:36.203 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T05:53:36.288 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T05:53:36.696 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-08T05:53:36.704 INFO:teuthology.task.internal:Starting timer... 2024-03-08T05:53:36.704 INFO:teuthology.run_tasks:Running task pcp... 2024-03-08T05:53:36.714 INFO:teuthology.run_tasks:Running task selinux... 2024-03-08T05:53:36.722 DEBUG:teuthology.orchestra.run.smithi076:> sudo service auditd rotate 2024-03-08T05:53:36.791 INFO:teuthology.orchestra.run.smithi076.stdout:Rotating logs: 2024-03-08T05:53:36.793 DEBUG:teuthology.orchestra.run.smithi081:> sudo service auditd rotate 2024-03-08T05:53:36.850 INFO:teuthology.orchestra.run.smithi081.stdout:Rotating logs: 2024-03-08T05:53:36.852 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-08T05:53:36.852 DEBUG:teuthology.orchestra.run.smithi076:> /usr/sbin/getenforce 2024-03-08T05:53:36.878 INFO:teuthology.orchestra.run.smithi076.stdout:Permissive 2024-03-08T05:53:36.878 DEBUG:teuthology.orchestra.run.smithi081:> /usr/sbin/getenforce 2024-03-08T05:53:36.902 INFO:teuthology.orchestra.run.smithi081.stdout:Permissive 2024-03-08T05:53:36.902 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi076.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi081.front.sepia.ceph.com': 'permissive'} 2024-03-08T05:53:36.903 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-08T05:53:36.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T05:53:36.945 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-08T05:53:36.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T05:53:36.974 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-08T05:53:36.974 DEBUG:teuthology.orchestra.run.smithi076:> sudo /usr/sbin/setenforce permissive 2024-03-08T05:53:37.022 DEBUG:teuthology.orchestra.run.smithi081:> sudo /usr/sbin/setenforce permissive 2024-03-08T05:53:37.054 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-08T05:53:37.064 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-08T05:53:37.079 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-08T05:53:37.081 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi076.front.sepia.ceph.com,smithi081.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-08T05:57:46.113 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi076.front.sepia.ceph.com'), Remote(name='ubuntu@smithi081.front.sepia.ceph.com')] 2024-03-08T05:57:46.114 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-03-08T05:57:46.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:57:46.205 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-03-08T05:57:46.286 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-03-08T05:57:46.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-08T05:57:46.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:57:46.378 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-03-08T05:57:46.457 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-08T05:57:46.458 INFO:teuthology.run_tasks:Running task clock... 2024-03-08T05:57:46.468 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-08T05:57:46.468 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-08T05:57:46.469 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T05:57:46.471 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-08T05:57:46.471 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T05:57:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-08T05:57:46.517 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-08T05:57:46.529 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-08T05:57:46.546 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-08T05:57:46.547 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: ntpd: command not found 2024-03-08T05:57:46.560 INFO:teuthology.orchestra.run.smithi076.stdout:506 Cannot talk to daemon 2024-03-08T05:57:46.575 INFO:teuthology.orchestra.run.smithi081.stderr:sudo: ntpd: command not found 2024-03-08T05:57:46.576 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-08T05:57:46.588 INFO:teuthology.orchestra.run.smithi081.stdout:506 Cannot talk to daemon 2024-03-08T05:57:46.592 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-08T05:57:46.604 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-08T05:57:46.620 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-08T05:57:46.657 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-03-08T05:57:46.659 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T05:57:46.659 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-03-08T05:57:46.660 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T05:57:46.660 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T05:57:46.660 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T05:57:46.660 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T05:57:46.670 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-03-08T05:57:46.673 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T05:57:46.673 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-03-08T05:57:46.673 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T05:57:46.673 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T05:57:46.673 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T05:57:46.673 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T05:57:46.674 INFO:teuthology.run_tasks:Running task install... 2024-03-08T05:57:46.682 DEBUG:teuthology.task.install:project ceph 2024-03-08T05:57:46.682 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'}} 2024-03-08T05:57:46.682 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'} 2024-03-08T05:57:46.683 INFO:teuthology.task.install:Using flavor: default 2024-03-08T05:57:46.688 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-08T05:57:46.688 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-08T05:57:46.689 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324', 'tag': None, 'wait_for_package': False} 2024-03-08T05:57:46.689 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-03-08T05:57:46.692 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324', 'tag': None, 'wait_for_package': False} 2024-03-08T05:57:46.692 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-03-08T05:57:46.912 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/ 2024-03-08T05:57:46.912 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1670.g1fb5d106 2024-03-08T05:57:46.915 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/ 2024-03-08T05:57:46.916 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1670.g1fb5d106 2024-03-08T05:57:47.022 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-08T05:57:47.023 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T05:57:47.023 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-08T05:57:47.035 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-08T05:57:47.035 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T05:57:47.035 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-08T05:57:47.051 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-08T05:57:47.052 DEBUG:teuthology.orchestra.run.smithi076:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1fb5d10608dccee73edd6ef8a2c7b804a097c324/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-08T05:57:47.063 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-08T05:57:47.064 DEBUG:teuthology.orchestra.run.smithi081:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1fb5d10608dccee73edd6ef8a2c7b804a097c324/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-08T05:57:47.120 DEBUG:teuthology.orchestra.run.smithi076:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-08T05:57:47.132 DEBUG:teuthology.orchestra.run.smithi081:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-08T05:57:47.200 DEBUG:teuthology.orchestra.run.smithi076:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-08T05:57:47.212 DEBUG:teuthology.orchestra.run.smithi081:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-08T05:57:47.266 INFO:teuthology.orchestra.run.smithi076.stdout:check_obsoletes = 1 2024-03-08T05:57:47.268 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum clean all 2024-03-08T05:57:47.278 INFO:teuthology.orchestra.run.smithi081.stdout:check_obsoletes = 1 2024-03-08T05:57:47.280 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum clean all 2024-03-08T05:57:47.509 INFO:teuthology.orchestra.run.smithi076.stdout:74 files removed 2024-03-08T05:57:47.527 INFO:teuthology.orchestra.run.smithi081.stdout:74 files removed 2024-03-08T05:57:47.535 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-08T05:57:47.555 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-08T05:57:48.654 INFO:teuthology.orchestra.run.smithi081.stdout:ceph packages for x86_64 104 kB/s | 82 kB 00:00 2024-03-08T05:57:48.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph packages for x86_64 99 kB/s | 82 kB 00:00 2024-03-08T05:57:49.389 INFO:teuthology.orchestra.run.smithi081.stdout:ceph noarch packages 15 kB/s | 10 kB 00:00 2024-03-08T05:57:49.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph noarch packages 14 kB/s | 10 kB 00:00 2024-03-08T05:57:50.097 INFO:teuthology.orchestra.run.smithi081.stdout:ceph source packages 2.8 kB/s | 1.9 kB 00:00 2024-03-08T05:57:50.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph source packages 2.9 kB/s | 1.9 kB 00:00 2024-03-08T05:57:50.724 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-08T05:57:50.854 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - AppStream 9.7 MB/s | 6.7 MB 00:00 2024-03-08T05:57:52.255 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.0 MB 00:00 2024-03-08T05:57:52.274 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS-9 - BaseOS 4.4 MB/s | 2.0 MB 00:00 2024-03-08T05:57:53.257 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-08T05:57:53.429 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 9.5 MB/s | 8.0 MB 00:00 2024-03-08T05:57:55.683 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-03-08T05:57:56.263 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-03-08T05:57:59.851 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.1 MB 00:00 2024-03-08T05:58:00.814 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.1 MB 00:01 2024-03-08T05:58:01.410 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 78 kB/s | 15 kB 00:00 2024-03-08T05:58:02.332 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 83 kB/s | 15 kB 00:00 2024-03-08T05:58:03.910 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-08T05:58:06.570 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 4.1 MB/s | 21 MB 00:05 2024-03-08T05:58:08.866 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-08T05:58:11.419 INFO:teuthology.orchestra.run.smithi076.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T05:58:11.420 INFO:teuthology.orchestra.run.smithi076.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T05:58:11.432 INFO:teuthology.orchestra.run.smithi076.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-08T05:58:11.432 INFO:teuthology.orchestra.run.smithi076.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-08T05:58:11.433 INFO:teuthology.orchestra.run.smithi076.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-08T05:58:11.434 INFO:teuthology.orchestra.run.smithi076.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-08T05:58:11.436 INFO:teuthology.orchestra.run.smithi076.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-08T05:58:11.439 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-08T05:58:11.439 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-08T05:58:11.521 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-03-08T05:58:11.534 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================================ 2024-03-08T05:58:11.534 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-03-08T05:58:11.534 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================================ 2024-03-08T05:58:11.534 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-03-08T05:58:11.534 INFO:teuthology.orchestra.run.smithi076.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-08T05:58:11.534 INFO:teuthology.orchestra.run.smithi076.stdout: ceph x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 6.6 k 2024-03-08T05:58:11.534 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 5.3 M 2024-03-08T05:58:11.534 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 887 k 2024-03-08T05:58:11.534 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 145 k 2024-03-08T05:58:11.534 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 1.5 M 2024-03-08T05:58:11.534 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 138 k 2024-03-08T05:58:11.535 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 1.8 M 2024-03-08T05:58:11.535 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 7.4 M 2024-03-08T05:58:11.535 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 49 k 2024-03-08T05:58:11.535 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 9.8 M 2024-03-08T05:58:11.535 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 47 M 2024-03-08T05:58:11.535 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 264 k 2024-03-08T05:58:11.535 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 748 k 2024-03-08T05:58:11.535 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 220 k 2024-03-08T05:58:11.535 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-shell x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 27 k 2024-03-08T05:58:11.535 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 18 k 2024-03-08T05:58:11.535 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-08T05:58:11.535 INFO:teuthology.orchestra.run.smithi076.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-08T05:58:11.535 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-08T05:58:11.535 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 31 k 2024-03-08T05:58:11.535 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 745 k 2024-03-08T05:58:11.536 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 127 k 2024-03-08T05:58:11.536 INFO:teuthology.orchestra.run.smithi076.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-08T05:58:11.536 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-08T05:58:11.536 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-08T05:58:11.536 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-08T05:58:11.536 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-08T05:58:11.536 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-08T05:58:11.536 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 162 k 2024-03-08T05:58:11.536 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 320 k 2024-03-08T05:58:11.536 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 298 k 2024-03-08T05:58:11.536 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 99 k 2024-03-08T05:58:11.536 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 85 k 2024-03-08T05:58:11.536 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.1 M 2024-03-08T05:58:11.536 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 170 k 2024-03-08T05:58:11.537 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-08T05:58:11.537 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrading: 2024-03-08T05:58:11.537 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-08T05:58:11.537 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-08T05:58:11.537 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-08T05:58:11.537 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-08T05:58:11.537 INFO:teuthology.orchestra.run.smithi076.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-08T05:58:11.537 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-08T05:58:11.537 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-08T05:58:11.537 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-08T05:58:11.537 INFO:teuthology.orchestra.run.smithi076.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-08T05:58:11.537 INFO:teuthology.orchestra.run.smithi076.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-08T05:58:11.537 INFO:teuthology.orchestra.run.smithi076.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-08T05:58:11.537 INFO:teuthology.orchestra.run.smithi076.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-08T05:58:11.538 INFO:teuthology.orchestra.run.smithi076.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-08T05:58:11.538 INFO:teuthology.orchestra.run.smithi076.stdout: librados2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.4 M 2024-03-08T05:58:11.538 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.1 M 2024-03-08T05:58:11.538 INFO:teuthology.orchestra.run.smithi076.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-08T05:58:11.538 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-08T05:58:11.538 INFO:teuthology.orchestra.run.smithi076.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-08T05:58:11.538 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-08T05:58:11.538 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-08T05:58:11.538 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-08T05:58:11.538 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-08T05:58:11.538 INFO:teuthology.orchestra.run.smithi076.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-08T05:58:11.538 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-08T05:58:11.538 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-03-08T05:58:11.538 INFO:teuthology.orchestra.run.smithi076.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-08T05:58:11.539 INFO:teuthology.orchestra.run.smithi076.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-08T05:58:11.539 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-08T05:58:11.539 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 20 M 2024-03-08T05:58:11.539 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 31 k 2024-03-08T05:58:11.539 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 2.1 M 2024-03-08T05:58:11.539 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 248 k 2024-03-08T05:58:11.539 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 4.7 M 2024-03-08T05:58:11.539 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 17 M 2024-03-08T05:58:11.539 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 15 k 2024-03-08T05:58:11.539 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 25 k 2024-03-08T05:58:11.539 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-08T05:58:11.539 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-08T05:58:11.539 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-08T05:58:11.539 INFO:teuthology.orchestra.run.smithi076.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-08T05:58:11.540 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-08T05:58:11.540 INFO:teuthology.orchestra.run.smithi076.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-08T05:58:11.540 INFO:teuthology.orchestra.run.smithi076.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-08T05:58:11.540 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-08T05:58:11.540 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-08T05:58:11.540 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 163 k 2024-03-08T05:58:11.540 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-08T05:58:11.540 INFO:teuthology.orchestra.run.smithi076.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-08T05:58:11.540 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-08T05:58:11.540 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-08T05:58:11.540 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-08T05:58:11.540 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 501 k 2024-03-08T05:58:11.541 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-08T05:58:11.541 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 5.1 M 2024-03-08T05:58:11.541 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-08T05:58:11.541 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-08T05:58:11.541 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-08T05:58:11.541 INFO:teuthology.orchestra.run.smithi076.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-08T05:58:11.541 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-08T05:58:11.541 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-08T05:58:11.541 INFO:teuthology.orchestra.run.smithi076.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-08T05:58:11.541 INFO:teuthology.orchestra.run.smithi076.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-08T05:58:11.541 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-08T05:58:11.541 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-08T05:58:11.541 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-08T05:58:11.542 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-08T05:58:11.542 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-08T05:58:11.542 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-08T05:58:11.542 INFO:teuthology.orchestra.run.smithi076.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-08T05:58:11.542 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-08T05:58:11.542 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-08T05:58:11.542 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-08T05:58:11.542 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 45 k 2024-03-08T05:58:11.542 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 131 k 2024-03-08T05:58:11.542 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-08T05:58:11.542 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-08T05:58:11.542 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-08T05:58:11.542 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-08T05:58:11.543 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-08T05:58:11.543 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-08T05:58:11.543 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-08T05:58:11.543 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-08T05:58:11.543 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-08T05:58:11.543 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-08T05:58:11.543 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-08T05:58:11.543 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-08T05:58:11.543 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-08T05:58:11.543 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-08T05:58:11.543 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-08T05:58:11.543 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-08T05:58:11.543 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-08T05:58:11.543 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-08T05:58:11.544 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-08T05:58:11.544 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-08T05:58:11.544 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-08T05:58:11.544 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-08T05:58:11.544 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-08T05:58:11.544 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-08T05:58:11.544 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-08T05:58:11.544 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-08T05:58:11.544 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-08T05:58:11.544 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-08T05:58:11.544 INFO:teuthology.orchestra.run.smithi076.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-08T05:58:11.544 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-08T05:58:11.544 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-08T05:58:11.544 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-08T05:58:11.544 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-08T05:58:11.545 INFO:teuthology.orchestra.run.smithi076.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-08T05:58:11.545 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-08T05:58:11.545 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-08T05:58:11.545 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-08T05:58:11.545 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-08T05:58:11.545 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-08T05:58:11.545 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-08T05:58:11.545 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-08T05:58:11.545 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-08T05:58:11.545 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-08T05:58:11.545 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-08T05:58:11.545 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-08T05:58:11.545 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-08T05:58:11.545 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-08T05:58:11.545 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-08T05:58:11.545 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-08T05:58:11.546 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-08T05:58:11.546 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-08T05:58:11.546 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-08T05:58:11.546 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-08T05:58:11.546 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-08T05:58:11.546 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-08T05:58:11.546 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-08T05:58:11.546 INFO:teuthology.orchestra.run.smithi076.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-08T05:58:11.546 INFO:teuthology.orchestra.run.smithi076.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-08T05:58:11.546 INFO:teuthology.orchestra.run.smithi076.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-08T05:58:11.546 INFO:teuthology.orchestra.run.smithi076.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-08T05:58:11.546 INFO:teuthology.orchestra.run.smithi076.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-08T05:58:11.546 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-08T05:58:11.547 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-08T05:58:11.547 INFO:teuthology.orchestra.run.smithi076.stdout:Installing weak dependencies: 2024-03-08T05:58:11.547 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-08T05:58:11.547 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:58:11.547 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-03-08T05:58:11.547 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================================ 2024-03-08T05:58:11.547 INFO:teuthology.orchestra.run.smithi076.stdout:Install 150 Packages 2024-03-08T05:58:11.547 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrade 24 Packages 2024-03-08T05:58:11.547 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:58:11.547 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 223 M 2024-03-08T05:58:11.548 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-03-08T05:58:11.653 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-03-08T05:58:12.526 INFO:teuthology.orchestra.run.smithi076.stdout:(1/174): ceph-19.0.0-1670.g1fb5d106.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-03-08T05:58:13.117 INFO:teuthology.orchestra.run.smithi076.stdout:(2/174): ceph-base-19.0.0-1670.g1fb5d106.el9.x8 5.1 MB/s | 5.3 MB 00:01 2024-03-08T05:58:13.184 INFO:teuthology.orchestra.run.smithi076.stdout:(3/174): ceph-immutable-object-cache-19.0.0-167 2.1 MB/s | 145 kB 00:00 2024-03-08T05:58:13.276 INFO:teuthology.orchestra.run.smithi076.stdout:(4/174): ceph-fuse-19.0.0-1670.g1fb5d106.el9.x8 1.2 MB/s | 887 kB 00:00 2024-03-08T05:58:13.401 INFO:teuthology.orchestra.run.smithi076.stdout:(5/174): ceph-mds-19.0.0-1670.g1fb5d106.el9.x86 9.8 MB/s | 2.1 MB 00:00 2024-03-08T05:58:13.760 INFO:teuthology.orchestra.run.smithi076.stdout:(6/174): ceph-mon-19.0.0-1670.g1fb5d106.el9.x86 13 MB/s | 4.7 MB 00:00 2024-03-08T05:58:14.035 INFO:teuthology.orchestra.run.smithi076.stdout:(7/174): ceph-mgr-19.0.0-1670.g1fb5d106.el9.x86 2.0 MB/s | 1.5 MB 00:00 2024-03-08T05:58:14.201 INFO:teuthology.orchestra.run.smithi081.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T05:58:14.202 INFO:teuthology.orchestra.run.smithi081.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T05:58:14.216 INFO:teuthology.orchestra.run.smithi081.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-08T05:58:14.217 INFO:teuthology.orchestra.run.smithi081.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-08T05:58:14.218 INFO:teuthology.orchestra.run.smithi081.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-08T05:58:14.219 INFO:teuthology.orchestra.run.smithi081.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-08T05:58:14.221 INFO:teuthology.orchestra.run.smithi081.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-08T05:58:14.224 INFO:teuthology.orchestra.run.smithi081.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-08T05:58:14.225 INFO:teuthology.orchestra.run.smithi081.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-08T05:58:14.305 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-08T05:58:14.320 INFO:teuthology.orchestra.run.smithi081.stdout:============================================================================================ 2024-03-08T05:58:14.320 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-03-08T05:58:14.320 INFO:teuthology.orchestra.run.smithi081.stdout:============================================================================================ 2024-03-08T05:58:14.320 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-08T05:58:14.320 INFO:teuthology.orchestra.run.smithi081.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-08T05:58:14.320 INFO:teuthology.orchestra.run.smithi081.stdout: ceph x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 6.6 k 2024-03-08T05:58:14.320 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 5.3 M 2024-03-08T05:58:14.320 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 887 k 2024-03-08T05:58:14.320 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 145 k 2024-03-08T05:58:14.321 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 1.5 M 2024-03-08T05:58:14.321 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 138 k 2024-03-08T05:58:14.321 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 1.8 M 2024-03-08T05:58:14.321 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 7.4 M 2024-03-08T05:58:14.321 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 49 k 2024-03-08T05:58:14.321 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 9.8 M 2024-03-08T05:58:14.321 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 47 M 2024-03-08T05:58:14.321 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 264 k 2024-03-08T05:58:14.321 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 748 k 2024-03-08T05:58:14.321 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 220 k 2024-03-08T05:58:14.321 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-shell x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 27 k 2024-03-08T05:58:14.321 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-top noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 18 k 2024-03-08T05:58:14.321 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-08T05:58:14.321 INFO:teuthology.orchestra.run.smithi081.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-08T05:58:14.322 INFO:teuthology.orchestra.run.smithi081.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-08T05:58:14.322 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 31 k 2024-03-08T05:58:14.322 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 745 k 2024-03-08T05:58:14.322 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 127 k 2024-03-08T05:58:14.322 INFO:teuthology.orchestra.run.smithi081.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-08T05:58:14.322 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-08T05:58:14.322 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-08T05:58:14.322 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-08T05:58:14.322 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-08T05:58:14.322 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-08T05:58:14.322 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 162 k 2024-03-08T05:58:14.322 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 320 k 2024-03-08T05:58:14.322 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 298 k 2024-03-08T05:58:14.322 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 99 k 2024-03-08T05:58:14.322 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 85 k 2024-03-08T05:58:14.323 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.1 M 2024-03-08T05:58:14.323 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 170 k 2024-03-08T05:58:14.323 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-08T05:58:14.323 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrading: 2024-03-08T05:58:14.323 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-08T05:58:14.323 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-08T05:58:14.323 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-08T05:58:14.323 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-08T05:58:14.323 INFO:teuthology.orchestra.run.smithi081.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-08T05:58:14.323 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-08T05:58:14.323 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-08T05:58:14.323 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-08T05:58:14.323 INFO:teuthology.orchestra.run.smithi081.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-08T05:58:14.323 INFO:teuthology.orchestra.run.smithi081.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-08T05:58:14.324 INFO:teuthology.orchestra.run.smithi081.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-08T05:58:14.324 INFO:teuthology.orchestra.run.smithi081.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-08T05:58:14.324 INFO:teuthology.orchestra.run.smithi081.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-08T05:58:14.324 INFO:teuthology.orchestra.run.smithi081.stdout: librados2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.4 M 2024-03-08T05:58:14.324 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.1 M 2024-03-08T05:58:14.324 INFO:teuthology.orchestra.run.smithi081.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-08T05:58:14.324 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-08T05:58:14.324 INFO:teuthology.orchestra.run.smithi081.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-08T05:58:14.324 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-08T05:58:14.324 INFO:teuthology.orchestra.run.smithi081.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-08T05:58:14.324 INFO:teuthology.orchestra.run.smithi081.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-08T05:58:14.324 INFO:teuthology.orchestra.run.smithi081.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-08T05:58:14.324 INFO:teuthology.orchestra.run.smithi081.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-08T05:58:14.324 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-08T05:58:14.324 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-03-08T05:58:14.325 INFO:teuthology.orchestra.run.smithi081.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-08T05:58:14.325 INFO:teuthology.orchestra.run.smithi081.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-08T05:58:14.325 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-08T05:58:14.325 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 20 M 2024-03-08T05:58:14.325 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 31 k 2024-03-08T05:58:14.325 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 2.1 M 2024-03-08T05:58:14.325 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 248 k 2024-03-08T05:58:14.325 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 4.7 M 2024-03-08T05:58:14.325 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 17 M 2024-03-08T05:58:14.325 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 15 k 2024-03-08T05:58:14.325 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 25 k 2024-03-08T05:58:14.325 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-08T05:58:14.325 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-08T05:58:14.326 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-08T05:58:14.326 INFO:teuthology.orchestra.run.smithi081.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-08T05:58:14.326 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-08T05:58:14.326 INFO:teuthology.orchestra.run.smithi081.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-08T05:58:14.326 INFO:teuthology.orchestra.run.smithi081.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-08T05:58:14.326 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-08T05:58:14.326 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-08T05:58:14.326 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 163 k 2024-03-08T05:58:14.326 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-08T05:58:14.326 INFO:teuthology.orchestra.run.smithi081.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-08T05:58:14.326 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-08T05:58:14.326 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-08T05:58:14.326 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-08T05:58:14.327 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 501 k 2024-03-08T05:58:14.327 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-08T05:58:14.327 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 5.1 M 2024-03-08T05:58:14.327 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-08T05:58:14.327 INFO:teuthology.orchestra.run.smithi081.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-08T05:58:14.327 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-08T05:58:14.327 INFO:teuthology.orchestra.run.smithi081.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-08T05:58:14.327 INFO:teuthology.orchestra.run.smithi081.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-08T05:58:14.327 INFO:teuthology.orchestra.run.smithi081.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-08T05:58:14.327 INFO:teuthology.orchestra.run.smithi081.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-08T05:58:14.327 INFO:teuthology.orchestra.run.smithi081.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-08T05:58:14.327 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-08T05:58:14.327 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-08T05:58:14.327 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-08T05:58:14.328 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-08T05:58:14.328 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-08T05:58:14.328 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-08T05:58:14.328 INFO:teuthology.orchestra.run.smithi081.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-08T05:58:14.328 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-08T05:58:14.328 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-08T05:58:14.328 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-08T05:58:14.328 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 45 k 2024-03-08T05:58:14.328 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 131 k 2024-03-08T05:58:14.328 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-08T05:58:14.328 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-08T05:58:14.328 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-08T05:58:14.328 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-08T05:58:14.329 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-08T05:58:14.329 INFO:teuthology.orchestra.run.smithi081.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-08T05:58:14.329 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-08T05:58:14.329 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-08T05:58:14.329 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-08T05:58:14.329 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-08T05:58:14.329 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-08T05:58:14.329 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-08T05:58:14.329 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-08T05:58:14.329 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-08T05:58:14.329 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-08T05:58:14.329 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-08T05:58:14.329 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-08T05:58:14.329 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-08T05:58:14.329 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-08T05:58:14.330 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-08T05:58:14.330 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-08T05:58:14.330 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-08T05:58:14.330 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-08T05:58:14.330 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-08T05:58:14.330 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-08T05:58:14.330 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-08T05:58:14.330 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-08T05:58:14.330 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-08T05:58:14.330 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-08T05:58:14.330 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-08T05:58:14.330 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-08T05:58:14.331 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-08T05:58:14.331 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-08T05:58:14.331 INFO:teuthology.orchestra.run.smithi081.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-08T05:58:14.331 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-08T05:58:14.331 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-08T05:58:14.331 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-08T05:58:14.331 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-08T05:58:14.331 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-08T05:58:14.331 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-08T05:58:14.331 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-08T05:58:14.331 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-08T05:58:14.331 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-08T05:58:14.331 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-08T05:58:14.331 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-08T05:58:14.331 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-08T05:58:14.332 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-08T05:58:14.332 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-08T05:58:14.332 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-08T05:58:14.332 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-08T05:58:14.332 INFO:teuthology.orchestra.run.smithi081.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-08T05:58:14.332 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-08T05:58:14.332 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-08T05:58:14.332 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-08T05:58:14.332 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-08T05:58:14.332 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-08T05:58:14.332 INFO:teuthology.orchestra.run.smithi081.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-08T05:58:14.332 INFO:teuthology.orchestra.run.smithi081.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-08T05:58:14.332 INFO:teuthology.orchestra.run.smithi081.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-08T05:58:14.332 INFO:teuthology.orchestra.run.smithi081.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-08T05:58:14.332 INFO:teuthology.orchestra.run.smithi081.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-08T05:58:14.333 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-08T05:58:14.333 INFO:teuthology.orchestra.run.smithi081.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-08T05:58:14.333 INFO:teuthology.orchestra.run.smithi081.stdout:Installing weak dependencies: 2024-03-08T05:58:14.333 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-08T05:58:14.333 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:58:14.333 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-08T05:58:14.333 INFO:teuthology.orchestra.run.smithi081.stdout:============================================================================================ 2024-03-08T05:58:14.333 INFO:teuthology.orchestra.run.smithi081.stdout:Install 150 Packages 2024-03-08T05:58:14.333 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrade 24 Packages 2024-03-08T05:58:14.333 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:58:14.334 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 223 M 2024-03-08T05:58:14.334 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-08T05:58:14.911 INFO:teuthology.orchestra.run.smithi076.stdout:(8/174): ceph-osd-19.0.0-1670.g1fb5d106.el9.x86 15 MB/s | 17 MB 00:01 2024-03-08T05:58:14.977 INFO:teuthology.orchestra.run.smithi076.stdout:(9/174): ceph-selinux-19.0.0-1670.g1fb5d106.el9 379 kB/s | 25 kB 00:00 2024-03-08T05:58:15.088 INFO:teuthology.orchestra.run.smithi081.stdout:(1/174): ceph-19.0.0-1670.g1fb5d106.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-03-08T05:58:15.186 INFO:teuthology.orchestra.run.smithi076.stdout:(10/174): ceph-common-19.0.0-1670.g1fb5d106.el9 6.4 MB/s | 20 MB 00:03 2024-03-08T05:58:15.261 INFO:teuthology.orchestra.run.smithi076.stdout:(11/174): cephfs-mirror-19.0.0-1670.g1fb5d106.e 2.9 MB/s | 220 kB 00:00 2024-03-08T05:58:15.345 INFO:teuthology.orchestra.run.smithi076.stdout:(12/174): cephfs-shell-19.0.0-1670.g1fb5d106.el 325 kB/s | 27 kB 00:00 2024-03-08T05:58:15.420 INFO:teuthology.orchestra.run.smithi076.stdout:(13/174): libcephfs-devel-19.0.0-1670.g1fb5d106 416 kB/s | 31 kB 00:00 2024-03-08T05:58:15.595 INFO:teuthology.orchestra.run.smithi076.stdout:(14/174): libcephfs2-19.0.0-1670.g1fb5d106.el9. 4.2 MB/s | 745 kB 00:00 2024-03-08T05:58:15.679 INFO:teuthology.orchestra.run.smithi076.stdout:(15/174): libcephsqlite-19.0.0-1670.g1fb5d106.e 1.9 MB/s | 163 kB 00:00 2024-03-08T05:58:15.754 INFO:teuthology.orchestra.run.smithi076.stdout:(16/174): librados-devel-19.0.0-1670.g1fb5d106. 1.7 MB/s | 127 kB 00:00 2024-03-08T05:58:15.813 INFO:teuthology.orchestra.run.smithi081.stdout:(2/174): ceph-fuse-19.0.0-1670.g1fb5d106.el9.x8 1.2 MB/s | 887 kB 00:00 2024-03-08T05:58:15.846 INFO:teuthology.orchestra.run.smithi076.stdout:(17/174): libradosstriper1-19.0.0-1670.g1fb5d10 5.4 MB/s | 501 kB 00:00 2024-03-08T05:58:15.947 INFO:teuthology.orchestra.run.smithi081.stdout:(3/174): ceph-immutable-object-cache-19.0.0-167 1.1 MB/s | 145 kB 00:00 2024-03-08T05:58:16.263 INFO:teuthology.orchestra.run.smithi076.stdout:(18/174): ceph-radosgw-19.0.0-1670.g1fb5d106.el 4.4 MB/s | 9.8 MB 00:02 2024-03-08T05:58:16.363 INFO:teuthology.orchestra.run.smithi076.stdout:(19/174): python3-ceph-argparse-19.0.0-1670.g1f 453 kB/s | 45 kB 00:00 2024-03-08T05:58:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:(20/174): librgw2-19.0.0-1670.g1fb5d106.el9.x86 7.8 MB/s | 5.1 MB 00:00 2024-03-08T05:58:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:(21/174): python3-ceph-common-19.0.0-1670.g1fb5 788 kB/s | 131 kB 00:00 2024-03-08T05:58:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:(22/174): python3-cephfs-19.0.0-1670.g1fb5d106. 2.1 MB/s | 162 kB 00:00 2024-03-08T05:58:16.639 INFO:teuthology.orchestra.run.smithi076.stdout:(23/174): python3-rados-19.0.0-1670.g1fb5d106.e 2.9 MB/s | 320 kB 00:00 2024-03-08T05:58:16.690 INFO:teuthology.orchestra.run.smithi076.stdout:(24/174): python3-rbd-19.0.0-1670.g1fb5d106.el9 2.7 MB/s | 298 kB 00:00 2024-03-08T05:58:16.756 INFO:teuthology.orchestra.run.smithi076.stdout:(25/174): python3-rgw-19.0.0-1670.g1fb5d106.el9 850 kB/s | 99 kB 00:00 2024-03-08T05:58:16.780 INFO:teuthology.orchestra.run.smithi081.stdout:(4/174): ceph-mds-19.0.0-1670.g1fb5d106.el9.x86 2.6 MB/s | 2.1 MB 00:00 2024-03-08T05:58:16.782 INFO:teuthology.orchestra.run.smithi076.stdout:(26/174): rbd-fuse-19.0.0-1670.g1fb5d106.el9.x8 929 kB/s | 85 kB 00:00 2024-03-08T05:58:16.874 INFO:teuthology.orchestra.run.smithi076.stdout:(27/174): rbd-nbd-19.0.0-1670.g1fb5d106.el9.x86 1.8 MB/s | 170 kB 00:00 2024-03-08T05:58:16.966 INFO:teuthology.orchestra.run.smithi076.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1670.g 343 kB/s | 31 kB 00:00 2024-03-08T05:58:17.058 INFO:teuthology.orchestra.run.smithi076.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1670.g1fb5d10 1.5 MB/s | 138 kB 00:00 2024-03-08T05:58:17.297 INFO:teuthology.orchestra.run.smithi081.stdout:(5/174): ceph-mgr-19.0.0-1670.g1fb5d106.el9.x86 2.9 MB/s | 1.5 MB 00:00 2024-03-08T05:58:17.375 INFO:teuthology.orchestra.run.smithi076.stdout:(30/174): ceph-mgr-dashboard-19.0.0-1670.g1fb5d 5.6 MB/s | 1.8 MB 00:00 2024-03-08T05:58:17.517 INFO:teuthology.orchestra.run.smithi076.stdout:(31/174): rbd-mirror-19.0.0-1670.g1fb5d106.el9. 4.1 MB/s | 3.1 MB 00:00 2024-03-08T05:58:17.617 INFO:teuthology.orchestra.run.smithi076.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1670.g1f 2.4 MB/s | 248 kB 00:00 2024-03-08T05:58:17.700 INFO:teuthology.orchestra.run.smithi076.stdout:(33/174): ceph-mgr-rook-19.0.0-1670.g1fb5d106.e 593 kB/s | 49 kB 00:00 2024-03-08T05:58:17.784 INFO:teuthology.orchestra.run.smithi076.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1670.g1 184 kB/s | 15 kB 00:00 2024-03-08T05:58:17.884 INFO:teuthology.orchestra.run.smithi076.stdout:(35/174): ceph-volume-19.0.0-1670.g1fb5d106.el9 2.6 MB/s | 264 kB 00:00 2024-03-08T05:58:18.143 INFO:teuthology.orchestra.run.smithi076.stdout:(36/174): cephadm-19.0.0-1670.g1fb5d106.el9.noa 2.8 MB/s | 748 kB 00:00 2024-03-08T05:58:18.235 INFO:teuthology.orchestra.run.smithi076.stdout:(37/174): cephfs-top-19.0.0-1670.g1fb5d106.el9. 199 kB/s | 18 kB 00:00 2024-03-08T05:58:18.394 INFO:teuthology.orchestra.run.smithi076.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 7.3 MB/s | 7.4 MB 00:01 2024-03-08T05:58:18.444 INFO:teuthology.orchestra.run.smithi076.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 662 kB 00:00 2024-03-08T05:58:18.464 INFO:teuthology.orchestra.run.smithi081.stdout:(6/174): ceph-mon-19.0.0-1670.g1fb5d106.el9.x86 4.0 MB/s | 4.7 MB 00:01 2024-03-08T05:58:18.486 INFO:teuthology.orchestra.run.smithi076.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 9.9 MB/s | 920 kB 00:00 2024-03-08T05:58:18.511 INFO:teuthology.orchestra.run.smithi076.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-08T05:58:18.537 INFO:teuthology.orchestra.run.smithi076.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-08T05:58:18.562 INFO:teuthology.orchestra.run.smithi076.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 638 kB/s | 32 kB 00:00 2024-03-08T05:58:18.740 INFO:teuthology.orchestra.run.smithi081.stdout:(7/174): ceph-base-19.0.0-1670.g1fb5d106.el9.x8 1.4 MB/s | 5.3 MB 00:03 2024-03-08T05:58:19.029 INFO:teuthology.orchestra.run.smithi076.stdout:(44/174): ceph-test-19.0.0-1670.g1fb5d106.el9.x 12 MB/s | 47 MB 00:04 2024-03-08T05:58:19.087 INFO:teuthology.orchestra.run.smithi076.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.4 MB/s | 3.0 MB 00:00 2024-03-08T05:58:19.106 INFO:teuthology.orchestra.run.smithi076.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 32 kB/s | 17 kB 00:00 2024-03-08T05:58:19.121 INFO:teuthology.orchestra.run.smithi076.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 112 kB/s | 10 kB 00:00 2024-03-08T05:58:19.138 INFO:teuthology.orchestra.run.smithi076.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-08T05:58:19.163 INFO:teuthology.orchestra.run.smithi076.stdout:(49/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.6 MB/s | 159 kB 00:00 2024-03-08T05:58:19.189 INFO:teuthology.orchestra.run.smithi076.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-03-08T05:58:19.213 INFO:teuthology.orchestra.run.smithi076.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-03-08T05:58:19.230 INFO:teuthology.orchestra.run.smithi076.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 711 kB/s | 46 kB 00:00 2024-03-08T05:58:19.264 INFO:teuthology.orchestra.run.smithi076.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-08T05:58:19.289 INFO:teuthology.orchestra.run.smithi076.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-08T05:58:19.323 INFO:teuthology.orchestra.run.smithi076.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.3 MB/s | 217 kB 00:00 2024-03-08T05:58:19.340 INFO:teuthology.orchestra.run.smithi076.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 458 kB/s | 34 kB 00:00 2024-03-08T05:58:19.432 INFO:teuthology.orchestra.run.smithi076.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 14 MB/s | 1.5 MB 00:00 2024-03-08T05:58:19.524 INFO:teuthology.orchestra.run.smithi076.stdout:(58/174): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-03-08T05:58:19.565 INFO:teuthology.orchestra.run.smithi076.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 3.5 MB/s | 811 kB 00:00 2024-03-08T05:58:19.591 INFO:teuthology.orchestra.run.smithi076.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 857 kB/s | 134 kB 00:00 2024-03-08T05:58:19.699 INFO:teuthology.orchestra.run.smithi076.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2024-03-08T05:58:19.724 INFO:teuthology.orchestra.run.smithi076.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 595 kB/s | 78 kB 00:00 2024-03-08T05:58:19.766 INFO:teuthology.orchestra.run.smithi076.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-03-08T05:58:19.892 INFO:teuthology.orchestra.run.smithi076.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 5.8 MB 00:00 2024-03-08T05:58:19.909 INFO:teuthology.orchestra.run.smithi076.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 136 kB/s | 24 kB 00:00 2024-03-08T05:58:19.934 INFO:teuthology.orchestra.run.smithi076.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 110 kB/s | 18 kB 00:00 2024-03-08T05:58:19.976 INFO:teuthology.orchestra.run.smithi076.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-03-08T05:58:20.001 INFO:teuthology.orchestra.run.smithi076.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 474 kB/s | 32 kB 00:00 2024-03-08T05:58:20.093 INFO:teuthology.orchestra.run.smithi076.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2024-03-08T05:58:20.126 INFO:teuthology.orchestra.run.smithi076.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-03-08T05:58:20.151 INFO:teuthology.orchestra.run.smithi076.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-03-08T05:58:20.177 INFO:teuthology.orchestra.run.smithi076.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 828 kB/s | 69 kB 00:00 2024-03-08T05:58:20.210 INFO:teuthology.orchestra.run.smithi076.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 238 kB 00:00 2024-03-08T05:58:20.227 INFO:teuthology.orchestra.run.smithi076.stdout:(74/174): python3-prettytable-0.7.2-27.el9.noar 547 kB/s | 41 kB 00:00 2024-03-08T05:58:20.239 INFO:teuthology.orchestra.run.smithi081.stdout:(8/174): ceph-osd-19.0.0-1670.g1fb5d106.el9.x86 9.5 MB/s | 17 MB 00:01 2024-03-08T05:58:20.261 INFO:teuthology.orchestra.run.smithi076.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-03-08T05:58:20.281 INFO:teuthology.orchestra.run.smithi081.stdout:(9/174): ceph-selinux-19.0.0-1670.g1fb5d106.el9 607 kB/s | 25 kB 00:00 2024-03-08T05:58:20.286 INFO:teuthology.orchestra.run.smithi076.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-08T05:58:20.303 INFO:teuthology.orchestra.run.smithi076.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-08T05:58:20.320 INFO:teuthology.orchestra.run.smithi076.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 738 kB/s | 43 kB 00:00 2024-03-08T05:58:20.437 INFO:teuthology.orchestra.run.smithi076.stdout:(79/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 359 kB/s | 41 kB 00:00 2024-03-08T05:58:20.454 INFO:teuthology.orchestra.run.smithi076.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 295 kB/s | 39 kB 00:00 2024-03-08T05:58:20.638 INFO:teuthology.orchestra.run.smithi076.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-03-08T05:58:20.671 INFO:teuthology.orchestra.run.smithi076.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 300 kB 00:00 2024-03-08T05:58:20.696 INFO:teuthology.orchestra.run.smithi076.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 235 kB/s | 56 kB 00:00 2024-03-08T05:58:20.722 INFO:teuthology.orchestra.run.smithi076.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 755 kB/s | 63 kB 00:00 2024-03-08T05:58:20.751 INFO:teuthology.orchestra.run.smithi076.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 878 kB/s | 44 kB 00:00 2024-03-08T05:58:20.751 INFO:teuthology.orchestra.run.smithi081.stdout:(10/174): ceph-radosgw-19.0.0-1670.g1fb5d106.el 4.9 MB/s | 9.8 MB 00:02 2024-03-08T05:58:20.772 INFO:teuthology.orchestra.run.smithi076.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-08T05:58:20.797 INFO:teuthology.orchestra.run.smithi076.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-08T05:58:20.822 INFO:teuthology.orchestra.run.smithi076.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-03-08T05:58:20.823 INFO:teuthology.orchestra.run.smithi081.stdout:(11/174): cephfs-mirror-19.0.0-1670.g1fb5d106.e 2.9 MB/s | 220 kB 00:00 2024-03-08T05:58:20.856 INFO:teuthology.orchestra.run.smithi076.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-08T05:58:20.865 INFO:teuthology.orchestra.run.smithi081.stdout:(12/174): cephfs-shell-19.0.0-1670.g1fb5d106.el 652 kB/s | 27 kB 00:00 2024-03-08T05:58:20.881 INFO:teuthology.orchestra.run.smithi076.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 124 kB 00:00 2024-03-08T05:58:20.915 INFO:teuthology.orchestra.run.smithi076.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2024-03-08T05:58:20.916 INFO:teuthology.orchestra.run.smithi081.stdout:(13/174): libcephfs-devel-19.0.0-1670.g1fb5d106 622 kB/s | 31 kB 00:00 2024-03-08T05:58:21.049 INFO:teuthology.orchestra.run.smithi076.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-03-08T05:58:21.074 INFO:teuthology.orchestra.run.smithi076.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-03-08T05:58:21.099 INFO:teuthology.orchestra.run.smithi076.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-08T05:58:21.132 INFO:teuthology.orchestra.run.smithi081.stdout:(14/174): libcephfs2-19.0.0-1670.g1fb5d106.el9. 3.4 MB/s | 745 kB 00:00 2024-03-08T05:58:21.133 INFO:teuthology.orchestra.run.smithi076.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.1 MB/s | 803 kB 00:00 2024-03-08T05:58:21.183 INFO:teuthology.orchestra.run.smithi076.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 197 kB/s | 21 kB 00:00 2024-03-08T05:58:21.208 INFO:teuthology.orchestra.run.smithi076.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 234 kB/s | 25 kB 00:00 2024-03-08T05:58:21.250 INFO:teuthology.orchestra.run.smithi081.stdout:(15/174): libcephsqlite-19.0.0-1670.g1fb5d106.e 1.4 MB/s | 163 kB 00:00 2024-03-08T05:58:21.325 INFO:teuthology.orchestra.run.smithi076.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-03-08T05:58:21.341 INFO:teuthology.orchestra.run.smithi081.stdout:(16/174): librados-devel-19.0.0-1670.g1fb5d106. 1.4 MB/s | 127 kB 00:00 2024-03-08T05:58:21.361 INFO:teuthology.orchestra.run.smithi076.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-03-08T05:58:21.384 INFO:teuthology.orchestra.run.smithi076.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 3.6 MB/s | 216 kB 00:00 2024-03-08T05:58:21.409 INFO:teuthology.orchestra.run.smithi076.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 695 kB/s | 35 kB 00:00 2024-03-08T05:58:21.434 INFO:teuthology.orchestra.run.smithi076.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-08T05:58:21.459 INFO:teuthology.orchestra.run.smithi076.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-08T05:58:21.485 INFO:teuthology.orchestra.run.smithi076.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-03-08T05:58:21.600 INFO:teuthology.orchestra.run.smithi081.stdout:(17/174): libradosstriper1-19.0.0-1670.g1fb5d10 1.9 MB/s | 501 kB 00:00 2024-03-08T05:58:21.635 INFO:teuthology.orchestra.run.smithi076.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-03-08T05:58:21.887 INFO:teuthology.orchestra.run.smithi076.stdout:(106/174): openssl-devel-3.0.7-27.el9.x86_64.rp 5.5 MB/s | 4.1 MB 00:00 2024-03-08T05:58:22.344 INFO:teuthology.orchestra.run.smithi076.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 43 kB/s | 19 kB 00:00 2024-03-08T05:58:22.511 INFO:teuthology.orchestra.run.smithi076.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 353 kB/s | 309 kB 00:00 2024-03-08T05:58:22.570 INFO:teuthology.orchestra.run.smithi076.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 505 kB/s | 548 kB 00:01 2024-03-08T05:58:22.611 INFO:teuthology.orchestra.run.smithi076.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-03-08T05:58:22.679 INFO:teuthology.orchestra.run.smithi076.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 447 kB/s | 49 kB 00:00 2024-03-08T05:58:22.720 INFO:teuthology.orchestra.run.smithi076.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 621 kB/s | 67 kB 00:00 2024-03-08T05:58:22.946 INFO:teuthology.orchestra.run.smithi076.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 566 kB/s | 151 kB 00:00 2024-03-08T05:58:23.501 INFO:teuthology.orchestra.run.smithi081.stdout:(18/174): ceph-common-19.0.0-1670.g1fb5d106.el9 2.3 MB/s | 20 MB 00:08 2024-03-08T05:58:23.576 INFO:teuthology.orchestra.run.smithi081.stdout:(19/174): librgw2-19.0.0-1670.g1fb5d106.el9.x86 2.6 MB/s | 5.1 MB 00:01 2024-03-08T05:58:23.592 INFO:teuthology.orchestra.run.smithi081.stdout:(20/174): python3-ceph-argparse-19.0.0-1670.g1f 493 kB/s | 45 kB 00:00 2024-03-08T05:58:23.668 INFO:teuthology.orchestra.run.smithi081.stdout:(21/174): python3-ceph-common-19.0.0-1670.g1fb5 1.4 MB/s | 131 kB 00:00 2024-03-08T05:58:23.687 INFO:teuthology.orchestra.run.smithi076.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 738 kB/s | 548 kB 00:00 2024-03-08T05:58:23.760 INFO:teuthology.orchestra.run.smithi081.stdout:(22/174): python3-cephfs-19.0.0-1670.g1fb5d106. 969 kB/s | 162 kB 00:00 2024-03-08T05:58:23.814 INFO:teuthology.orchestra.run.smithi076.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 767 kB/s | 838 kB 00:01 2024-03-08T05:58:23.826 INFO:teuthology.orchestra.run.smithi081.stdout:(23/174): python3-rados-19.0.0-1670.g1fb5d106.e 2.0 MB/s | 320 kB 00:00 2024-03-08T05:58:23.838 INFO:teuthology.orchestra.run.smithi076.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 289 kB/s | 43 kB 00:00 2024-03-08T05:58:23.919 INFO:teuthology.orchestra.run.smithi081.stdout:(24/174): python3-rgw-19.0.0-1670.g1fb5d106.el9 1.1 MB/s | 99 kB 00:00 2024-03-08T05:58:23.922 INFO:teuthology.orchestra.run.smithi076.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-03-08T05:58:23.944 INFO:teuthology.orchestra.run.smithi081.stdout:(25/174): python3-rbd-19.0.0-1670.g1fb5d106.el9 1.6 MB/s | 298 kB 00:00 2024-03-08T05:58:23.947 INFO:teuthology.orchestra.run.smithi076.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-03-08T05:58:24.011 INFO:teuthology.orchestra.run.smithi081.stdout:(26/174): rbd-fuse-19.0.0-1670.g1fb5d106.el9.x8 929 kB/s | 85 kB 00:00 2024-03-08T05:58:24.103 INFO:teuthology.orchestra.run.smithi081.stdout:(27/174): rbd-nbd-19.0.0-1670.g1fb5d106.el9.x86 1.8 MB/s | 170 kB 00:00 2024-03-08T05:58:24.203 INFO:teuthology.orchestra.run.smithi081.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1670.g 314 kB/s | 31 kB 00:00 2024-03-08T05:58:24.281 INFO:teuthology.orchestra.run.smithi076.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 479 kB/s | 172 kB 00:00 2024-03-08T05:58:24.295 INFO:teuthology.orchestra.run.smithi081.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1670.g1fb5d10 1.5 MB/s | 138 kB 00:00 2024-03-08T05:58:24.489 INFO:teuthology.orchestra.run.smithi076.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 661 kB/s | 358 kB 00:00 2024-03-08T05:58:24.598 INFO:teuthology.orchestra.run.smithi076.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 326 kB/s | 35 kB 00:00 2024-03-08T05:58:24.654 INFO:teuthology.orchestra.run.smithi081.stdout:(30/174): ceph-mgr-dashboard-19.0.0-1670.g1fb5d 5.0 MB/s | 1.8 MB 00:00 2024-03-08T05:58:24.732 INFO:teuthology.orchestra.run.smithi076.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 673 kB/s | 303 kB 00:00 2024-03-08T05:58:24.779 INFO:teuthology.orchestra.run.smithi081.stdout:(31/174): rbd-mirror-19.0.0-1670.g1fb5d106.el9. 3.7 MB/s | 3.1 MB 00:00 2024-03-08T05:58:24.855 INFO:teuthology.orchestra.run.smithi081.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1670.g1f 3.2 MB/s | 248 kB 00:00 2024-03-08T05:58:24.865 INFO:teuthology.orchestra.run.smithi076.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 760 kB/s | 203 kB 00:00 2024-03-08T05:58:24.905 INFO:teuthology.orchestra.run.smithi081.stdout:(33/174): ceph-mgr-rook-19.0.0-1670.g1fb5d106.e 987 kB/s | 49 kB 00:00 2024-03-08T05:58:24.955 INFO:teuthology.orchestra.run.smithi081.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1670.g1 307 kB/s | 15 kB 00:00 2024-03-08T05:58:25.047 INFO:teuthology.orchestra.run.smithi081.stdout:(35/174): ceph-volume-19.0.0-1670.g1fb5d106.el9 2.8 MB/s | 264 kB 00:00 2024-03-08T05:58:25.132 INFO:teuthology.orchestra.run.smithi076.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 542 kB/s | 144 kB 00:00 2024-03-08T05:58:25.206 INFO:teuthology.orchestra.run.smithi081.stdout:(36/174): cephadm-19.0.0-1670.g1fb5d106.el9.noa 4.6 MB/s | 748 kB 00:00 2024-03-08T05:58:25.241 INFO:teuthology.orchestra.run.smithi076.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-03-08T05:58:25.248 INFO:teuthology.orchestra.run.smithi081.stdout:(37/174): cephfs-top-19.0.0-1670.g1fb5d106.el9. 439 kB/s | 18 kB 00:00 2024-03-08T05:58:25.349 INFO:teuthology.orchestra.run.smithi076.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-03-08T05:58:25.382 INFO:teuthology.orchestra.run.smithi081.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.9 MB/s | 662 kB 00:00 2024-03-08T05:58:25.432 INFO:teuthology.orchestra.run.smithi081.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 920 kB 00:00 2024-03-08T05:58:25.465 INFO:teuthology.orchestra.run.smithi081.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-03-08T05:58:25.466 INFO:teuthology.orchestra.run.smithi076.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2024-03-08T05:58:25.507 INFO:teuthology.orchestra.run.smithi081.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-03-08T05:58:25.541 INFO:teuthology.orchestra.run.smithi081.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 947 kB/s | 32 kB 00:00 2024-03-08T05:58:25.566 INFO:teuthology.orchestra.run.smithi076.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 195 kB/s | 19 kB 00:00 2024-03-08T05:58:25.641 INFO:teuthology.orchestra.run.smithi081.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-08T05:58:25.675 INFO:teuthology.orchestra.run.smithi081.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 513 kB/s | 17 kB 00:00 2024-03-08T05:58:25.692 INFO:teuthology.orchestra.run.smithi076.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 157 kB/s | 20 kB 00:00 2024-03-08T05:58:25.717 INFO:teuthology.orchestra.run.smithi081.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 241 kB/s | 10 kB 00:00 2024-03-08T05:58:25.751 INFO:teuthology.orchestra.run.smithi081.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.6 MB/s | 159 kB 00:00 2024-03-08T05:58:25.784 INFO:teuthology.orchestra.run.smithi081.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-08T05:58:25.868 INFO:teuthology.orchestra.run.smithi081.stdout:(48/174): ceph-mgr-diskprediction-local-19.0.0- 6.1 MB/s | 7.4 MB 00:01 2024-03-08T05:58:25.893 INFO:teuthology.orchestra.run.smithi081.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 5.9 MB/s | 661 kB 00:00 2024-03-08T05:58:25.918 INFO:teuthology.orchestra.run.smithi081.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2024-03-08T05:58:25.944 INFO:teuthology.orchestra.run.smithi081.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 939 kB/s | 46 kB 00:00 2024-03-08T05:58:25.969 INFO:teuthology.orchestra.run.smithi081.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-08T05:58:25.994 INFO:teuthology.orchestra.run.smithi081.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-08T05:58:26.020 INFO:teuthology.orchestra.run.smithi081.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-08T05:58:26.045 INFO:teuthology.orchestra.run.smithi081.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 685 kB/s | 34 kB 00:00 2024-03-08T05:58:26.145 INFO:teuthology.orchestra.run.smithi081.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-03-08T05:58:26.187 INFO:teuthology.orchestra.run.smithi081.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 11 MB/s | 1.5 MB 00:00 2024-03-08T05:58:26.221 INFO:teuthology.orchestra.run.smithi081.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2024-03-08T05:58:26.246 INFO:teuthology.orchestra.run.smithi081.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-03-08T05:58:26.355 INFO:teuthology.orchestra.run.smithi081.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 43 MB/s | 5.7 MB 00:00 2024-03-08T05:58:26.388 INFO:teuthology.orchestra.run.smithi081.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 7.2 MB/s | 1.0 MB 00:00 2024-03-08T05:58:26.414 INFO:teuthology.orchestra.run.smithi081.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-08T05:58:26.623 INFO:teuthology.orchestra.run.smithi081.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 5.8 MB 00:00 2024-03-08T05:58:26.639 INFO:teuthology.orchestra.run.smithi081.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 108 kB/s | 24 kB 00:00 2024-03-08T05:58:27.065 INFO:teuthology.orchestra.run.smithi081.stdout:(65/174): ceph-test-19.0.0-1670.g1fb5d106.el9.x 7.0 MB/s | 47 MB 00:06 2024-03-08T05:58:27.090 INFO:teuthology.orchestra.run.smithi081.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 40 kB/s | 18 kB 00:00 2024-03-08T05:58:27.115 INFO:teuthology.orchestra.run.smithi081.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 303 kB/s | 144 kB 00:00 2024-03-08T05:58:27.140 INFO:teuthology.orchestra.run.smithi081.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 421 kB/s | 32 kB 00:00 2024-03-08T05:58:27.165 INFO:teuthology.orchestra.run.smithi081.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 217 kB 00:00 2024-03-08T05:58:27.249 INFO:teuthology.orchestra.run.smithi081.stdout:(70/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-03-08T05:58:27.275 INFO:teuthology.orchestra.run.smithi081.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 174 kB 00:00 2024-03-08T05:58:27.300 INFO:teuthology.orchestra.run.smithi081.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 526 kB/s | 69 kB 00:00 2024-03-08T05:58:27.325 INFO:teuthology.orchestra.run.smithi081.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 552 kB/s | 41 kB 00:00 2024-03-08T05:58:27.350 INFO:teuthology.orchestra.run.smithi081.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-08T05:58:27.375 INFO:teuthology.orchestra.run.smithi081.stdout:(75/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-08T05:58:27.401 INFO:teuthology.orchestra.run.smithi081.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-08T05:58:27.426 INFO:teuthology.orchestra.run.smithi081.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 635 kB/s | 47 kB 00:00 2024-03-08T05:58:27.451 INFO:teuthology.orchestra.run.smithi081.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 857 kB/s | 43 kB 00:00 2024-03-08T05:58:27.475 INFO:teuthology.orchestra.run.smithi076.stdout:(130/174): python3-grpcio-1.46.7-10.el9.x86_64. 761 kB/s | 2.0 MB 00:02 2024-03-08T05:58:27.542 INFO:teuthology.orchestra.run.smithi076.stdout:(131/174): libarrow-9.0.0-10.el9.x86_64.rpm 870 kB/s | 4.4 MB 00:05 2024-03-08T05:58:27.577 INFO:teuthology.orchestra.run.smithi081.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 257 kB/s | 39 kB 00:00 2024-03-08T05:58:27.584 INFO:teuthology.orchestra.run.smithi076.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2024-03-08T05:58:27.651 INFO:teuthology.orchestra.run.smithi076.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 727 kB/s | 79 kB 00:00 2024-03-08T05:58:27.684 INFO:teuthology.orchestra.run.smithi076.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 576 kB/s | 58 kB 00:00 2024-03-08T05:58:27.735 INFO:teuthology.orchestra.run.smithi081.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 47 MB/s | 16 MB 00:00 2024-03-08T05:58:27.760 INFO:teuthology.orchestra.run.smithi081.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 134 kB/s | 41 kB 00:00 2024-03-08T05:58:27.794 INFO:teuthology.orchestra.run.smithi081.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2024-03-08T05:58:27.801 INFO:teuthology.orchestra.run.smithi076.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-03-08T05:58:27.819 INFO:teuthology.orchestra.run.smithi081.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 972 kB/s | 56 kB 00:00 2024-03-08T05:58:27.835 INFO:teuthology.orchestra.run.smithi076.stdout:(136/174): python3-kubernetes-26.1.0-2.el9.noar 488 kB/s | 1.0 MB 00:02 2024-03-08T05:58:27.844 INFO:teuthology.orchestra.run.smithi081.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 754 kB/s | 63 kB 00:00 2024-03-08T05:58:27.860 INFO:teuthology.orchestra.run.smithi076.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-03-08T05:58:27.869 INFO:teuthology.orchestra.run.smithi081.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 586 kB/s | 44 kB 00:00 2024-03-08T05:58:27.903 INFO:teuthology.orchestra.run.smithi081.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-08T05:58:27.928 INFO:teuthology.orchestra.run.smithi081.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.8 MB/s | 241 kB 00:00 2024-03-08T05:58:27.944 INFO:teuthology.orchestra.run.smithi076.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-03-08T05:58:27.962 INFO:teuthology.orchestra.run.smithi081.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-08T05:58:27.969 INFO:teuthology.orchestra.run.smithi076.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 282 kB/s | 31 kB 00:00 2024-03-08T05:58:27.987 INFO:teuthology.orchestra.run.smithi081.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2024-03-08T05:58:27.994 INFO:teuthology.orchestra.run.smithi076.stdout:(140/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 466 kB/s | 90 kB 00:00 2024-03-08T05:58:28.013 INFO:teuthology.orchestra.run.smithi081.stdout:(90/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-08T05:58:28.038 INFO:teuthology.orchestra.run.smithi081.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-08T05:58:28.078 INFO:teuthology.orchestra.run.smithi076.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-03-08T05:58:28.103 INFO:teuthology.orchestra.run.smithi076.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 330 kB/s | 36 kB 00:00 2024-03-08T05:58:28.187 INFO:teuthology.orchestra.run.smithi076.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 469 kB/s | 51 kB 00:00 2024-03-08T05:58:28.212 INFO:teuthology.orchestra.run.smithi076.stdout:(144/174): python3-routes-2.5.1-5.el9.noarch.rp 704 kB/s | 188 kB 00:00 2024-03-08T05:58:28.222 INFO:teuthology.orchestra.run.smithi081.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 802 kB/s | 187 kB 00:00 2024-03-08T05:58:28.247 INFO:teuthology.orchestra.run.smithi081.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 935 kB/s | 195 kB 00:00 2024-03-08T05:58:28.272 INFO:teuthology.orchestra.run.smithi081.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-08T05:58:28.296 INFO:teuthology.orchestra.run.smithi076.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 828 kB/s | 90 kB 00:00 2024-03-08T05:58:28.331 INFO:teuthology.orchestra.run.smithi081.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-03-08T05:58:28.356 INFO:teuthology.orchestra.run.smithi081.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 198 kB/s | 21 kB 00:00 2024-03-08T05:58:28.381 INFO:teuthology.orchestra.run.smithi081.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 234 kB/s | 25 kB 00:00 2024-03-08T05:58:28.413 INFO:teuthology.orchestra.run.smithi076.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-03-08T05:58:28.474 INFO:teuthology.orchestra.run.smithi081.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-03-08T05:58:28.479 INFO:teuthology.orchestra.run.smithi076.stdout:(147/174): python3-webob-1.8.7-6.el9.noarch.rpm 611 kB/s | 230 kB 00:00 2024-03-08T05:58:28.524 INFO:teuthology.orchestra.run.smithi081.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 4.3 MB/s | 216 kB 00:00 2024-03-08T05:58:28.549 INFO:teuthology.orchestra.run.smithi081.stdout:(100/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.4 MB/s | 248 kB 00:00 2024-03-08T05:58:28.574 INFO:teuthology.orchestra.run.smithi081.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 693 kB/s | 35 kB 00:00 2024-03-08T05:58:28.599 INFO:teuthology.orchestra.run.smithi081.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 886 kB/s | 44 kB 00:00 2024-03-08T05:58:28.605 INFO:teuthology.orchestra.run.smithi076.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 999 kB/s | 192 kB 00:00 2024-03-08T05:58:28.625 INFO:teuthology.orchestra.run.smithi081.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-08T05:58:28.692 INFO:teuthology.orchestra.run.smithi081.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 11 MB/s | 4.1 MB 00:00 2024-03-08T05:58:28.830 INFO:teuthology.orchestra.run.smithi076.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 691 kB/s | 427 kB 00:00 2024-03-08T05:58:28.892 INFO:teuthology.orchestra.run.smithi081.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-03-08T05:58:28.913 INFO:teuthology.orchestra.run.smithi076.stdout:(150/174): librados2-19.0.0-1670.g1fb5d106.el9. 11 MB/s | 3.4 MB 00:00 2024-03-08T05:58:28.955 INFO:teuthology.orchestra.run.smithi076.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-08T05:58:28.967 INFO:teuthology.orchestra.run.smithi081.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-03-08T05:58:29.026 INFO:teuthology.orchestra.run.smithi081.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 342 kB/s | 19 kB 00:00 2024-03-08T05:58:29.147 INFO:teuthology.orchestra.run.smithi076.stdout:(152/174): librbd1-19.0.0-1670.g1fb5d106.el9.x8 9.9 MB/s | 3.1 MB 00:00 2024-03-08T05:58:29.164 INFO:teuthology.orchestra.run.smithi081.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-03-08T05:58:29.198 INFO:teuthology.orchestra.run.smithi076.stdout:(153/174): NetworkManager-1.46.0-1.el9.x86_64.r 9.5 MB/s | 2.3 MB 00:00 2024-03-08T05:58:29.235 INFO:teuthology.orchestra.run.smithi081.stdout:(109/174): libarrow-9.0.0-10.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-03-08T05:58:29.239 INFO:teuthology.orchestra.run.smithi076.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-08T05:58:29.268 INFO:teuthology.orchestra.run.smithi081.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 251 kB/s | 26 kB 00:00 2024-03-08T05:58:29.281 INFO:teuthology.orchestra.run.smithi076.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-03-08T05:58:29.293 INFO:teuthology.orchestra.run.smithi081.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 831 kB/s | 49 kB 00:00 2024-03-08T05:58:29.327 INFO:teuthology.orchestra.run.smithi081.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-08T05:58:29.352 INFO:teuthology.orchestra.run.smithi081.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-03-08T05:58:29.445 INFO:teuthology.orchestra.run.smithi081.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-03-08T05:58:29.486 INFO:teuthology.orchestra.run.smithi081.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.1 MB/s | 838 kB 00:00 2024-03-08T05:58:29.512 INFO:teuthology.orchestra.run.smithi081.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 650 kB/s | 43 kB 00:00 2024-03-08T05:58:29.515 INFO:teuthology.orchestra.run.smithi076.stdout:(156/174): NetworkManager-libnm-1.46.0-1.el9.x8 4.9 MB/s | 1.8 MB 00:00 2024-03-08T05:58:29.537 INFO:teuthology.orchestra.run.smithi081.stdout:(117/174): protobuf-compiler-3.14.0-13.el9.x86_ 947 kB/s | 863 kB 00:00 2024-03-08T05:58:29.558 INFO:teuthology.orchestra.run.smithi076.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 258 kB/s | 71 kB 00:00 2024-03-08T05:58:29.562 INFO:teuthology.orchestra.run.smithi081.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 427 kB/s | 32 kB 00:00 2024-03-08T05:58:29.582 INFO:teuthology.orchestra.run.smithi076.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 545 kB/s | 36 kB 00:00 2024-03-08T05:58:29.608 INFO:teuthology.orchestra.run.smithi076.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2024-03-08T05:58:29.612 INFO:teuthology.orchestra.run.smithi081.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-03-08T05:58:29.641 INFO:teuthology.orchestra.run.smithi076.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 256 kB 00:00 2024-03-08T05:58:29.658 INFO:teuthology.orchestra.run.smithi076.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 459 kB/s | 23 kB 00:00 2024-03-08T05:58:29.684 INFO:teuthology.orchestra.run.smithi076.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 108 kB 00:00 2024-03-08T05:58:29.688 INFO:teuthology.orchestra.run.smithi081.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-03-08T05:58:29.709 INFO:teuthology.orchestra.run.smithi076.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-03-08T05:58:29.722 INFO:teuthology.orchestra.run.smithi081.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-03-08T05:58:29.742 INFO:teuthology.orchestra.run.smithi076.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 136 kB 00:00 2024-03-08T05:58:29.747 INFO:teuthology.orchestra.run.smithi081.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 605 kB/s | 35 kB 00:00 2024-03-08T05:58:29.768 INFO:teuthology.orchestra.run.smithi076.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-03-08T05:58:29.772 INFO:teuthology.orchestra.run.smithi081.stdout:(123/174): python3-cheroot-8.6.0-4.el9.noarch.r 731 kB/s | 172 kB 00:00 2024-03-08T05:58:29.784 INFO:teuthology.orchestra.run.smithi076.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 687 kB/s | 29 kB 00:00 2024-03-08T05:58:29.798 INFO:teuthology.orchestra.run.smithi081.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-03-08T05:58:29.827 INFO:teuthology.orchestra.run.smithi076.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 21 MB/s | 1.2 MB 00:00 2024-03-08T05:58:29.848 INFO:teuthology.orchestra.run.smithi081.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-03-08T05:58:29.873 INFO:teuthology.orchestra.run.smithi081.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-03-08T05:58:29.899 INFO:teuthology.orchestra.run.smithi081.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-03-08T05:58:29.901 INFO:teuthology.orchestra.run.smithi076.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 18 MB/s | 2.2 MB 00:00 2024-03-08T05:58:29.924 INFO:teuthology.orchestra.run.smithi081.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-03-08T05:58:29.949 INFO:teuthology.orchestra.run.smithi081.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-03-08T05:58:29.952 INFO:teuthology.orchestra.run.smithi076.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 18 MB/s | 2.3 MB 00:00 2024-03-08T05:58:29.974 INFO:teuthology.orchestra.run.smithi081.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-03-08T05:58:29.977 INFO:teuthology.orchestra.run.smithi076.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.0 MB/s | 465 kB 00:00 2024-03-08T05:58:30.003 INFO:teuthology.orchestra.run.smithi076.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 618 kB/s | 31 kB 00:00 2024-03-08T05:58:30.024 INFO:teuthology.orchestra.run.smithi081.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 7.4 MB/s | 2.0 MB 00:00 2024-03-08T05:58:30.045 INFO:teuthology.orchestra.run.smithi076.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-03-08T05:58:30.050 INFO:teuthology.orchestra.run.smithi081.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 614 kB/s | 46 kB 00:00 2024-03-08T05:58:30.075 INFO:teuthology.orchestra.run.smithi081.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-03-08T05:58:30.086 INFO:teuthology.orchestra.run.smithi076.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 892 kB/s | 74 kB 00:00 2024-03-08T05:58:30.100 INFO:teuthology.orchestra.run.smithi081.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-08T05:58:30.134 INFO:teuthology.orchestra.run.smithi081.stdout:(135/174): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-03-08T05:58:30.228 INFO:teuthology.orchestra.run.smithi081.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 129 kB/s | 16 kB 00:00 2024-03-08T05:58:30.293 INFO:teuthology.orchestra.run.smithi081.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 567 kB/s | 90 kB 00:00 2024-03-08T05:58:30.327 INFO:teuthology.orchestra.run.smithi081.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 288 kB/s | 28 kB 00:00 2024-03-08T05:58:30.352 INFO:teuthology.orchestra.run.smithi081.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 526 kB/s | 31 kB 00:00 2024-03-08T05:58:30.386 INFO:teuthology.orchestra.run.smithi081.stdout:(140/174): python3-pecan-1.4.2-3.el9.noarch.rpm 878 kB/s | 272 kB 00:00 2024-03-08T05:58:30.411 INFO:teuthology.orchestra.run.smithi081.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-03-08T05:58:30.436 INFO:teuthology.orchestra.run.smithi081.stdout:(142/174): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-03-08T05:58:30.461 INFO:teuthology.orchestra.run.smithi081.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-03-08T05:58:30.487 INFO:teuthology.orchestra.run.smithi081.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-03-08T05:58:30.512 INFO:teuthology.orchestra.run.smithi081.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-03-08T05:58:30.537 INFO:teuthology.orchestra.run.smithi081.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-03-08T05:58:30.562 INFO:teuthology.orchestra.run.smithi081.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-03-08T05:58:30.588 INFO:teuthology.orchestra.run.smithi081.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-03-08T05:58:30.613 INFO:teuthology.orchestra.run.smithi081.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-08T05:58:30.697 INFO:teuthology.orchestra.run.smithi081.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-03-08T05:58:30.837 INFO:teuthology.orchestra.run.smithi076.stdout:(174/174): thrift-0.15.0-2.el9.x86_64.rpm 687 kB/s | 1.6 MB 00:02 2024-03-08T05:58:30.843 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-03-08T05:58:30.843 INFO:teuthology.orchestra.run.smithi076.stdout:Total 12 MB/s | 223 MB 00:19 2024-03-08T05:58:30.847 INFO:teuthology.orchestra.run.smithi081.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.5 MB/s | 321 kB 00:00 2024-03-08T05:58:31.327 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-03-08T05:58:31.364 INFO:teuthology.orchestra.run.smithi081.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 4.4 MB/s | 2.3 MB 00:00 2024-03-08T05:58:31.634 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-03-08T05:58:31.634 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-03-08T05:58:31.881 INFO:teuthology.orchestra.run.smithi081.stdout:(153/174): NetworkManager-libnm-1.46.0-1.el9.x8 3.5 MB/s | 1.8 MB 00:00 2024-03-08T05:58:31.914 INFO:teuthology.orchestra.run.smithi081.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 1.2 MB/s | 42 kB 00:00 2024-03-08T05:58:32.006 INFO:teuthology.orchestra.run.smithi081.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 2.6 MB/s | 248 kB 00:00 2024-03-08T05:58:32.048 INFO:teuthology.orchestra.run.smithi081.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-03-08T05:58:32.090 INFO:teuthology.orchestra.run.smithi081.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 875 kB/s | 36 kB 00:00 2024-03-08T05:58:32.182 INFO:teuthology.orchestra.run.smithi081.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.0 MB/s | 192 kB 00:00 2024-03-08T05:58:32.291 INFO:teuthology.orchestra.run.smithi081.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-03-08T05:58:32.325 INFO:teuthology.orchestra.run.smithi081.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 695 kB/s | 23 kB 00:00 2024-03-08T05:58:32.367 INFO:teuthology.orchestra.run.smithi081.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 108 kB 00:00 2024-03-08T05:58:32.433 INFO:teuthology.orchestra.run.smithi081.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 155 kB 00:00 2024-03-08T05:58:32.484 INFO:teuthology.orchestra.run.smithi081.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.7 MB/s | 136 kB 00:00 2024-03-08T05:58:32.534 INFO:teuthology.orchestra.run.smithi081.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.3 MB/s | 64 kB 00:00 2024-03-08T05:58:32.568 INFO:teuthology.orchestra.run.smithi081.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 863 kB/s | 29 kB 00:00 2024-03-08T05:58:32.910 INFO:teuthology.orchestra.run.smithi081.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 3.5 MB/s | 1.2 MB 00:00 2024-03-08T05:58:33.368 INFO:teuthology.orchestra.run.smithi081.stdout:(167/174): librbd1-19.0.0-1670.g1fb5d106.el9.x8 1.1 MB/s | 3.1 MB 00:02 2024-03-08T05:58:33.460 INFO:teuthology.orchestra.run.smithi081.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.9 MB/s | 2.2 MB 00:00 2024-03-08T05:58:33.519 INFO:teuthology.orchestra.run.smithi081.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 7.8 MB/s | 465 kB 00:00 2024-03-08T05:58:33.569 INFO:teuthology.orchestra.run.smithi081.stdout:(170/174): librados2-19.0.0-1670.g1fb5d106.el9. 1.1 MB/s | 3.4 MB 00:02 2024-03-08T05:58:33.586 INFO:teuthology.orchestra.run.smithi081.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 467 kB/s | 31 kB 00:00 2024-03-08T05:58:33.678 INFO:teuthology.orchestra.run.smithi081.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 810 kB/s | 74 kB 00:00 2024-03-08T05:58:33.724 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-03-08T05:58:33.724 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-03-08T05:58:33.778 INFO:teuthology.orchestra.run.smithi081.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.1 MB/s | 1.1 MB 00:00 2024-03-08T05:58:33.978 INFO:teuthology.orchestra.run.smithi081.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.7 MB/s | 2.3 MB 00:00 2024-03-08T05:58:33.985 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-08T05:58:33.985 INFO:teuthology.orchestra.run.smithi081.stdout:Total 11 MB/s | 223 MB 00:19 2024-03-08T05:58:34.448 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-08T05:58:34.759 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-08T05:58:34.760 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-08T05:58:36.787 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-08T05:58:36.787 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-08T05:58:40.759 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-03-08T05:58:40.880 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-08T05:58:40.918 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-08T05:58:40.964 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T05:58:40.973 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T05:58:41.014 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-08T05:58:41.080 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-08T05:58:41.122 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-08T05:58:41.156 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-08T05:58:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-08T05:58:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-08T05:58:41.582 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T05:58:41.727 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T05:58:41.758 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T05:58:41.849 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T05:58:41.885 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 12/198 2024-03-08T05:58:41.931 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-08T05:58:42.117 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-08T05:58:42.176 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-08T05:58:42.212 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T05:58:42.316 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T05:58:42.338 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 17/198 2024-03-08T05:58:42.404 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 17/198 2024-03-08T05:58:42.459 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-08T05:58:42.490 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T05:58:42.506 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T05:58:42.705 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T05:58:42.740 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T05:58:42.970 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-08T05:58:42.973 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-08T05:58:43.090 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T05:58:43.142 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T05:58:43.245 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T05:58:43.318 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T05:58:43.525 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T05:58:43.641 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-08T05:58:43.699 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-08T05:58:43.734 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-08T05:58:43.774 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-08T05:58:43.787 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-08T05:58:43.800 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-08T05:58:43.836 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-08T05:58:43.909 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-08T05:58:43.963 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-08T05:58:44.002 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-08T05:58:44.018 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-08T05:58:44.075 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-08T05:58:44.106 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T05:58:44.118 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-08T05:58:44.118 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T05:58:44.163 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-08T05:58:44.173 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-08T05:58:44.195 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-08T05:58:44.231 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-08T05:58:44.236 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-08T05:58:44.277 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-08T05:58:44.301 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-08T05:58:44.302 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-08T05:58:44.336 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9. 39/198 2024-03-08T05:58:44.349 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-08T05:58:44.380 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el 40/198 2024-03-08T05:58:44.408 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_6 41/198 2024-03-08T05:58:44.478 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-08T05:58:44.523 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-08T05:58:44.605 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-08T05:58:44.672 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-08T05:58:44.736 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T05:58:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-08T05:58:44.850 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-08T05:58:44.866 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T05:58:44.896 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T05:58:44.996 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T05:58:45.028 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 12/198 2024-03-08T05:58:45.075 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-08T05:58:45.263 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-08T05:58:45.315 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-08T05:58:45.343 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T05:58:45.376 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-08T05:58:45.438 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T05:58:45.461 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 17/198 2024-03-08T05:58:45.462 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-08T05:58:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 17/198 2024-03-08T05:58:45.549 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-08T05:58:45.552 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-08T05:58:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T05:58:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T05:58:45.615 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T05:58:45.625 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T05:58:45.668 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-08T05:58:45.730 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-08T05:58:45.768 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_6 53/198 2024-03-08T05:58:45.778 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T05:58:45.798 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-08T05:58:45.811 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T05:58:45.851 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-08T05:58:46.014 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-08T05:58:46.146 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-08T05:58:46.153 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T05:58:46.205 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T05:58:46.313 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T05:58:46.401 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T05:58:46.511 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-08T05:58:46.608 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-08T05:58:46.630 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T05:58:46.653 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-08T05:58:46.742 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-08T05:58:46.794 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-08T05:58:46.835 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-08T05:58:46.839 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-08T05:58:46.865 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-08T05:58:46.893 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-08T05:58:46.901 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-08T05:58:46.952 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-08T05:58:46.974 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-08T05:58:47.014 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-08T05:58:47.025 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-08T05:58:47.051 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-08T05:58:47.087 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-08T05:58:47.097 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-08T05:58:47.126 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-08T05:58:47.134 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-08T05:58:47.177 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-08T05:58:47.198 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-08T05:58:47.219 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-08T05:58:47.234 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-08T05:58:47.277 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-08T05:58:47.310 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-08T05:58:47.340 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-08T05:58:47.353 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-08T05:58:47.399 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-08T05:58:47.409 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-08T05:58:47.430 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-08T05:58:47.444 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9. 39/198 2024-03-08T05:58:47.471 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-08T05:58:47.488 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el 40/198 2024-03-08T05:58:47.517 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_6 41/198 2024-03-08T05:58:47.564 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-08T05:58:47.586 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-08T05:58:47.604 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-08T05:58:47.639 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-08T05:58:47.645 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-08T05:58:47.679 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-08T05:58:47.703 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-08T05:58:47.713 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-08T05:58:47.800 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-08T05:58:47.848 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-08T05:58:47.902 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-08T05:58:47.930 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-08T05:58:48.038 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-08T05:58:48.079 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-08T05:58:48.136 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-08T05:58:48.192 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-08T05:58:48.386 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-08T05:58:48.426 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-08T05:58:48.433 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-08T05:58:48.506 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-08T05:58:48.560 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-08T05:58:48.589 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-08T05:58:48.598 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-08T05:58:48.624 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T05:58:48.638 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-08T05:58:48.639 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T05:58:48.666 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-08T05:58:48.677 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-08T05:58:48.705 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-08T05:58:48.714 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-08T05:58:48.740 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-08T05:58:48.751 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_6 53/198 2024-03-08T05:58:48.784 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-08T05:58:48.795 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-08T05:58:48.827 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-08T05:58:48.857 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-08T05:58:48.933 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-08T05:58:48.968 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-08T05:58:49.001 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-08T05:58:49.043 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-08T05:58:49.051 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-08T05:58:49.085 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-08T05:58:49.201 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-08T05:58:49.233 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-08T05:58:49.290 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-08T05:58:49.317 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-08T05:58:49.333 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-08T05:58:49.361 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-08T05:58:49.364 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-08T05:58:49.412 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-08T05:58:49.495 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-08T05:58:49.509 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-08T05:58:49.547 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-08T05:58:49.566 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-08T05:58:49.610 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-08T05:58:49.675 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-08T05:58:49.714 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-08T05:58:49.750 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-08T05:58:49.797 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-08T05:58:49.833 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-08T05:58:49.876 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-08T05:58:49.935 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-08T05:58:50.068 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-08T05:58:50.107 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-08T05:58:50.138 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-08T05:58:50.179 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-08T05:58:50.274 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-08T05:58:50.294 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-08T05:58:50.313 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-08T05:58:50.340 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-08T05:58:50.353 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-08T05:58:50.374 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-08T05:58:50.378 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-08T05:58:50.403 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-08T05:58:50.422 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-08T05:58:50.491 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-08T05:58:50.510 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-08T05:58:50.559 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-08T05:58:50.612 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-08T05:58:50.723 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-08T05:58:50.740 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-08T05:58:50.765 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-08T05:58:50.770 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-08T05:58:50.818 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-08T05:58:50.845 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-08T05:58:50.874 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-08T05:58:51.057 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-08T05:58:51.107 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-08T05:58:51.243 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-08T05:58:51.281 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-08T05:58:51.312 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-08T05:58:51.343 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-08T05:58:51.371 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-08T05:58:51.405 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-08T05:58:51.444 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-08T05:58:51.490 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-08T05:58:51.493 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-08T05:58:51.523 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-08T05:58:51.590 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-08T05:58:51.604 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-08T05:58:51.633 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-08T05:58:51.666 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-08T05:58:51.708 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-08T05:58:51.743 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-08T05:58:51.831 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-08T05:58:51.914 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-08T05:58:51.957 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-08T05:58:51.996 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-08T05:58:52.043 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-08T05:58:52.140 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-08T05:58:52.189 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-08T05:58:52.813 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-08T05:58:52.907 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-08T05:58:52.914 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-08T05:58:52.939 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-08T05:58:52.962 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-08T05:58:52.969 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-08T05:58:52.996 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-08T05:58:52.997 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-08T05:58:53.044 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-08T05:58:53.134 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-08T05:58:53.171 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-08T05:58:53.316 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-08T05:58:53.347 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-08T05:58:53.377 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-08T05:58:53.379 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 124/198 2024-03-08T05:58:53.455 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-08T05:58:53.741 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 124/198 2024-03-08T05:58:53.832 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 125/198 2024-03-08T05:58:53.869 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-08T05:58:54.091 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-08T05:58:54.211 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-08T05:58:54.321 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-08T05:58:54.356 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 128/198 2024-03-08T05:58:54.426 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 128/198 2024-03-08T05:58:54.449 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 129/198 2024-03-08T05:58:55.446 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-08T05:58:55.521 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-08T05:58:55.552 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-08T05:58:55.583 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-08T05:58:55.611 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-08T05:58:55.930 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-08T05:58:55.966 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 124/198 2024-03-08T05:58:56.049 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 124/198 2024-03-08T05:58:56.079 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 125/198 2024-03-08T05:58:56.115 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-08T05:58:56.542 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T05:58:56.562 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-08T05:58:56.579 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T05:58:56.594 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 128/198 2024-03-08T05:58:56.648 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 128/198 2024-03-08T05:58:56.671 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 129/198 2024-03-08T05:58:57.121 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T05:58:57.157 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 131/198 2024-03-08T05:58:57.247 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 131/198 2024-03-08T05:58:57.247 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-08T05:58:57.247 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-08T05:58:57.247 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:58:57.283 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 132/198 2024-03-08T05:58:58.870 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T05:58:58.903 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T05:58:59.445 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T05:58:59.489 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 131/198 2024-03-08T05:58:59.560 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 131/198 2024-03-08T05:58:59.560 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-08T05:58:59.560 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-08T05:58:59.560 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:58:59.599 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 132/198 2024-03-08T05:59:08.844 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 132/198 2024-03-08T05:59:08.845 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /sys 2024-03-08T05:59:08.845 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /proc 2024-03-08T05:59:08.845 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /mnt 2024-03-08T05:59:08.845 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /var/tmp 2024-03-08T05:59:08.845 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /home 2024-03-08T05:59:08.845 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /root 2024-03-08T05:59:08.845 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /tmp 2024-03-08T05:59:08.845 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:09.014 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 133/198 2024-03-08T05:59:09.030 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 133/198 2024-03-08T05:59:09.030 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:09.030 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-08T05:59:09.030 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:10.525 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 134/198 2024-03-08T05:59:10.551 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 134/198 2024-03-08T05:59:10.551 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:10.551 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-08T05:59:10.551 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T05:59:10.551 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T05:59:10.551 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:10.759 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 135/198 2024-03-08T05:59:10.782 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 135/198 2024-03-08T05:59:10.782 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:10.782 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-08T05:59:10.782 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T05:59:10.783 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T05:59:10.783 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:11.042 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 132/198 2024-03-08T05:59:11.042 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /sys 2024-03-08T05:59:11.042 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /proc 2024-03-08T05:59:11.042 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /mnt 2024-03-08T05:59:11.042 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /var/tmp 2024-03-08T05:59:11.042 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /home 2024-03-08T05:59:11.042 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /root 2024-03-08T05:59:11.042 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /tmp 2024-03-08T05:59:11.042 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:11.220 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 133/198 2024-03-08T05:59:11.226 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 136/198 2024-03-08T05:59:11.235 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 133/198 2024-03-08T05:59:11.235 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:11.235 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-08T05:59:11.235 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:11.249 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 136/198 2024-03-08T05:59:11.249 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:11.249 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-08T05:59:11.250 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T05:59:11.250 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T05:59:11.250 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:11.345 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-08T05:59:11.780 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 138/198 2024-03-08T05:59:11.810 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 138/198 2024-03-08T05:59:11.842 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.e 139/198 2024-03-08T05:59:11.911 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106. 140/198 2024-03-08T05:59:11.933 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 141/198 2024-03-08T05:59:12.161 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 141/198 2024-03-08T05:59:12.192 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 142/198 2024-03-08T05:59:12.698 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 134/198 2024-03-08T05:59:12.724 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 134/198 2024-03-08T05:59:12.724 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:12.724 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-08T05:59:12.724 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T05:59:12.724 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T05:59:12.724 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:12.932 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 135/198 2024-03-08T05:59:12.954 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 135/198 2024-03-08T05:59:12.954 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:12.954 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-08T05:59:12.954 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T05:59:12.954 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T05:59:12.954 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:13.163 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 142/198 2024-03-08T05:59:13.193 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 143/198 2024-03-08T05:59:13.295 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 143/198 2024-03-08T05:59:13.391 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 136/198 2024-03-08T05:59:13.414 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 136/198 2024-03-08T05:59:13.414 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:13.414 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-08T05:59:13.414 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T05:59:13.414 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T05:59:13.414 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:13.455 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el 144/198 2024-03-08T05:59:13.495 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 145/198 2024-03-08T05:59:13.509 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-08T05:59:13.518 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 145/198 2024-03-08T05:59:13.518 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:13.518 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-08T05:59:13.518 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T05:59:13.519 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T05:59:13.519 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:13.562 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 146/198 2024-03-08T05:59:13.574 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 146/198 2024-03-08T05:59:13.624 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 147/198 2024-03-08T05:59:13.936 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 138/198 2024-03-08T05:59:13.966 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 138/198 2024-03-08T05:59:13.997 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.e 139/198 2024-03-08T05:59:14.057 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106. 140/198 2024-03-08T05:59:14.098 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 141/198 2024-03-08T05:59:14.328 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 141/198 2024-03-08T05:59:14.365 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 142/198 2024-03-08T05:59:14.488 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-08T05:59:14.523 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 149/198 2024-03-08T05:59:14.546 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 149/198 2024-03-08T05:59:14.547 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:14.547 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-08T05:59:14.547 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T05:59:14.547 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T05:59:14.547 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:14.590 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 150/198 2024-03-08T05:59:14.613 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 150/198 2024-03-08T05:59:14.613 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:14.613 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-08T05:59:14.613 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:14.657 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T05:59:14.680 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T05:59:14.680 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:14.681 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-08T05:59:14.681 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T05:59:14.681 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T05:59:14.681 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:14.983 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 152/198 2024-03-08T05:59:15.006 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 152/198 2024-03-08T05:59:15.006 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:15.006 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-08T05:59:15.007 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T05:59:15.007 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T05:59:15.007 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:15.328 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 142/198 2024-03-08T05:59:15.357 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 143/198 2024-03-08T05:59:15.457 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 143/198 2024-03-08T05:59:15.620 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el 144/198 2024-03-08T05:59:15.651 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 145/198 2024-03-08T05:59:15.674 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 145/198 2024-03-08T05:59:15.674 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:15.674 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-08T05:59:15.674 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T05:59:15.674 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T05:59:15.674 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:15.718 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 146/198 2024-03-08T05:59:15.729 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 146/198 2024-03-08T05:59:15.783 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 147/198 2024-03-08T05:59:16.637 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-08T05:59:16.662 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 149/198 2024-03-08T05:59:16.685 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 149/198 2024-03-08T05:59:16.685 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:16.685 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-08T05:59:16.685 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T05:59:16.685 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T05:59:16.685 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:16.729 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 150/198 2024-03-08T05:59:16.751 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 150/198 2024-03-08T05:59:16.751 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:16.751 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-08T05:59:16.751 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:16.797 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T05:59:16.819 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T05:59:16.819 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:16.820 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-08T05:59:16.820 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T05:59:16.820 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T05:59:16.820 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:17.131 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 152/198 2024-03-08T05:59:17.154 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 152/198 2024-03-08T05:59:17.154 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T05:59:17.154 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-08T05:59:17.154 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T05:59:17.154 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T05:59:17.154 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:19.457 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 153/198 2024-03-08T05:59:19.497 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 154/198 2024-03-08T05:59:19.528 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 155/198 2024-03-08T05:59:19.555 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-08T05:59:19.755 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 157/198 2024-03-08T05:59:19.871 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-08T05:59:19.912 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 159/198 2024-03-08T05:59:19.964 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-08T05:59:20.029 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-08T05:59:20.072 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-08T05:59:20.104 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_ 163/198 2024-03-08T05:59:20.145 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-08T05:59:20.175 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-08T05:59:20.242 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-08T05:59:20.328 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-08T05:59:20.365 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-08T05:59:20.887 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T05:59:20.938 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T05:59:20.958 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T05:59:21.602 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 153/198 2024-03-08T05:59:21.643 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 154/198 2024-03-08T05:59:21.673 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 155/198 2024-03-08T05:59:21.701 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-08T05:59:21.787 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 170/198 2024-03-08T05:59:21.867 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-08T05:59:21.891 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 157/198 2024-03-08T05:59:21.908 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-08T05:59:22.002 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-08T05:59:22.007 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-08T05:59:22.037 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-08T05:59:22.049 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 159/198 2024-03-08T05:59:22.062 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-08T05:59:22.088 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-08T05:59:22.102 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-08T05:59:22.114 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-08T05:59:22.114 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T05:59:22.157 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-08T05:59:22.192 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-08T05:59:22.219 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T05:59:22.224 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_ 163/198 2024-03-08T05:59:22.257 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-08T05:59:22.257 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T05:59:22.265 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-08T05:59:22.295 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-08T05:59:22.350 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T05:59:22.358 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-08T05:59:22.386 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-08T05:59:22.411 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-08T05:59:22.417 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-08T05:59:22.443 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-08T05:59:22.443 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-08T05:59:22.468 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-08T05:59:22.502 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-08T05:59:22.549 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-08T05:59:22.590 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-08T05:59:22.615 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-08T05:59:22.640 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-08T05:59:22.665 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-08T05:59:22.691 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-08T05:59:22.697 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T05:59:22.698 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T05:59:22.894 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T05:59:22.894 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T05:59:22.977 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T05:59:23.003 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-08T05:59:23.003 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T05:59:23.026 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-08T05:59:23.043 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T05:59:23.060 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-08T05:59:23.063 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T05:59:23.086 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-08T05:59:23.152 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T05:59:23.876 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 170/198 2024-03-08T05:59:23.955 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-08T05:59:23.997 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-08T05:59:24.078 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-08T05:59:24.117 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-08T05:59:24.142 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-08T05:59:24.177 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-08T05:59:24.202 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-08T05:59:24.202 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T05:59:24.307 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T05:59:24.337 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-08T05:59:24.337 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T05:59:24.422 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T05:59:24.481 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-08T05:59:24.522 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-08T05:59:24.548 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-08T05:59:24.572 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-08T05:59:24.606 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-08T05:59:24.653 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-08T05:59:24.694 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-08T05:59:24.719 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-08T05:59:24.761 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-08T05:59:24.786 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-08T05:59:24.812 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-08T05:59:24.819 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T05:59:24.819 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T05:59:24.998 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T05:59:24.998 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T05:59:25.081 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T05:59:25.106 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-08T05:59:25.131 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-08T05:59:25.164 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-08T05:59:25.190 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-08T05:59:25.256 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T05:59:28.098 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T05:59:28.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 1/198 2024-03-08T05:59:28.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 2/198 2024-03-08T05:59:28.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 3/198 2024-03-08T05:59:28.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 4/198 2024-03-08T05:59:28.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 5/198 2024-03-08T05:59:28.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 6/198 2024-03-08T05:59:28.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 7/198 2024-03-08T05:59:28.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 8/198 2024-03-08T05:59:28.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 9/198 2024-03-08T05:59:28.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T05:59:28.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T05:59:28.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 12/198 2024-03-08T05:59:28.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 13/198 2024-03-08T05:59:28.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 14/198 2024-03-08T05:59:28.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_ 15/198 2024-03-08T05:59:28.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T05:59:28.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 17/198 2024-03-08T05:59:28.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_6 18/198 2024-03-08T05:59:28.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 19/198 2024-03-08T05:59:28.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 20/198 2024-03-08T05:59:28.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el 21/198 2024-03-08T05:59:28.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9. 22/198 2024-03-08T05:59:28.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_6 23/198 2024-03-08T05:59:28.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 24/198 2024-03-08T05:59:28.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 25/198 2024-03-08T05:59:28.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 26/198 2024-03-08T05:59:28.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 27/198 2024-03-08T05:59:28.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 28/198 2024-03-08T05:59:28.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 29/198 2024-03-08T05:59:28.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106. 30/198 2024-03-08T05:59:28.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 31/198 2024-03-08T05:59:28.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 32/198 2024-03-08T05:59:28.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 33/198 2024-03-08T05:59:28.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el 34/198 2024-03-08T05:59:28.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 35/198 2024-03-08T05:59:28.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.e 36/198 2024-03-08T05:59:28.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 37/198 2024-03-08T05:59:28.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 38/198 2024-03-08T05:59:28.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 39/198 2024-03-08T05:59:28.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-08T05:59:28.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-08T05:59:28.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-08T05:59:28.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-08T05:59:28.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-08T05:59:28.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-08T05:59:28.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-08T05:59:28.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-08T05:59:28.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-08T05:59:28.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-08T05:59:28.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-08T05:59:28.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-08T05:59:28.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-08T05:59:28.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-08T05:59:28.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-08T05:59:28.103 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-08T05:59:28.103 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-08T05:59:28.103 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-08T05:59:28.103 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-08T05:59:28.103 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-08T05:59:28.103 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-08T05:59:28.103 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-08T05:59:28.103 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-08T05:59:28.103 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-08T05:59:28.103 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-08T05:59:28.103 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-08T05:59:28.103 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-08T05:59:28.103 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-08T05:59:28.103 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-08T05:59:28.104 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-08T05:59:28.104 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-08T05:59:28.104 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-08T05:59:28.104 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-08T05:59:28.104 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-08T05:59:28.104 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-08T05:59:28.104 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-08T05:59:28.104 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-08T05:59:28.104 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-08T05:59:28.104 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-08T05:59:28.104 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-08T05:59:28.104 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-08T05:59:28.104 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-08T05:59:28.104 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-08T05:59:28.104 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-08T05:59:28.105 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-08T05:59:28.105 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-08T05:59:28.105 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-08T05:59:28.105 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-08T05:59:28.105 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-08T05:59:28.105 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-08T05:59:28.105 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-08T05:59:28.105 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-08T05:59:28.105 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-08T05:59:28.105 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-08T05:59:28.105 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-08T05:59:28.105 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-08T05:59:28.105 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-08T05:59:28.106 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-08T05:59:28.106 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-08T05:59:28.106 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-08T05:59:28.106 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-08T05:59:28.106 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-08T05:59:28.106 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-08T05:59:28.106 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-08T05:59:28.106 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-08T05:59:28.106 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-08T05:59:28.106 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-08T05:59:28.106 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-08T05:59:28.106 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-08T05:59:28.106 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-08T05:59:28.107 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-08T05:59:28.107 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-08T05:59:28.107 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-08T05:59:28.107 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-08T05:59:28.107 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-08T05:59:28.107 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-08T05:59:28.107 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-08T05:59:28.107 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-08T05:59:28.107 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-08T05:59:28.107 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-08T05:59:28.107 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-08T05:59:28.107 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-08T05:59:28.107 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-08T05:59:28.107 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-08T05:59:28.107 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-08T05:59:28.108 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-08T05:59:28.108 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-08T05:59:28.108 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-08T05:59:28.108 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-08T05:59:28.108 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-08T05:59:28.108 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-08T05:59:28.108 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-08T05:59:28.108 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-08T05:59:28.108 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-08T05:59:28.108 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-08T05:59:28.108 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-08T05:59:28.108 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-08T05:59:28.108 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-08T05:59:28.108 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-08T05:59:28.109 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-08T05:59:28.109 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-08T05:59:28.109 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-08T05:59:28.109 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-08T05:59:28.109 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-08T05:59:28.109 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-08T05:59:28.109 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-08T05:59:28.109 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-08T05:59:28.109 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-08T05:59:28.109 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-08T05:59:28.109 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-08T05:59:28.109 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-08T05:59:28.109 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T05:59:28.109 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-08T05:59:28.109 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 153/198 2024-03-08T05:59:28.110 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-08T05:59:28.110 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-08T05:59:28.110 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-08T05:59:28.110 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-08T05:59:28.110 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-08T05:59:28.110 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-08T05:59:28.110 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-08T05:59:28.110 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-08T05:59:28.110 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-08T05:59:28.110 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-08T05:59:28.110 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-08T05:59:28.110 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-08T05:59:28.110 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-08T05:59:28.110 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-08T05:59:28.111 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-08T05:59:28.111 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-08T05:59:28.111 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-08T05:59:28.111 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-08T05:59:28.111 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-08T05:59:28.111 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-08T05:59:28.111 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-08T05:59:28.111 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-08T05:59:28.111 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-08T05:59:28.111 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-08T05:59:28.111 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-08T05:59:28.111 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-08T05:59:28.111 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-08T05:59:28.111 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-08T05:59:28.113 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-08T05:59:28.113 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-08T05:59:28.113 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-08T05:59:28.113 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-08T05:59:28.113 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-08T05:59:28.113 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-08T05:59:28.113 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-08T05:59:28.113 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-08T05:59:28.113 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-08T05:59:28.113 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-08T05:59:28.113 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-08T05:59:28.113 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-08T05:59:28.113 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-08T05:59:28.113 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-08T05:59:28.114 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-08T05:59:28.114 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-08T05:59:30.209 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T05:59:30.209 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 1/198 2024-03-08T05:59:30.209 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 2/198 2024-03-08T05:59:30.209 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 3/198 2024-03-08T05:59:30.209 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 4/198 2024-03-08T05:59:30.209 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 5/198 2024-03-08T05:59:30.209 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 6/198 2024-03-08T05:59:30.209 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 7/198 2024-03-08T05:59:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 8/198 2024-03-08T05:59:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 9/198 2024-03-08T05:59:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T05:59:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T05:59:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 12/198 2024-03-08T05:59:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 13/198 2024-03-08T05:59:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 14/198 2024-03-08T05:59:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_ 15/198 2024-03-08T05:59:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T05:59:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 17/198 2024-03-08T05:59:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_6 18/198 2024-03-08T05:59:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 19/198 2024-03-08T05:59:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 20/198 2024-03-08T05:59:30.211 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el 21/198 2024-03-08T05:59:30.211 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9. 22/198 2024-03-08T05:59:30.211 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_6 23/198 2024-03-08T05:59:30.211 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 24/198 2024-03-08T05:59:30.211 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 25/198 2024-03-08T05:59:30.211 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 26/198 2024-03-08T05:59:30.211 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 27/198 2024-03-08T05:59:30.211 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 28/198 2024-03-08T05:59:30.211 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 29/198 2024-03-08T05:59:30.211 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106. 30/198 2024-03-08T05:59:30.211 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 31/198 2024-03-08T05:59:30.211 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 32/198 2024-03-08T05:59:30.211 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 33/198 2024-03-08T05:59:30.211 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el 34/198 2024-03-08T05:59:30.212 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 35/198 2024-03-08T05:59:30.212 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.e 36/198 2024-03-08T05:59:30.212 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 37/198 2024-03-08T05:59:30.212 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 38/198 2024-03-08T05:59:30.212 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 39/198 2024-03-08T05:59:30.212 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-08T05:59:30.212 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-08T05:59:30.212 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-08T05:59:30.212 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-08T05:59:30.212 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-08T05:59:30.212 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-08T05:59:30.212 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-08T05:59:30.212 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-08T05:59:30.212 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-08T05:59:30.212 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-08T05:59:30.212 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-08T05:59:30.213 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-08T05:59:30.213 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-08T05:59:30.213 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-08T05:59:30.213 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-08T05:59:30.213 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-08T05:59:30.213 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-08T05:59:30.213 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-08T05:59:30.213 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-08T05:59:30.213 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-08T05:59:30.213 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-08T05:59:30.213 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-08T05:59:30.213 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-08T05:59:30.213 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-08T05:59:30.213 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-08T05:59:30.214 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-08T05:59:30.214 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-08T05:59:30.214 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-08T05:59:30.214 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-08T05:59:30.214 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-08T05:59:30.214 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-08T05:59:30.214 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-08T05:59:30.214 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-08T05:59:30.214 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-08T05:59:30.214 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-08T05:59:30.214 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-08T05:59:30.214 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-08T05:59:30.214 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-08T05:59:30.215 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-08T05:59:30.215 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-08T05:59:30.215 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-08T05:59:30.215 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-08T05:59:30.215 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-08T05:59:30.215 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-08T05:59:30.215 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-08T05:59:30.215 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-08T05:59:30.215 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-08T05:59:30.215 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-08T05:59:30.215 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-08T05:59:30.215 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-08T05:59:30.215 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-08T05:59:30.215 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-08T05:59:30.215 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-08T05:59:30.216 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-08T05:59:30.216 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-08T05:59:30.216 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-08T05:59:30.216 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-08T05:59:30.216 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-08T05:59:30.216 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-08T05:59:30.216 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-08T05:59:30.216 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-08T05:59:30.216 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-08T05:59:30.216 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-08T05:59:30.216 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-08T05:59:30.216 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-08T05:59:30.216 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-08T05:59:30.216 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-08T05:59:30.217 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-08T05:59:30.217 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-08T05:59:30.217 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-08T05:59:30.217 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-08T05:59:30.217 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-08T05:59:30.217 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-08T05:59:30.217 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-08T05:59:30.217 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-08T05:59:30.217 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-08T05:59:30.217 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-08T05:59:30.217 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-08T05:59:30.217 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-08T05:59:30.217 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-08T05:59:30.217 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-08T05:59:30.217 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-08T05:59:30.218 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-08T05:59:30.218 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-08T05:59:30.218 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-08T05:59:30.218 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-08T05:59:30.218 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-08T05:59:30.218 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-08T05:59:30.218 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-08T05:59:30.218 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-08T05:59:30.218 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-08T05:59:30.218 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-08T05:59:30.218 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-08T05:59:30.218 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-08T05:59:30.218 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-08T05:59:30.218 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-08T05:59:30.219 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-08T05:59:30.219 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-08T05:59:30.219 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-08T05:59:30.219 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-08T05:59:30.219 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-08T05:59:30.219 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-08T05:59:30.219 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-08T05:59:30.219 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-08T05:59:30.219 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-08T05:59:30.219 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-08T05:59:30.219 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-08T05:59:30.219 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-08T05:59:30.219 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-08T05:59:30.219 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-08T05:59:30.220 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-08T05:59:30.220 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T05:59:30.220 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-08T05:59:30.220 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 153/198 2024-03-08T05:59:30.220 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-08T05:59:30.220 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-08T05:59:30.220 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-08T05:59:30.220 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-08T05:59:30.220 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-08T05:59:30.220 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-08T05:59:30.220 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-08T05:59:30.220 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-08T05:59:30.220 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-08T05:59:30.220 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-08T05:59:30.221 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-08T05:59:30.221 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-08T05:59:30.221 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-08T05:59:30.221 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-08T05:59:30.221 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-08T05:59:30.221 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-08T05:59:30.221 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-08T05:59:30.221 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-08T05:59:30.221 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-08T05:59:30.221 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-08T05:59:30.221 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-08T05:59:30.221 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-08T05:59:30.221 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-08T05:59:30.221 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-08T05:59:30.222 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-08T05:59:30.222 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-08T05:59:30.222 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-08T05:59:30.222 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-08T05:59:30.222 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-08T05:59:30.222 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-08T05:59:30.222 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-08T05:59:30.222 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-08T05:59:30.222 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-08T05:59:30.222 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-08T05:59:30.222 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-08T05:59:30.222 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-08T05:59:30.222 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-08T05:59:30.222 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-08T05:59:30.223 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-08T05:59:30.223 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-08T05:59:30.223 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-08T05:59:30.223 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-08T05:59:30.223 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-08T05:59:30.223 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-08T05:59:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-08T05:59:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:30.989 INFO:teuthology.orchestra.run.smithi076.stdout:Upgraded: 2024-03-08T05:59:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-08T05:59:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-08T05:59:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-08T05:59:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-08T05:59:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: acl-2.3.1-4.el9.x86_64 2024-03-08T05:59:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-08T05:59:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-08T05:59:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-08T05:59:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-08T05:59:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-08T05:59:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-08T05:59:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-08T05:59:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-08T05:59:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-08T05:59:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-08T05:59:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-08T05:59:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-08T05:59:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-08T05:59:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-08T05:59:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-08T05:59:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-08T05:59:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-08T05:59:30.991 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-03-08T05:59:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-08T05:59:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: automake-1.16.2-8.el9.noarch 2024-03-08T05:59:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: bison-3.7.4-5.el9.x86_64 2024-03-08T05:59:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-08T05:59:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-08T05:59:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: flex-2.6.4-9.el9.x86_64 2024-03-08T05:59:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-08T05:59:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-08T05:59:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-08T05:59:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-08T05:59:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-08T05:59:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-08T05:59:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: jq-1.6-16.el9.x86_64 2024-03-08T05:59:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-08T05:59:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-08T05:59:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-08T05:59:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-08T05:59:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-08T05:59:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-08T05:59:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-08T05:59:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-08T05:59:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-08T05:59:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-08T05:59:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-08T05:59:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-08T05:59:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-08T05:59:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-08T05:59:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: lua-5.4.4-4.el9.x86_64 2024-03-08T05:59:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-08T05:59:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-08T05:59:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-08T05:59:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-08T05:59:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-08T05:59:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-08T05:59:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-08T05:59:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-13.11-1.el9.x86_64 2024-03-08T05:59:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-08T05:59:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-08T05:59:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-08T05:59:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-08T05:59:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-08T05:59:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-08T05:59:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-08T05:59:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-08T05:59:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-08T05:59:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-08T05:59:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-08T05:59:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-08T05:59:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-08T05:59:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-08T05:59:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-08T05:59:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-08T05:59:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-08T05:59:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-08T05:59:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-08T05:59:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-08T05:59:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-08T05:59:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-08T05:59:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-08T05:59:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-08T05:59:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-08T05:59:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-08T05:59:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-08T05:59:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-08T05:59:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-08T05:59:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-08T05:59:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-08T05:59:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-08T05:59:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-08T05:59:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-08T05:59:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-08T05:59:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-08T05:59:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-08T05:59:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-08T05:59:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-08T05:59:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-08T05:59:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply-3.11-14.el9.noarch 2024-03-08T05:59:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-08T05:59:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-08T05:59:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-08T05:59:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-08T05:59:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-08T05:59:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-08T05:59:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-08T05:59:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-08T05:59:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-08T05:59:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-08T05:59:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-08T05:59:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-08T05:59:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-08T05:59:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-08T05:59:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-08T05:59:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-08T05:59:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-08T05:59:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-08T05:59:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-08T05:59:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-08T05:59:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-08T05:59:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-08T05:59:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-08T05:59:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-08T05:59:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-08T05:59:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-08T05:59:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-08T05:59:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: re2-1:20211101-3.el9.x86_64 2024-03-08T05:59:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-08T05:59:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-08T05:59:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-08T05:59:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-08T05:59:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-08T05:59:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-08T05:59:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:31.002 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-03-08T05:59:31.508 DEBUG:teuthology.parallel:result is None 2024-03-08T05:59:33.205 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-08T05:59:33.205 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:33.205 INFO:teuthology.orchestra.run.smithi081.stdout:Upgraded: 2024-03-08T05:59:33.205 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-08T05:59:33.205 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-08T05:59:33.205 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-08T05:59:33.205 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-08T05:59:33.205 INFO:teuthology.orchestra.run.smithi081.stdout: acl-2.3.1-4.el9.x86_64 2024-03-08T05:59:33.206 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-08T05:59:33.206 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-08T05:59:33.206 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-08T05:59:33.206 INFO:teuthology.orchestra.run.smithi081.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-08T05:59:33.206 INFO:teuthology.orchestra.run.smithi081.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-08T05:59:33.206 INFO:teuthology.orchestra.run.smithi081.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-08T05:59:33.206 INFO:teuthology.orchestra.run.smithi081.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-08T05:59:33.206 INFO:teuthology.orchestra.run.smithi081.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-08T05:59:33.206 INFO:teuthology.orchestra.run.smithi081.stdout: librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.206 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.206 INFO:teuthology.orchestra.run.smithi081.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-08T05:59:33.206 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-08T05:59:33.206 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-08T05:59:33.207 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-08T05:59:33.207 INFO:teuthology.orchestra.run.smithi081.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-08T05:59:33.207 INFO:teuthology.orchestra.run.smithi081.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-08T05:59:33.207 INFO:teuthology.orchestra.run.smithi081.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-08T05:59:33.207 INFO:teuthology.orchestra.run.smithi081.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-08T05:59:33.207 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-08T05:59:33.207 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-08T05:59:33.207 INFO:teuthology.orchestra.run.smithi081.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-08T05:59:33.207 INFO:teuthology.orchestra.run.smithi081.stdout: automake-1.16.2-8.el9.noarch 2024-03-08T05:59:33.207 INFO:teuthology.orchestra.run.smithi081.stdout: bison-3.7.4-5.el9.x86_64 2024-03-08T05:59:33.207 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-08T05:59:33.207 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.207 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.207 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.208 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.208 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:33.208 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.208 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.208 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.208 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:33.208 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:33.208 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:33.208 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:33.208 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:33.208 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.208 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.208 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:33.209 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.209 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.209 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.209 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:33.209 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:33.209 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.209 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.209 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T05:59:33.209 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-08T05:59:33.209 INFO:teuthology.orchestra.run.smithi081.stdout: flex-2.6.4-9.el9.x86_64 2024-03-08T05:59:33.209 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-08T05:59:33.209 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-08T05:59:33.209 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-08T05:59:33.209 INFO:teuthology.orchestra.run.smithi081.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-08T05:59:33.210 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-08T05:59:33.210 INFO:teuthology.orchestra.run.smithi081.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-08T05:59:33.210 INFO:teuthology.orchestra.run.smithi081.stdout: jq-1.6-16.el9.x86_64 2024-03-08T05:59:33.210 INFO:teuthology.orchestra.run.smithi081.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-08T05:59:33.210 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-08T05:59:33.210 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-08T05:59:33.210 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.210 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.210 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.210 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-08T05:59:33.210 INFO:teuthology.orchestra.run.smithi081.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-08T05:59:33.210 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-08T05:59:33.210 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-08T05:59:33.211 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-08T05:59:33.211 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.211 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.211 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-08T05:59:33.211 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.211 INFO:teuthology.orchestra.run.smithi081.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-08T05:59:33.211 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-08T05:59:33.211 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-08T05:59:33.211 INFO:teuthology.orchestra.run.smithi081.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-08T05:59:33.211 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-08T05:59:33.211 INFO:teuthology.orchestra.run.smithi081.stdout: lua-5.4.4-4.el9.x86_64 2024-03-08T05:59:33.211 INFO:teuthology.orchestra.run.smithi081.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-08T05:59:33.211 INFO:teuthology.orchestra.run.smithi081.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-08T05:59:33.211 INFO:teuthology.orchestra.run.smithi081.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-08T05:59:33.212 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-08T05:59:33.212 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-08T05:59:33.212 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-08T05:59:33.212 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-08T05:59:33.212 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-13.11-1.el9.x86_64 2024-03-08T05:59:33.212 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-08T05:59:33.212 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-08T05:59:33.212 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-08T05:59:33.212 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-08T05:59:33.212 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-08T05:59:33.212 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-08T05:59:33.212 INFO:teuthology.orchestra.run.smithi081.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-08T05:59:33.212 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-08T05:59:33.212 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-08T05:59:33.213 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-08T05:59:33.213 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.213 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.213 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.213 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-08T05:59:33.213 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-08T05:59:33.213 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-08T05:59:33.213 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-08T05:59:33.213 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-08T05:59:33.213 INFO:teuthology.orchestra.run.smithi081.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-08T05:59:33.213 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-08T05:59:33.213 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-08T05:59:33.213 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-08T05:59:33.213 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-08T05:59:33.214 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-08T05:59:33.214 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-08T05:59:33.214 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-08T05:59:33.214 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-08T05:59:33.214 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-08T05:59:33.214 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-08T05:59:33.214 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-08T05:59:33.214 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-08T05:59:33.214 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-08T05:59:33.214 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-08T05:59:33.214 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-08T05:59:33.214 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-08T05:59:33.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-08T05:59:33.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-08T05:59:33.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-08T05:59:33.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-08T05:59:33.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-08T05:59:33.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-08T05:59:33.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-08T05:59:33.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-08T05:59:33.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply-3.11-14.el9.noarch 2024-03-08T05:59:33.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-08T05:59:33.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-08T05:59:33.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-08T05:59:33.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-08T05:59:33.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-08T05:59:33.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-08T05:59:33.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-08T05:59:33.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-08T05:59:33.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-08T05:59:33.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-08T05:59:33.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-08T05:59:33.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-08T05:59:33.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-08T05:59:33.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-08T05:59:33.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-08T05:59:33.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-08T05:59:33.217 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-08T05:59:33.217 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-08T05:59:33.217 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-08T05:59:33.217 INFO:teuthology.orchestra.run.smithi081.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-08T05:59:33.217 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-08T05:59:33.217 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-08T05:59:33.217 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-08T05:59:33.217 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-08T05:59:33.217 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-08T05:59:33.217 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-08T05:59:33.217 INFO:teuthology.orchestra.run.smithi081.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-08T05:59:33.217 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.217 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.218 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T05:59:33.218 INFO:teuthology.orchestra.run.smithi081.stdout: re2-1:20211101-3.el9.x86_64 2024-03-08T05:59:33.218 INFO:teuthology.orchestra.run.smithi081.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-08T05:59:33.218 INFO:teuthology.orchestra.run.smithi081.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-08T05:59:33.218 INFO:teuthology.orchestra.run.smithi081.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-08T05:59:33.218 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-08T05:59:33.218 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-08T05:59:33.218 INFO:teuthology.orchestra.run.smithi081.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-08T05:59:33.219 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:33.219 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-08T05:59:33.662 DEBUG:teuthology.parallel:result is None 2024-03-08T05:59:33.663 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-03-08T05:59:33.962 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-08T05:59:33.986 INFO:teuthology.orchestra.run.smithi076.stdout:19.0.0-1670.g1fb5d106.el9 2024-03-08T05:59:33.987 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1670.g1fb5d106.el9 2024-03-08T05:59:33.987 INFO:teuthology.task.install:The correct ceph version 19.0.0-1670.g1fb5d106 is installed. 2024-03-08T05:59:33.989 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-03-08T05:59:34.124 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-08T05:59:34.147 INFO:teuthology.orchestra.run.smithi081.stdout:19.0.0-1670.g1fb5d106.el9 2024-03-08T05:59:34.148 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1670.g1fb5d106.el9 2024-03-08T05:59:34.148 INFO:teuthology.task.install:The correct ceph version 19.0.0-1670.g1fb5d106 is installed. 2024-03-08T05:59:34.149 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-08T05:59:34.150 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T05:59:34.150 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-08T05:59:34.184 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T05:59:34.184 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-08T05:59:34.219 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-08T05:59:34.220 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T05:59:34.220 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/daemon-helper 2024-03-08T05:59:34.257 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-08T05:59:34.336 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T05:59:34.336 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/daemon-helper 2024-03-08T05:59:34.366 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-08T05:59:34.437 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-08T05:59:34.438 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T05:59:34.438 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-08T05:59:34.470 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-08T05:59:34.541 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T05:59:34.541 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-08T05:59:34.570 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-08T05:59:34.640 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-08T05:59:34.640 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T05:59:34.640 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/stdin-killer 2024-03-08T05:59:34.670 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-08T05:59:34.739 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T05:59:34.739 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/stdin-killer 2024-03-08T05:59:34.766 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-08T05:59:34.833 INFO:teuthology.run_tasks:Running task ceph... 2024-03-08T05:59:34.915 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-08T05:59:34.915 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 777 /var/log/ceph 2024-03-08T05:59:34.917 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 777 /var/log/ceph 2024-03-08T05:59:34.946 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-08T05:59:34.946 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-08T05:59:34.988 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-08T05:59:35.015 INFO:tasks.ceph:Creating extra log directories... 2024-03-08T05:59:35.015 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-08T05:59:35.057 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-08T05:59:35.089 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-08T05:59:35.089 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-08T05:59:35.089 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-08T05:59:35.089 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T05:59:35.089 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-08T05:59:35.137 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-08T05:59:35.255 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-08T05:59:35.281 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T05:59:35.281 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-08T05:59:35.296 DEBUG:teuthology.orchestra.run.smithi081:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-08T05:59:35.415 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-08T05:59:35.446 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-08T05:59:35.447 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'lockdep': True, 'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'missing required features'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-08T05:59:35.447 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417', 'branch': 'main', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin}', 'email': 'jcollin@redhat.com', 'first_in_suite': False, 'job_id': '7585417', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'lockdep': True, 'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'missing required features'], 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'}}, 'kclient': {'syntax': 'v2'}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-jcollin-test_per-client-perf-counters', 'sha1': 'de7a7e179fbee9e314fa702f8f6c09d121c5e1a7'}}, 'owner': 'scheduled_jcollin@teuthology', 'priority': 10, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7313, 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-jcollin-test_per-client-perf-counters', 'suite_path': '/home/teuthworker/src/github.com_joscollin_ceph_de7a7e179fbee9e314fa702f8f6c09d121c5e1a7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/joscollin/ceph.git', 'suite_sha1': 'de7a7e179fbee9e314fa702f8f6c09d121c5e1a7', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi076.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKhnkFqUvrG7IuDf2BSBP5sxy4mKIsOWPSx8+7GZs+T1PXg7Ik6V+ReoC7jNDuzTxr44jT+2K9nPAyuWL4OFTgk=', 'smithi081.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBRX6/RIgJsEJ5Pga1/gJkuha9CjhD24Ar+FSY6norAtgGSq5IKJS2vFEpHLUJ/WwLDDmn/t3NCnOcTMIfvYZwc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'lockdep': True, 'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'missing required features'], 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_admin.TestLabeledPerfCounters']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-08_05:43:46', 'tube': 'smithi', 'user': 'jcollin', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-08T05:59:35.447 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-08T05:59:35.450 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-08T05:59:35.497 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/run/ceph 2024-03-08T05:59:35.505 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m0777 -- /var/run/ceph 2024-03-08T05:59:35.564 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T05:59:35.564 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/scratch_devs of=/dev/stdout 2024-03-08T05:59:35.585 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-08T05:59:35.586 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_1 2024-03-08T05:59:35.641 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-08T05:59:35.641 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T05:59:35.641 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-08T05:59:35.641 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T05:59:35.641 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T05:59:35.641 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-03-08 05:59:23.493079896 +0000 2024-03-08T05:59:35.641 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-03-08 05:57:23.566793340 +0000 2024-03-08T05:59:35.641 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-03-08 05:57:23.566793340 +0000 2024-03-08T05:59:35.641 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-03-08 05:57:23.566793340 +0000 2024-03-08T05:59:35.641 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-08T05:59:35.707 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-03-08T05:59:35.707 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-03-08T05:59:35.707 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000218073 s, 2.3 MB/s 2024-03-08T05:59:35.708 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-08T05:59:35.763 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_2 2024-03-08T05:59:35.819 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-08T05:59:35.819 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T05:59:35.819 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-08T05:59:35.819 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T05:59:35.819 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T05:59:35.819 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-03-08 05:59:23.493079896 +0000 2024-03-08T05:59:35.819 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-03-08 05:57:23.917781819 +0000 2024-03-08T05:59:35.819 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-03-08 05:57:23.917781819 +0000 2024-03-08T05:59:35.820 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-03-08 05:57:23.917781819 +0000 2024-03-08T05:59:35.820 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-08T05:59:35.884 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-03-08T05:59:35.884 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-03-08T05:59:35.884 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000172105 s, 3.0 MB/s 2024-03-08T05:59:35.885 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-08T05:59:35.940 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_3 2024-03-08T05:59:35.995 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-08T05:59:35.995 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T05:59:35.995 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-08T05:59:35.995 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T05:59:35.995 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T05:59:35.995 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-03-08 05:59:23.494079864 +0000 2024-03-08T05:59:35.995 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-03-08 05:57:24.253770791 +0000 2024-03-08T05:59:35.995 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-03-08 05:57:24.253770791 +0000 2024-03-08T05:59:35.995 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-03-08 05:57:24.253770791 +0000 2024-03-08T05:59:35.995 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-08T05:59:36.060 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-03-08T05:59:36.060 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-03-08T05:59:36.060 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000170088 s, 3.0 MB/s 2024-03-08T05:59:36.061 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-08T05:59:36.117 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_4 2024-03-08T05:59:36.172 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-08T05:59:36.172 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T05:59:36.172 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-08T05:59:36.172 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T05:59:36.172 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T05:59:36.172 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-03-08 05:59:23.494079864 +0000 2024-03-08T05:59:36.172 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-03-08 05:57:24.595759566 +0000 2024-03-08T05:59:36.172 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-03-08 05:57:24.595759566 +0000 2024-03-08T05:59:36.172 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-03-08 05:57:24.595759566 +0000 2024-03-08T05:59:36.173 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-08T05:59:36.237 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-03-08T05:59:36.238 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-03-08T05:59:36.238 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000178492 s, 2.9 MB/s 2024-03-08T05:59:36.239 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-08T05:59:36.294 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-08T05:59:36.294 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T05:59:36.295 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/scratch_devs of=/dev/stdout 2024-03-08T05:59:36.311 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-08T05:59:36.312 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_1 2024-03-08T05:59:36.367 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-08T05:59:36.368 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T05:59:36.368 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-08T05:59:36.368 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T05:59:36.368 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T05:59:36.368 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-03-08 05:59:25.612347205 +0000 2024-03-08T05:59:36.368 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-03-08 05:57:23.559679933 +0000 2024-03-08T05:59:36.368 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-03-08 05:57:23.559679933 +0000 2024-03-08T05:59:36.368 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-03-08 05:57:23.559679933 +0000 2024-03-08T05:59:36.368 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-08T05:59:36.436 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-03-08T05:59:36.436 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-03-08T05:59:36.436 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000211246 s, 2.4 MB/s 2024-03-08T05:59:36.437 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-08T05:59:36.494 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_2 2024-03-08T05:59:36.550 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-08T05:59:36.550 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T05:59:36.550 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-08T05:59:36.550 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T05:59:36.550 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T05:59:36.550 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-03-08 05:59:25.612347205 +0000 2024-03-08T05:59:36.550 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-03-08 05:57:23.883673310 +0000 2024-03-08T05:59:36.550 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-03-08 05:57:23.883673310 +0000 2024-03-08T05:59:36.550 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-03-08 05:57:23.883673310 +0000 2024-03-08T05:59:36.551 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-08T05:59:36.616 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-03-08T05:59:36.617 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-03-08T05:59:36.617 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000278504 s, 1.8 MB/s 2024-03-08T05:59:36.618 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-08T05:59:36.675 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_3 2024-03-08T05:59:36.731 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-08T05:59:36.731 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T05:59:36.731 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-08T05:59:36.731 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T05:59:36.731 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T05:59:36.731 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-03-08 05:59:25.613347185 +0000 2024-03-08T05:59:36.732 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-03-08 05:57:24.220666421 +0000 2024-03-08T05:59:36.732 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-03-08 05:57:24.220666421 +0000 2024-03-08T05:59:36.732 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-03-08 05:57:24.220666421 +0000 2024-03-08T05:59:36.732 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-08T05:59:36.798 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-03-08T05:59:36.798 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-03-08T05:59:36.798 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000228595 s, 2.2 MB/s 2024-03-08T05:59:36.800 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-08T05:59:36.857 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_4 2024-03-08T05:59:36.913 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-08T05:59:36.913 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T05:59:36.913 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-08T05:59:36.913 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T05:59:36.913 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T05:59:36.913 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-03-08 05:59:25.613347185 +0000 2024-03-08T05:59:36.913 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-03-08 05:57:24.545659778 +0000 2024-03-08T05:59:36.913 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-03-08 05:57:24.545659778 +0000 2024-03-08T05:59:36.913 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-03-08 05:57:24.545659778 +0000 2024-03-08T05:59:36.914 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-08T05:59:36.980 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-03-08T05:59:36.980 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-03-08T05:59:36.980 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000212947 s, 2.4 MB/s 2024-03-08T05:59:36.982 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-08T05:59:37.039 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-08T05:59:37.039 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi076.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi081.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-08T05:59:37.039 INFO:tasks.ceph:Generating config... 2024-03-08T05:59:37.040 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-08T05:59:37.040 INFO:tasks.ceph:[client] debug client = 20 2024-03-08T05:59:37.040 INFO:tasks.ceph:[client] debug ms = 1 2024-03-08T05:59:37.040 INFO:tasks.ceph:[client] pid file = 2024-03-08T05:59:37.040 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-08T05:59:37.040 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-08T05:59:37.040 INFO:tasks.ceph:[global] lockdep = True 2024-03-08T05:59:37.041 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-08T05:59:37.041 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-08T05:59:37.041 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-08T05:59:37.041 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-08T05:59:37.041 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-08T05:59:37.041 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-08T05:59:37.041 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-08T05:59:37.041 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-08T05:59:37.041 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-08T05:59:37.041 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-08T05:59:37.041 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-08T05:59:37.041 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-08T05:59:37.041 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-08T05:59:37.041 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-08T05:59:37.041 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-08T05:59:37.041 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-08T05:59:37.042 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-08T05:59:37.042 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-08T05:59:37.042 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-08T05:59:37.042 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-03-08T05:59:37.042 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-03-08T05:59:37.042 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-08T05:59:37.042 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-08T05:59:37.042 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-08T05:59:37.042 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-08T05:59:37.042 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-08T05:59:37.042 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-08T05:59:37.042 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-08T05:59:37.042 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-08T05:59:37.042 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-08T05:59:37.043 INFO:tasks.ceph:Setting up mon.a... 2024-03-08T05:59:37.043 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-08T05:59:37.101 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.keyring 2024-03-08T05:59:37.104 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-08T05:59:37.149 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-08T05:59:37.176 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.76'), ('mon.b', '172.21.15.81'), ('mon.c', '[v2:172.21.15.81:3301,v1:172.21.15.81:6790]')] 2024-03-08T05:59:37.176 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.76,172.21.15.81,[v2:172.21.15.81:3301,v1:172.21.15.81:6790]', 'lockdep': True, 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-08T05:59:37.177 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T05:59:37.177 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-08T05:59:37.231 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.76 --add b 172.21.15.81 --addv c '[v2:172.21.15.81:3301,v1:172.21.15.81:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-08T05:59:37.315 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-08T05:59:37.315 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool: generated fsid 8a42235b-b7c3-45b2-aa98-336e6b1846d1 2024-03-08T05:59:37.315 INFO:teuthology.orchestra.run.smithi076.stdout:setting min_mon_release = quincy 2024-03-08T05:59:37.315 INFO:teuthology.orchestra.run.smithi076.stdout:epoch 0 2024-03-08T05:59:37.315 INFO:teuthology.orchestra.run.smithi076.stdout:fsid 8a42235b-b7c3-45b2-aa98-336e6b1846d1 2024-03-08T05:59:37.315 INFO:teuthology.orchestra.run.smithi076.stdout:last_changed 2024-03-08T05:59:37.315976+0000 2024-03-08T05:59:37.315 INFO:teuthology.orchestra.run.smithi076.stdout:created 2024-03-08T05:59:37.315976+0000 2024-03-08T05:59:37.315 INFO:teuthology.orchestra.run.smithi076.stdout:min_mon_release 17 (quincy) 2024-03-08T05:59:37.316 INFO:teuthology.orchestra.run.smithi076.stdout:election_strategy: 1 2024-03-08T05:59:37.316 INFO:teuthology.orchestra.run.smithi076.stdout:0: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.a 2024-03-08T05:59:37.316 INFO:teuthology.orchestra.run.smithi076.stdout:1: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-03-08T05:59:37.316 INFO:teuthology.orchestra.run.smithi076.stdout:2: [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] mon.c 2024-03-08T05:59:37.316 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-08T05:59:37.317 DEBUG:teuthology.orchestra.run.smithi076:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-08T05:59:37.331 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8a42235b-b7c3-45b2-aa98-336e6b1846d1... 2024-03-08T05:59:37.332 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-08T05:59:37.373 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-08T05:59:37.421 INFO:teuthology.orchestra.run.smithi076.stdout:[global] 2024-03-08T05:59:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: chdir = "" 2024-03-08T05:59:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-08T05:59:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: auth supported = cephx 2024-03-08T05:59:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: filestore xattr use omap = true 2024-03-08T05:59:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: mon clock drift allowed = 1.000 2024-03-08T05:59:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: osd crush chooseleaf type = 0 2024-03-08T05:59:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: auth debug = true 2024-03-08T05:59:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on old message = true 2024-03-08T05:59:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on bug = true 2024-03-08T05:59:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-08T05:59:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: mon pg warn max object skew = 0 2024-03-08T05:59:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: # disable pg_autoscaler by default for new pools 2024-03-08T05:59:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-08T05:59:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default size = 2 2024-03-08T05:59:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow primary affinity = true 2024-03-08T05:59:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow pg remap = true 2024-03-08T05:59:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on legacy crush tunables = false 2024-03-08T05:59:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on crush straw calc version zero = false 2024-03-08T05:59:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on no sortbitwise = false 2024-03-08T05:59:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on osd down out interval zero = false 2024-03-08T05:59:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on too few osds = false 2024-03-08T05:59:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-08T05:59:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-08T05:59:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: mon_allow_pool_size_one = true 2024-03-08T05:59:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-08T05:59:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: osd default data pool replay window = 5 2024-03-08T05:59:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: mon allow pool delete = true 2024-03-08T05:59:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: mon cluster log file level = debug 2024-03-08T05:59:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: debug asserts on shutdown = true 2024-03-08T05:59:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: mon health detail to clog = false 2024-03-08T05:59:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: mon host = "172.21.15.76,172.21.15.81,[v2:172.21.15.81:3301,v1:172.21.15.81:6790]" 2024-03-08T05:59:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: lockdep = True 2024-03-08T05:59:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on skipped message = False 2024-03-08T05:59:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: fsid = 8a42235b-b7c3-45b2-aa98-336e6b1846d1 2024-03-08T05:59:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.425 INFO:teuthology.orchestra.run.smithi076.stdout:[osd] 2024-03-08T05:59:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: osd journal size = 100 2024-03-08T05:59:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub load threshold = 5.0 2024-03-08T05:59:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub max interval = 600 2024-03-08T05:59:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: osd mclock profile = high_recovery_ops 2024-03-08T05:59:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: osd recover clone overlap = true 2024-03-08T05:59:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: osd recovery max chunk = 1048576 2024-03-08T05:59:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug shutdown = true 2024-03-08T05:59:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug op order = true 2024-03-08T05:59:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug verify stray on activate = true 2024-03-08T05:59:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug trim objects = true 2024-03-08T05:59:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: osd open classes on start = true 2024-03-08T05:59:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug pg log writeout = true 2024-03-08T05:59:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: osd deep scrub update digest min age = 30 2024-03-08T05:59:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: osd map max advance = 10 2024-03-08T05:59:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: journal zero on create = true 2024-03-08T05:59:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: filestore ondisk finisher threads = 3 2024-03-08T05:59:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: filestore apply finisher threads = 3 2024-03-08T05:59:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: bdev debug aio = true 2024-03-08T05:59:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug misdirected ops = true 2024-03-08T05:59:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore block size = 96636764160 2024-03-08T05:59:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore fsck on mount = True 2024-03-08T05:59:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluefs = 20 2024-03-08T05:59:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluestore = 20 2024-03-08T05:59:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-03-08T05:59:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: debug osd = 20 2024-03-08T05:59:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: debug rocksdb = 10 2024-03-08T05:59:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd backfillfull_ratio = 0.85 2024-03-08T05:59:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd full ratio = 0.9 2024-03-08T05:59:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd nearfull ratio = 0.8 2024-03-08T05:59:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: osd failsafe full ratio = 0.95 2024-03-08T05:59:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: osd objectstore = bluestore 2024-03-08T05:59:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2024-03-08T05:59:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.428 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr] 2024-03-08T05:59:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-03-08T05:59:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: debug mgr = 20 2024-03-08T05:59:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-03-08T05:59:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-03-08T05:59:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-03-08T05:59:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-03-08T05:59:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: mgr/telemetry/nag = false 2024-03-08T05:59:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.428 INFO:teuthology.orchestra.run.smithi076.stdout:[mon] 2024-03-08T05:59:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-03-08T05:59:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-03-08T05:59:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: debug paxos = 20 2024-03-08T05:59:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-03-08T05:59:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: mon data avail warn = 5 2024-03-08T05:59:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: mon mgr mkfs grace = 240 2024-03-08T05:59:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-03-08T05:59:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd reporter subtree level = osd 2024-03-08T05:59:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd prime pg temp = true 2024-03-08T05:59:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-03-08T05:59:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-08T05:59:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: auth mon ticket ttl = 660 # 11m 2024-03-08T05:59:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: auth service ticket ttl = 240 # 4m 2024-03-08T05:59:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: # don't complain about insecure global_id in the test suite 2024-03-08T05:59:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-08T05:59:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-08T05:59:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: # 1m isn't quite enough 2024-03-08T05:59:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: mon_down_mkfs_grace = 2m 2024-03-08T05:59:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_filestore_osds = false 2024-03-08T05:59:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: mon op complaint time = 120 2024-03-08T05:59:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T05:59:37.430 INFO:teuthology.orchestra.run.smithi076.stdout:[client] 2024-03-08T05:59:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: rgw cache enabled = true 2024-03-08T05:59:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable ops log = true 2024-03-08T05:59:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable usage log = true 2024-03-08T05:59:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-08T05:59:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-08T05:59:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: client mount timeout = 600 2024-03-08T05:59:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2024-03-08T05:59:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-03-08T05:59:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = "" 2024-03-08T05:59:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2024-03-08T05:59:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2024-03-08T05:59:37.431 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.a] 2024-03-08T05:59:37.431 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.b] 2024-03-08T05:59:37.431 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.c] 2024-03-08T05:59:37.431 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2024-03-08T05:59:37.431 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2024-03-08T05:59:37.431 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2024-03-08T05:59:37.431 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2024-03-08T05:59:37.431 INFO:teuthology.orchestra.run.smithi076.stdout:[mds] 2024-03-08T05:59:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds = 20 2024-03-08T05:59:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds balancer = 20 2024-03-08T05:59:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-03-08T05:59:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug frag = True 2024-03-08T05:59:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug scatterstat = True 2024-03-08T05:59:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: mds op complaint time = 180 2024-03-08T05:59:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: mds verify scatter = True 2024-03-08T05:59:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2024-03-08T05:59:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2024-03-08T05:59:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2024-03-08T05:59:37.432 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2024-03-08T05:59:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: chdir = "" 2024-03-08T05:59:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-08T05:59:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: auth supported = cephx 2024-03-08T05:59:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: filestore xattr use omap = true 2024-03-08T05:59:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: mon clock drift allowed = 1.000 2024-03-08T05:59:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: osd crush chooseleaf type = 0 2024-03-08T05:59:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: auth debug = true 2024-03-08T05:59:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on old message = true 2024-03-08T05:59:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on bug = true 2024-03-08T05:59:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-08T05:59:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: mon pg warn max object skew = 0 2024-03-08T05:59:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: # disable pg_autoscaler by default for new pools 2024-03-08T05:59:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-08T05:59:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default size = 2 2024-03-08T05:59:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow primary affinity = true 2024-03-08T05:59:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow pg remap = true 2024-03-08T05:59:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on legacy crush tunables = false 2024-03-08T05:59:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on crush straw calc version zero = false 2024-03-08T05:59:37.435 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on no sortbitwise = false 2024-03-08T05:59:37.435 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on osd down out interval zero = false 2024-03-08T05:59:37.435 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on too few osds = false 2024-03-08T05:59:37.435 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-08T05:59:37.435 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-08T05:59:37.435 INFO:teuthology.orchestra.run.smithi081.stdout: mon_allow_pool_size_one = true 2024-03-08T05:59:37.435 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.435 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-08T05:59:37.435 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.435 INFO:teuthology.orchestra.run.smithi081.stdout: osd default data pool replay window = 5 2024-03-08T05:59:37.435 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.435 INFO:teuthology.orchestra.run.smithi081.stdout: mon allow pool delete = true 2024-03-08T05:59:37.435 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.435 INFO:teuthology.orchestra.run.smithi081.stdout: mon cluster log file level = debug 2024-03-08T05:59:37.435 INFO:teuthology.orchestra.run.smithi081.stdout: debug asserts on shutdown = true 2024-03-08T05:59:37.436 INFO:teuthology.orchestra.run.smithi081.stdout: mon health detail to clog = false 2024-03-08T05:59:37.436 INFO:teuthology.orchestra.run.smithi081.stdout: mon host = "172.21.15.76,172.21.15.81,[v2:172.21.15.81:3301,v1:172.21.15.81:6790]" 2024-03-08T05:59:37.436 INFO:teuthology.orchestra.run.smithi081.stdout: lockdep = True 2024-03-08T05:59:37.436 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on skipped message = False 2024-03-08T05:59:37.436 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = 8a42235b-b7c3-45b2-aa98-336e6b1846d1 2024-03-08T05:59:37.436 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.436 INFO:teuthology.orchestra.run.smithi081.stdout:[osd] 2024-03-08T05:59:37.436 INFO:teuthology.orchestra.run.smithi081.stdout: osd journal size = 100 2024-03-08T05:59:37.436 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.436 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub load threshold = 5.0 2024-03-08T05:59:37.436 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub max interval = 600 2024-03-08T05:59:37.436 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock profile = high_recovery_ops 2024-03-08T05:59:37.436 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.436 INFO:teuthology.orchestra.run.smithi081.stdout: osd recover clone overlap = true 2024-03-08T05:59:37.436 INFO:teuthology.orchestra.run.smithi081.stdout: osd recovery max chunk = 1048576 2024-03-08T05:59:37.437 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.437 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug shutdown = true 2024-03-08T05:59:37.437 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug op order = true 2024-03-08T05:59:37.437 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify stray on activate = true 2024-03-08T05:59:37.437 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug trim objects = true 2024-03-08T05:59:37.437 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.437 INFO:teuthology.orchestra.run.smithi081.stdout: osd open classes on start = true 2024-03-08T05:59:37.437 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug pg log writeout = true 2024-03-08T05:59:37.437 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.437 INFO:teuthology.orchestra.run.smithi081.stdout: osd deep scrub update digest min age = 30 2024-03-08T05:59:37.437 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.437 INFO:teuthology.orchestra.run.smithi081.stdout: osd map max advance = 10 2024-03-08T05:59:37.437 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.437 INFO:teuthology.orchestra.run.smithi081.stdout: journal zero on create = true 2024-03-08T05:59:37.437 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.438 INFO:teuthology.orchestra.run.smithi081.stdout: filestore ondisk finisher threads = 3 2024-03-08T05:59:37.438 INFO:teuthology.orchestra.run.smithi081.stdout: filestore apply finisher threads = 3 2024-03-08T05:59:37.438 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.438 INFO:teuthology.orchestra.run.smithi081.stdout: bdev debug aio = true 2024-03-08T05:59:37.438 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug misdirected ops = true 2024-03-08T05:59:37.438 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore block size = 96636764160 2024-03-08T05:59:37.438 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore fsck on mount = True 2024-03-08T05:59:37.438 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluefs = 20 2024-03-08T05:59:37.438 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluestore = 20 2024-03-08T05:59:37.438 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-03-08T05:59:37.438 INFO:teuthology.orchestra.run.smithi081.stdout: debug osd = 20 2024-03-08T05:59:37.438 INFO:teuthology.orchestra.run.smithi081.stdout: debug rocksdb = 10 2024-03-08T05:59:37.438 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd backfillfull_ratio = 0.85 2024-03-08T05:59:37.438 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd full ratio = 0.9 2024-03-08T05:59:37.439 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd nearfull ratio = 0.8 2024-03-08T05:59:37.439 INFO:teuthology.orchestra.run.smithi081.stdout: osd failsafe full ratio = 0.95 2024-03-08T05:59:37.439 INFO:teuthology.orchestra.run.smithi081.stdout: osd objectstore = bluestore 2024-03-08T05:59:37.439 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2024-03-08T05:59:37.439 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.439 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr] 2024-03-08T05:59:37.439 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-03-08T05:59:37.439 INFO:teuthology.orchestra.run.smithi081.stdout: debug mgr = 20 2024-03-08T05:59:37.439 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2024-03-08T05:59:37.439 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2024-03-08T05:59:37.439 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2024-03-08T05:59:37.439 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2024-03-08T05:59:37.439 INFO:teuthology.orchestra.run.smithi081.stdout: mgr/telemetry/nag = false 2024-03-08T05:59:37.439 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.439 INFO:teuthology.orchestra.run.smithi081.stdout:[mon] 2024-03-08T05:59:37.440 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-03-08T05:59:37.440 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2024-03-08T05:59:37.440 INFO:teuthology.orchestra.run.smithi081.stdout: debug paxos = 20 2024-03-08T05:59:37.440 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2024-03-08T05:59:37.440 INFO:teuthology.orchestra.run.smithi081.stdout: mon data avail warn = 5 2024-03-08T05:59:37.440 INFO:teuthology.orchestra.run.smithi081.stdout: mon mgr mkfs grace = 240 2024-03-08T05:59:37.440 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2024-03-08T05:59:37.440 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd reporter subtree level = osd 2024-03-08T05:59:37.440 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd prime pg temp = true 2024-03-08T05:59:37.440 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2024-03-08T05:59:37.440 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.440 INFO:teuthology.orchestra.run.smithi081.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-08T05:59:37.440 INFO:teuthology.orchestra.run.smithi081.stdout: auth mon ticket ttl = 660 # 11m 2024-03-08T05:59:37.440 INFO:teuthology.orchestra.run.smithi081.stdout: auth service ticket ttl = 240 # 4m 2024-03-08T05:59:37.440 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.441 INFO:teuthology.orchestra.run.smithi081.stdout: # don't complain about insecure global_id in the test suite 2024-03-08T05:59:37.441 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-08T05:59:37.441 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-08T05:59:37.441 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.441 INFO:teuthology.orchestra.run.smithi081.stdout: # 1m isn't quite enough 2024-03-08T05:59:37.441 INFO:teuthology.orchestra.run.smithi081.stdout: mon_down_mkfs_grace = 2m 2024-03-08T05:59:37.441 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.441 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_filestore_osds = false 2024-03-08T05:59:37.441 INFO:teuthology.orchestra.run.smithi081.stdout: mon op complaint time = 120 2024-03-08T05:59:37.441 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-08T05:59:37.441 INFO:teuthology.orchestra.run.smithi081.stdout:[client] 2024-03-08T05:59:37.441 INFO:teuthology.orchestra.run.smithi081.stdout: rgw cache enabled = true 2024-03-08T05:59:37.441 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable ops log = true 2024-03-08T05:59:37.441 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable usage log = true 2024-03-08T05:59:37.441 INFO:teuthology.orchestra.run.smithi081.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-08T05:59:37.442 INFO:teuthology.orchestra.run.smithi081.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-08T05:59:37.442 INFO:teuthology.orchestra.run.smithi081.stdout: client mount timeout = 600 2024-03-08T05:59:37.442 INFO:teuthology.orchestra.run.smithi081.stdout: debug client = 20 2024-03-08T05:59:37.442 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-03-08T05:59:37.442 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = "" 2024-03-08T05:59:37.442 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2024-03-08T05:59:37.442 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2024-03-08T05:59:37.442 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.a] 2024-03-08T05:59:37.442 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.b] 2024-03-08T05:59:37.442 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.c] 2024-03-08T05:59:37.442 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2024-03-08T05:59:37.442 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2024-03-08T05:59:37.442 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2024-03-08T05:59:37.442 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2024-03-08T05:59:37.442 INFO:teuthology.orchestra.run.smithi081.stdout:[mds] 2024-03-08T05:59:37.443 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds = 20 2024-03-08T05:59:37.443 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds balancer = 20 2024-03-08T05:59:37.443 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-03-08T05:59:37.443 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug frag = True 2024-03-08T05:59:37.443 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug scatterstat = True 2024-03-08T05:59:37.443 INFO:teuthology.orchestra.run.smithi081.stdout: mds op complaint time = 180 2024-03-08T05:59:37.443 INFO:teuthology.orchestra.run.smithi081.stdout: mds verify scatter = True 2024-03-08T05:59:37.443 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2024-03-08T05:59:37.443 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2024-03-08T05:59:37.443 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2024-03-08T05:59:37.444 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-08T05:59:37.445 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-08T05:59:37.540 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-08T05:59:37.541 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T05:59:37.541 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-08T05:59:37.556 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T05:59:37.556 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-08T05:59:37.611 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi076.front.sepia.ceph.com 2024-03-08T05:59:37.612 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T05:59:37.612 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-08T05:59:37.612 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-08T05:59:37.691 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T05:59:37.691 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-08T05:59:37.747 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi081.front.sepia.ceph.com 2024-03-08T05:59:37.747 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T05:59:37.747 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-08T05:59:37.747 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-08T05:59:37.789 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T05:59:37.789 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-08T05:59:37.845 INFO:tasks.ceph:Setting up mon nodes... 2024-03-08T05:59:37.846 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-08T05:59:37.846 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-08T05:59:37.901 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-08T05:59:37.904 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-08T05:59:37.965 INFO:teuthology.orchestra.run.smithi081.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-08T05:59:37.967 INFO:tasks.ceph:Setting up mds nodes... 2024-03-08T05:59:37.968 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-08T05:59:38.022 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-08T05:59:38.025 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-08T05:59:38.052 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-08T05:59:38.146 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-08T05:59:38.149 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-08T05:59:38.175 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-08T05:59:38.230 INFO:teuthology.orchestra.run.smithi081.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-08T05:59:38.232 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-08T05:59:38.259 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-08T05:59:38.353 INFO:teuthology.orchestra.run.smithi081.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-08T05:59:38.356 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-08T05:59:38.383 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-08T05:59:38.383 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-08T05:59:38.425 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-08T05:59:38.441 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-08T05:59:38.523 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-08T05:59:38.539 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-08T05:59:38.581 INFO:teuthology.orchestra.run.smithi081.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-08T05:59:38.596 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-08T05:59:38.678 INFO:teuthology.orchestra.run.smithi081.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-08T05:59:38.693 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-08T05:59:38.693 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi076.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi081.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-08T05:59:38.693 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-08T05:59:38.720 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-08T05:59:38.720 INFO:tasks.ceph:role: osd.0 2024-03-08T05:59:38.720 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi076.front.sepia.ceph.com 2024-03-08T05:59:38.720 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-08T05:59:38.794 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T05:59:38.795 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T05:59:38.795 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T05:59:38.795 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T05:59:38.795 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T05:59:38.795 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-03-08T05:59:38.795 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T05:59:38.795 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T05:59:38.796 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T05:59:38.796 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T05:59:39.308 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-03-08T05:59:39.310 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-03-08T05:59:39.310 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-08T05:59:39.454 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-08T05:59:39.488 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-08T05:59:39.556 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-08T05:59:39.556 INFO:tasks.ceph:role: osd.1 2024-03-08T05:59:39.556 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi076.front.sepia.ceph.com 2024-03-08T05:59:39.556 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-08T05:59:39.631 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T05:59:39.631 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T05:59:39.631 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T05:59:39.631 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T05:59:39.631 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T05:59:39.631 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-03-08T05:59:39.631 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T05:59:39.631 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T05:59:39.632 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T05:59:39.632 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T05:59:40.080 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-03-08T05:59:40.082 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-03-08T05:59:40.082 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-08T05:59:40.166 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-08T05:59:40.238 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-08T05:59:40.305 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-08T05:59:40.305 INFO:tasks.ceph:role: osd.2 2024-03-08T05:59:40.305 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi076.front.sepia.ceph.com 2024-03-08T05:59:40.306 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-08T05:59:40.379 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T05:59:40.379 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T05:59:40.379 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T05:59:40.379 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T05:59:40.379 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T05:59:40.379 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-03-08T05:59:40.379 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T05:59:40.379 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T05:59:40.379 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T05:59:40.380 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T05:59:40.866 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-03-08T05:59:40.868 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-03-08T05:59:40.868 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-08T05:59:41.007 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-08T05:59:41.039 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-08T05:59:41.106 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-08T05:59:41.106 INFO:tasks.ceph:role: osd.3 2024-03-08T05:59:41.106 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi076.front.sepia.ceph.com 2024-03-08T05:59:41.106 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-08T05:59:41.180 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T05:59:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T05:59:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T05:59:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T05:59:41.181 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T05:59:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-03-08T05:59:41.181 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T05:59:41.181 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T05:59:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T05:59:41.181 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T05:59:41.694 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-03-08T05:59:41.695 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-03-08T05:59:41.695 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-08T05:59:41.825 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-08T05:59:41.858 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T05:59:41.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:41.954+0000 7fb7ad745640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-08T05:59:41.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:41.954+0000 7fb7ad745640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-08T05:59:41.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:41.954+0000 7fb7ad745640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-08T05:59:41.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:41.954+0000 7fb7ad745640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-08T05:59:41.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:41.954+0000 7fb7ad745640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-08T05:59:41.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:41.955+0000 7fb7ad745640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-08T05:59:44.811 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-08T05:59:44.844 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T05:59:44.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:44.938+0000 7f32b3a17640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-08T05:59:44.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:44.938+0000 7f32b3a17640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-08T05:59:44.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:44.938+0000 7f32b3a17640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-08T05:59:44.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:44.938+0000 7f32b3a17640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-08T05:59:44.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:44.938+0000 7f32b3a17640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-08T05:59:44.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:44.939+0000 7f32b3a17640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-08T05:59:47.788 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-08T05:59:47.820 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T05:59:47.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:47.912+0000 7f3d9c5dd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-08T05:59:47.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:47.913+0000 7f3d9c5dd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-08T05:59:47.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:47.913+0000 7f3d9c5dd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-08T05:59:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:47.913+0000 7f3d9c5dd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-08T05:59:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:47.913+0000 7f3d9c5dd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-08T05:59:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:47.913+0000 7f3d9c5dd640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-08T05:59:50.764 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-08T05:59:50.796 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T05:59:50.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:50.891+0000 7f477bcf9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-08T05:59:50.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:50.892+0000 7f477bcf9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-08T05:59:50.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:50.892+0000 7f477bcf9640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-08T05:59:50.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:50.892+0000 7f477bcf9640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-08T05:59:50.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:50.892+0000 7f477bcf9640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-08T05:59:50.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T05:59:50.892+0000 7f477bcf9640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-08T05:59:53.743 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-08T05:59:53.774 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-08T05:59:53.806 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-08T05:59:53.806 INFO:tasks.ceph:role: osd.4 2024-03-08T05:59:53.807 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi081.front.sepia.ceph.com 2024-03-08T05:59:53.807 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-08T05:59:53.884 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T05:59:53.884 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T05:59:53.884 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T05:59:53.884 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T05:59:53.884 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T05:59:53.884 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-03-08T05:59:53.884 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T05:59:53.885 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T05:59:53.885 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T05:59:53.885 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T05:59:54.373 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2024-03-08T05:59:54.375 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-03-08T05:59:54.375 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-08T05:59:54.487 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-08T05:59:54.524 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-08T05:59:54.593 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-08T05:59:54.593 INFO:tasks.ceph:role: osd.5 2024-03-08T05:59:54.593 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi081.front.sepia.ceph.com 2024-03-08T05:59:54.593 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-08T05:59:54.668 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T05:59:54.669 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T05:59:54.669 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T05:59:54.669 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T05:59:54.669 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T05:59:54.669 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-03-08T05:59:54.669 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T05:59:54.669 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T05:59:54.669 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T05:59:54.669 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T05:59:55.116 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2024-03-08T05:59:55.119 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-03-08T05:59:55.119 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-08T05:59:55.196 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-08T05:59:55.267 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-08T05:59:55.334 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-08T05:59:55.334 INFO:tasks.ceph:role: osd.6 2024-03-08T05:59:55.334 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi081.front.sepia.ceph.com 2024-03-08T05:59:55.335 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-08T05:59:55.408 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T05:59:55.408 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T05:59:55.408 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T05:59:55.408 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T05:59:55.408 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T05:59:55.409 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-03-08T05:59:55.409 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T05:59:55.409 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T05:59:55.409 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T05:59:55.409 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T05:59:55.912 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2024-03-08T05:59:55.914 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-03-08T05:59:55.914 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-08T05:59:56.012 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-08T05:59:56.044 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-08T05:59:56.111 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-08T05:59:56.111 INFO:tasks.ceph:role: osd.7 2024-03-08T05:59:56.111 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi081.front.sepia.ceph.com 2024-03-08T05:59:56.111 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-08T05:59:56.185 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T05:59:56.186 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T05:59:56.186 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T05:59:56.186 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T05:59:56.186 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T05:59:56.186 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-03-08T05:59:56.186 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T05:59:56.186 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T05:59:56.186 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T05:59:56.186 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T05:59:56.702 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2024-03-08T05:59:56.704 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-03-08T05:59:56.704 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-08T05:59:56.852 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-08T05:59:56.884 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T05:59:56.990 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T05:59:56.989+0000 7fa447610640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-08T05:59:56.990 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T05:59:56.989+0000 7fa447610640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-08T05:59:56.990 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T05:59:56.989+0000 7fa447610640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-08T05:59:56.990 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T05:59:56.989+0000 7fa447610640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-08T05:59:56.990 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T05:59:56.989+0000 7fa447610640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-08T05:59:56.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T05:59:56.992+0000 7fa447610640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-08T05:59:59.843 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-08T05:59:59.874 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T05:59:59.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T05:59:59.963+0000 7ff2b7aa4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-08T05:59:59.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T05:59:59.963+0000 7ff2b7aa4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-08T05:59:59.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T05:59:59.963+0000 7ff2b7aa4640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-08T05:59:59.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T05:59:59.963+0000 7ff2b7aa4640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-08T05:59:59.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T05:59:59.963+0000 7ff2b7aa4640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-08T05:59:59.965 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T05:59:59.964+0000 7ff2b7aa4640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-08T06:00:02.803 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-08T06:00:02.832 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:00:02.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:02.918+0000 7f7d69c0b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-08T06:00:02.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:02.918+0000 7f7d69c0b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-08T06:00:02.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:02.918+0000 7f7d69c0b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-08T06:00:02.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:02.918+0000 7f7d69c0b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-08T06:00:02.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:02.918+0000 7f7d69c0b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-08T06:00:02.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:02.919+0000 7f7d69c0b640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-08T06:00:05.449 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:00:05.452 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:00:05.763 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-08T06:00:05.791 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:00:05.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:05.877+0000 7f7f9d6b1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-08T06:00:05.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:05.877+0000 7f7f9d6b1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-08T06:00:05.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:05.877+0000 7f7f9d6b1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-08T06:00:05.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:05.877+0000 7f7f9d6b1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-08T06:00:05.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:05.877+0000 7f7f9d6b1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-08T06:00:05.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:05.878+0000 7f7f9d6b1640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-08T06:00:08.722 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-08T06:00:08.753 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-08T06:00:08.753 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:00:08.753 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-08T06:00:08.785 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:00:08.785 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-08T06:00:08.856 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:00:08.856 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-08T06:00:08.925 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:00:08.925 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-08T06:00:08.993 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:00:08.993 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-08T06:00:09.061 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:00:09.061 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-08T06:00:09.129 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:00:09.129 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-08T06:00:09.196 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:00:09.196 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-08T06:00:09.228 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:00:09.228 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-08T06:00:09.298 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:00:09.298 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-08T06:00:09.367 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:00:09.367 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-08T06:00:09.435 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:00:09.435 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-08T06:00:09.503 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:00:09.503 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-08T06:00:09.571 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:00:09.571 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-08T06:00:09.638 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:00:09.638 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-08T06:00:09.654 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:00:09.654 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-08T06:00:09.710 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:00:09.710 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-08T06:00:09.724 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:00:09.725 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-08T06:00:09.779 INFO:tasks.ceph:Adding keys to all mons... 2024-03-08T06:00:09.779 DEBUG:teuthology.orchestra.run.smithi076:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-08T06:00:09.781 DEBUG:teuthology.orchestra.run.smithi081:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-08T06:00:09.822 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr.y] 2024-03-08T06:00:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBJqeplOJ3CNRAA6hv+A1Bmwarz4eo2wqVNZQ== 2024-03-08T06:00:09.822 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2024-03-08T06:00:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBKqeplZeJjARAAlfjyyegTL7rXiO5oZlnk0w== 2024-03-08T06:00:09.822 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2024-03-08T06:00:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBKqepljg3KCBAAloYX61Kt+Mo/LxsQ6ZWbkA== 2024-03-08T06:00:09.822 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.0] 2024-03-08T06:00:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBNqeplwpn1OBAAACkPVBFCXbuovt8Eu6gswQ== 2024-03-08T06:00:09.822 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.1] 2024-03-08T06:00:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBQqeplSbr8NxAASBqBdSC3wDAkaXnJ4aveEg== 2024-03-08T06:00:09.823 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.2] 2024-03-08T06:00:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBTqepl9kB2NhAArI6PJK6a9jPuby86K+sviA== 2024-03-08T06:00:09.823 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.3] 2024-03-08T06:00:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBWqeplLwQ4NRAAjR9l83jSoKb8cJrIBcxZMg== 2024-03-08T06:00:09.823 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr.x] 2024-03-08T06:00:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBJqeplqvWVORAAUysSLEF4wAs62maeFdG60w== 2024-03-08T06:00:09.823 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2024-03-08T06:00:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBKqepla2TDDRAAKz8pGbF08Fj6g6wH9plqEQ== 2024-03-08T06:00:09.823 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2024-03-08T06:00:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBKqepl8AMiFRAAzfoDnTWrB8kMbPrjIhseVg== 2024-03-08T06:00:09.823 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.4] 2024-03-08T06:00:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBcqeplqQsLOxAABHoIXSPMqfFZK0rZmV8GbQ== 2024-03-08T06:00:09.823 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.5] 2024-03-08T06:00:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBfqepl+o58ORAAC0EruVCdKFdUokPiKS8OAA== 2024-03-08T06:00:09.823 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.6] 2024-03-08T06:00:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBiqeplabHNNhAA7Uso+ALm1TwwkbtUVkmMvw== 2024-03-08T06:00:09.824 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.7] 2024-03-08T06:00:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBlqepl16BeNBAAkHN6ceewaEZzsTZcqTqmDA== 2024-03-08T06:00:09.824 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2024-03-08T06:00:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBKqepl3elrGRAAXLq7DBlUer23iSXOoBZatA== 2024-03-08T06:00:09.824 INFO:teuthology.orchestra.run.smithi076.stdout:[client.2] 2024-03-08T06:00:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBKqepls1FEHxAA+hIAqCWzjGxFRYdAgdDRQw== 2024-03-08T06:00:09.824 INFO:teuthology.orchestra.run.smithi076.stdout:[client.1] 2024-03-08T06:00:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBKqeplhteqIhAAuwfBLYJRuYCaz3BTclKqwg== 2024-03-08T06:00:09.824 INFO:teuthology.orchestra.run.smithi076.stdout:[client.3] 2024-03-08T06:00:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBKqepl38B0KBAAwHBqLkwC4Sl9N9jDYFvfUg== 2024-03-08T06:00:09.844 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr.y] 2024-03-08T06:00:09.844 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBJqeplOJ3CNRAA6hv+A1Bmwarz4eo2wqVNZQ== 2024-03-08T06:00:09.844 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2024-03-08T06:00:09.844 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBKqeplZeJjARAAlfjyyegTL7rXiO5oZlnk0w== 2024-03-08T06:00:09.844 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2024-03-08T06:00:09.844 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBKqepljg3KCBAAloYX61Kt+Mo/LxsQ6ZWbkA== 2024-03-08T06:00:09.844 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.0] 2024-03-08T06:00:09.844 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBNqeplwpn1OBAAACkPVBFCXbuovt8Eu6gswQ== 2024-03-08T06:00:09.844 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.1] 2024-03-08T06:00:09.844 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBQqeplSbr8NxAASBqBdSC3wDAkaXnJ4aveEg== 2024-03-08T06:00:09.844 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.2] 2024-03-08T06:00:09.845 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBTqepl9kB2NhAArI6PJK6a9jPuby86K+sviA== 2024-03-08T06:00:09.845 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.3] 2024-03-08T06:00:09.845 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBWqeplLwQ4NRAAjR9l83jSoKb8cJrIBcxZMg== 2024-03-08T06:00:09.845 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr.x] 2024-03-08T06:00:09.845 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBJqeplqvWVORAAUysSLEF4wAs62maeFdG60w== 2024-03-08T06:00:09.845 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2024-03-08T06:00:09.845 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBKqepla2TDDRAAKz8pGbF08Fj6g6wH9plqEQ== 2024-03-08T06:00:09.845 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2024-03-08T06:00:09.845 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBKqepl8AMiFRAAzfoDnTWrB8kMbPrjIhseVg== 2024-03-08T06:00:09.845 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.4] 2024-03-08T06:00:09.845 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBcqeplqQsLOxAABHoIXSPMqfFZK0rZmV8GbQ== 2024-03-08T06:00:09.845 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.5] 2024-03-08T06:00:09.845 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBfqepl+o58ORAAC0EruVCdKFdUokPiKS8OAA== 2024-03-08T06:00:09.845 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.6] 2024-03-08T06:00:09.845 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBiqeplabHNNhAA7Uso+ALm1TwwkbtUVkmMvw== 2024-03-08T06:00:09.846 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.7] 2024-03-08T06:00:09.846 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBlqepl16BeNBAAkHN6ceewaEZzsTZcqTqmDA== 2024-03-08T06:00:09.846 INFO:teuthology.orchestra.run.smithi081.stdout:[client.0] 2024-03-08T06:00:09.846 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBKqepl3elrGRAAXLq7DBlUer23iSXOoBZatA== 2024-03-08T06:00:09.846 INFO:teuthology.orchestra.run.smithi081.stdout:[client.2] 2024-03-08T06:00:09.846 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBKqepls1FEHxAA+hIAqCWzjGxFRYdAgdDRQw== 2024-03-08T06:00:09.846 INFO:teuthology.orchestra.run.smithi081.stdout:[client.1] 2024-03-08T06:00:09.846 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBKqeplhteqIhAAuwfBLYJRuYCaz3BTclKqwg== 2024-03-08T06:00:09.846 INFO:teuthology.orchestra.run.smithi081.stdout:[client.3] 2024-03-08T06:00:09.846 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBKqepl38B0KBAAwHBqLkwC4Sl9N9jDYFvfUg== 2024-03-08T06:00:09.847 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-08T06:00:09.867 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-08T06:00:09.940 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T06:00:09.942 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T06:00:09.996 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T06:00:09.999 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T06:00:10.053 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:00:10.055 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:00:10.106 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:00:10.108 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:00:10.159 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:00:10.162 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:00:10.213 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:00:10.215 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:00:10.265 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-08T06:00:10.267 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-08T06:00:10.318 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T06:00:10.320 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T06:00:10.375 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T06:00:10.377 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T06:00:10.431 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:00:10.433 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:00:10.485 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:00:10.486 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:00:10.538 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:00:10.540 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:00:10.591 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:00:10.593 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:00:10.645 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T06:00:10.647 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T06:00:10.701 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T06:00:10.703 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T06:00:10.758 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T06:00:10.760 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T06:00:10.815 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T06:00:10.817 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T06:00:10.871 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-08T06:00:10.871 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-08T06:00:10.898 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-08T06:00:10.995 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-08T06:00:11.021 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-08T06:00:11.048 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-08T06:00:11.144 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-08T06:00:11.170 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-08T06:00:11.236 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-08T06:00:11.331 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-08T06:00:11.357 DEBUG:teuthology.orchestra.run.smithi076:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:00:11.360 DEBUG:teuthology.orchestra.run.smithi081:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:00:11.412 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-08T06:00:11.412 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-08T06:00:11.413 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-08T06:00:11.415 INFO:tasks.ceph.mon.a:Started 2024-03-08T06:00:11.415 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-08T06:00:11.415 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-08T06:00:11.454 INFO:tasks.ceph.mon.b:Started 2024-03-08T06:00:11.454 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-08T06:00:11.455 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-08T06:00:11.456 INFO:tasks.ceph.mon.c:Started 2024-03-08T06:00:11.456 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-08T06:00:11.457 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-08T06:00:11.457 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-08T06:00:11.459 INFO:tasks.ceph.mgr.y:Started 2024-03-08T06:00:11.459 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-08T06:00:11.459 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-08T06:00:11.461 INFO:tasks.ceph.mgr.x:Started 2024-03-08T06:00:11.461 INFO:tasks.ceph:Setting crush tunables to default 2024-03-08T06:00:11.461 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd crush tunables default 2024-03-08T06:00:11.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.561+0000 7f24f2c46640 1 Processor -- start 2024-03-08T06:00:11.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.561+0000 7f24f2c46640 1 -- start start 2024-03-08T06:00:11.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.562+0000 7f24f2c46640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f24ec0cfbb0 0x7f24ec06b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:11.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.562+0000 7f24f2c46640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f24ec15e1b0 0x7f24ec0cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:11.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.562+0000 7f24f2c46640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ec0cb360 con 0x7f24ec0caee0 2024-03-08T06:00:11.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.562+0000 7f24f2c46640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ec15e9d0 con 0x7f24ec0cfbb0 2024-03-08T06:00:11.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.562+0000 7f24f2c46640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ec15ecb0 con 0x7f24ec15e1b0 2024-03-08T06:00:11.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.562+0000 7f24f09e2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f24ec0cfbb0 0x7f24ec06b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:11.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.562+0000 7f24ebfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f24ec0caee0 0x7f24ec0cff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53950/0 (socket says 172.21.15.76:53950) 2024-03-08T06:00:11.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.562+0000 7f24ebfff640 1 -- 172.21.15.76:0/1746886061 learned_addr learned my addr 172.21.15.76:0/1746886061 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:11.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.562+0000 7f24f09e2640 1 -- 172.21.15.76:0/1746886061 >> v2:172.21.15.76:3300/0 conn(0x7f24ec0cfbb0 msgr2=0x7f24ec06b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-08T06:00:11.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.562+0000 7f24f09e2640 1 -- 172.21.15.76:0/1746886061 >> v2:172.21.15.76:3300/0 conn(0x7f24ec0cfbb0 msgr2=0x7f24ec06b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-08T06:00:11.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.562+0000 7f24f09e2640 1 --2- 172.21.15.76:0/1746886061 >> v2:172.21.15.76:3300/0 conn(0x7f24ec0cfbb0 0x7f24ec06b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-08T06:00:11.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.562+0000 7f24f09e2640 1 --2- 172.21.15.76:0/1746886061 >> v2:172.21.15.76:3300/0 conn(0x7f24ec0cfbb0 0x7f24ec06b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T06:00:11.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.562+0000 7f24f11e3640 1 --2- 172.21.15.76:0/1746886061 >> v2:172.21.15.81:3300/0 conn(0x7f24ec15e1b0 0x7f24ec0cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:11.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.563+0000 7f24f11e3640 1 -- 172.21.15.76:0/1746886061 >> v2:172.21.15.81:3300/0 conn(0x7f24ec15e1b0 msgr2=0x7f24ec0cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-08T06:00:11.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.563+0000 7f24f11e3640 1 -- 172.21.15.76:0/1746886061 >> v2:172.21.15.81:3300/0 conn(0x7f24ec15e1b0 msgr2=0x7f24ec0cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-08T06:00:11.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.563+0000 7f24f11e3640 1 --2- 172.21.15.76:0/1746886061 >> v2:172.21.15.81:3300/0 conn(0x7f24ec15e1b0 0x7f24ec0cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-08T06:00:11.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.563+0000 7f24f11e3640 1 --2- 172.21.15.76:0/1746886061 >> v2:172.21.15.81:3300/0 conn(0x7f24ec15e1b0 0x7f24ec0cd4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T06:00:11.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24eb7fe640 1 -- 172.21.15.76:0/1746886061 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154003024 0 0) 0x7f24ec0cb360 con 0x7f24ec0caee0 2024-03-08T06:00:11.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24eb7fe640 1 -- 172.21.15.76:0/1746886061 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24d4003520 con 0x7f24ec0caee0 2024-03-08T06:00:11.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24eb7fe640 1 -- 172.21.15.76:0/1746886061 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1521106601 0 0) 0x7f24d4003520 con 0x7f24ec0caee0 2024-03-08T06:00:11.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24eb7fe640 1 -- 172.21.15.76:0/1746886061 >> v2:172.21.15.81:3300/0 conn(0x7f24ec15e1b0 msgr2=0x7f24ec0cd4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-08T06:00:11.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24eb7fe640 1 --2- 172.21.15.76:0/1746886061 >> v2:172.21.15.81:3300/0 conn(0x7f24ec15e1b0 0x7f24ec0cd4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:11.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24eb7fe640 1 -- 172.21.15.76:0/1746886061 >> v2:172.21.15.76:3300/0 conn(0x7f24ec0cfbb0 msgr2=0x7f24ec06b690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-08T06:00:11.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24eb7fe640 1 --2- 172.21.15.76:0/1746886061 >> v2:172.21.15.76:3300/0 conn(0x7f24ec0cfbb0 0x7f24ec06b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:11.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24eb7fe640 1 -- 172.21.15.76:0/1746886061 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec15ef90 con 0x7f24ec0caee0 2024-03-08T06:00:11.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24eb7fe640 1 -- 172.21.15.76:0/1746886061 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f24e0001670 con 0x7f24ec0caee0 2024-03-08T06:00:11.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24eb7fe640 1 -- 172.21.15.76:0/1746886061 >> v1:172.21.15.76:6789/0 conn(0x7f24ec0caee0 legacy=0x7f24ec0cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:11.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24eb7fe640 1 --2- 172.21.15.76:0/1746886061 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f24d4003a20 0x7f24d4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:11.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24eb7fe640 1 --2- 172.21.15.76:0/1746886061 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24d4004800 0x7f24d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:11.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24eb7fe640 1 --2- 172.21.15.76:0/1746886061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24d40051c0 0x7f24d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:11.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24eb7fe640 1 -- 172.21.15.76:0/1746886061 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec15e9d0 con 0x7f24d4004800 2024-03-08T06:00:11.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24eb7fe640 1 -- 172.21.15.76:0/1746886061 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec15ecb0 con 0x7f24d40051c0 2024-03-08T06:00:11.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24eb7fe640 1 -- 172.21.15.76:0/1746886061 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d400c140 con 0x7f24d4003a20 2024-03-08T06:00:11.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24ebfff640 1 --2- 172.21.15.76:0/1746886061 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24d4004800 0x7f24d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:11.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24f11e3640 1 --2- 172.21.15.76:0/1746886061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24d40051c0 0x7f24d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:11.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.760+0000 7f24f09e2640 1 --2- 172.21.15.76:0/1746886061 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f24d4003a20 0x7f24d4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:11.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.761+0000 7f24ebfff640 1 -- 172.21.15.76:0/1746886061 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f24d4003a20 msgr2=0x7f24d4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:11.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.761+0000 7f24ebfff640 1 --2- 172.21.15.76:0/1746886061 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f24d4003a20 0x7f24d4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:11.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.761+0000 7f24ebfff640 1 -- 172.21.15.76:0/1746886061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24d40051c0 msgr2=0x7f24d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:11.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.761+0000 7f24ebfff640 1 --2- 172.21.15.76:0/1746886061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24d40051c0 0x7f24d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:11.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.761+0000 7f24f11e3640 1 -- 172.21.15.76:0/1746886061 reap_dead start 2024-03-08T06:00:11.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.761+0000 7f24ebfff640 1 -- 172.21.15.76:0/1746886061 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d4003520 con 0x7f24d4004800 2024-03-08T06:00:11.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.761+0000 7f24ebfff640 1 --2- 172.21.15.76:0/1746886061 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24d4004800 0x7f24d4004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f24ec06bbd0 tx=0x7f24d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad13d9cd3069753 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:11.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.761+0000 7f24eb7fe640 1 -- 172.21.15.76:0/1746886061 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0007d50 con 0x7f24d4004800 2024-03-08T06:00:11.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.761+0000 7f24eb7fe640 1 -- 172.21.15.76:0/1746886061 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e0030070 con 0x7f24d4004800 2024-03-08T06:00:11.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.761+0000 7f24eb7fe640 1 -- 172.21.15.76:0/1746886061 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0005d10 con 0x7f24d4004800 2024-03-08T06:00:11.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.762+0000 7f24f2c46640 1 -- 172.21.15.76:0/1746886061 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24d4004800 msgr2=0x7f24d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:11.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.762+0000 7f24f2c46640 1 --2- 172.21.15.76:0/1746886061 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24d4004800 0x7f24d4004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f24ec06bbd0 tx=0x7f24d4005830 comp rx=0 tx=0).stop 2024-03-08T06:00:11.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.762+0000 7f24f2c46640 1 -- 172.21.15.76:0/1746886061 shutdown_connections 2024-03-08T06:00:11.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.762+0000 7f24f2c46640 1 --2- 172.21.15.76:0/1746886061 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24d4004800 0x7f24d4004c80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:11.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.762+0000 7f24f2c46640 1 -- 172.21.15.76:0/1746886061 >> 172.21.15.76:0/1746886061 conn(0x7f24ec0d0890 msgr2=0x7f24ec0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:11.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.762+0000 7f24f2c46640 1 -- 172.21.15.76:0/1746886061 shutdown_connections 2024-03-08T06:00:11.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.762+0000 7f24f2c46640 1 -- 172.21.15.76:0/1746886061 wait complete. 2024-03-08T06:00:11.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.763+0000 7f24f2c46640 1 Processor -- start 2024-03-08T06:00:11.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.764+0000 7f24f2c46640 1 -- start start 2024-03-08T06:00:11.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.764+0000 7f24f2c46640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24d4004800 0x7f24ec1f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:11.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.764+0000 7f24f2c46640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24ec0b4970 0x7f24ec14f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:11.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.764+0000 7f24f2c46640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f24ec0b4df0 0x7f24ec14fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:11.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.764+0000 7f24f2c46640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec0ccb70 con 0x7f24d4004800 2024-03-08T06:00:11.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.764+0000 7f24f2c46640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec15e660 con 0x7f24ec0b4970 2024-03-08T06:00:11.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.764+0000 7f24f2c46640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec0cccf0 con 0x7f24ec0b4df0 2024-03-08T06:00:11.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.764+0000 7f24f09e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24d4004800 0x7f24ec1f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:11.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24f09e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24d4004800 0x7f24ec1f3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37648/0 (socket says 172.21.15.76:37648) 2024-03-08T06:00:11.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24f09e2640 1 -- 172.21.15.76:0/3491459748 learned_addr learned my addr 172.21.15.76:0/3491459748 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:11.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24ebfff640 1 --2- 172.21.15.76:0/3491459748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24ec0b4970 0x7f24ec14f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:11.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24f11e3640 1 --2- 172.21.15.76:0/3491459748 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f24ec0b4df0 0x7f24ec14fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:11.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24f09e2640 1 -- 172.21.15.76:0/3491459748 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f24ec0b4df0 msgr2=0x7f24ec14fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:11.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24f09e2640 1 --2- 172.21.15.76:0/3491459748 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f24ec0b4df0 0x7f24ec14fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:11.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24f09e2640 1 -- 172.21.15.76:0/3491459748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24ec0b4970 msgr2=0x7f24ec14f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:11.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24f09e2640 1 --2- 172.21.15.76:0/3491459748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24ec0b4970 0x7f24ec14f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:11.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24f09e2640 1 -- 172.21.15.76:0/3491459748 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec0b5330 con 0x7f24d4004800 2024-03-08T06:00:11.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24f09e2640 1 --2- 172.21.15.76:0/3491459748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24d4004800 0x7f24ec1f3aa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00f5a0 tx=0x7f24dc00fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:11.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24e97fa640 1 -- 172.21.15.76:0/3491459748 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc019020 con 0x7f24d4004800 2024-03-08T06:00:11.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24e97fa640 1 -- 172.21.15.76:0/3491459748 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24dc014870 con 0x7f24d4004800 2024-03-08T06:00:11.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24f2c46640 1 -- 172.21.15.76:0/3491459748 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24ec0b5610 con 0x7f24d4004800 2024-03-08T06:00:11.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24e97fa640 1 -- 172.21.15.76:0/3491459748 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc014c20 con 0x7f24d4004800 2024-03-08T06:00:11.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24f2c46640 1 -- 172.21.15.76:0/3491459748 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ec15c430 con 0x7f24d4004800 2024-03-08T06:00:11.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24e97fa640 1 -- 172.21.15.76:0/3491459748 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f24dc01c020 con 0x7f24d4004800 2024-03-08T06:00:11.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.765+0000 7f24e97fa640 1 -- 172.21.15.76:0/3491459748 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f24dc01e440 con 0x7f24d4004800 2024-03-08T06:00:11.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.766+0000 7f24f2c46640 1 -- 172.21.15.76:0/3491459748 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24ec04f030 con 0x7f24d4004800 2024-03-08T06:00:11.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.768+0000 7f24e97fa640 1 -- 172.21.15.76:0/3491459748 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f24dc023020 con 0x7f24d4004800 2024-03-08T06:00:11.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.834+0000 7f24f2c46640 1 -- 172.21.15.76:0/3491459748 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f24ec15a3c0 con 0x7f24d4004800 2024-03-08T06:00:11.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.835+0000 7f24e97fa640 1 -- 172.21.15.76:0/3491459748 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f24dc01e6e0 con 0x7f24d4004800 2024-03-08T06:00:11.836 INFO:teuthology.orchestra.run.smithi076.stderr:adjusted tunables profile to default 2024-03-08T06:00:11.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.836+0000 7f24f2c46640 1 -- 172.21.15.76:0/3491459748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24d4004800 msgr2=0x7f24ec1f3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:11.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.836+0000 7f24f2c46640 1 --2- 172.21.15.76:0/3491459748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24d4004800 0x7f24ec1f3aa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00f5a0 tx=0x7f24dc00fa60 comp rx=0 tx=0).stop 2024-03-08T06:00:11.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.836+0000 7f24f2c46640 1 -- 172.21.15.76:0/3491459748 shutdown_connections 2024-03-08T06:00:11.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.836+0000 7f24f2c46640 1 --2- 172.21.15.76:0/3491459748 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f24ec0b4df0 0x7f24ec14fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:11.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.836+0000 7f24f2c46640 1 --2- 172.21.15.76:0/3491459748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24ec0b4970 0x7f24ec14f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:11.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.836+0000 7f24f2c46640 1 --2- 172.21.15.76:0/3491459748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24d4004800 0x7f24ec1f3aa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:11.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.836+0000 7f24f2c46640 1 -- 172.21.15.76:0/3491459748 >> 172.21.15.76:0/3491459748 conn(0x7f24ec0d0890 msgr2=0x7f24ec0b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:11.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.836+0000 7f24f2c46640 1 -- 172.21.15.76:0/3491459748 shutdown_connections 2024-03-08T06:00:11.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:11.836+0000 7f24f2c46640 1 -- 172.21.15.76:0/3491459748 wait complete. 2024-03-08T06:00:11.847 INFO:tasks.ceph:check_enable_crimson: False 2024-03-08T06:00:11.847 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-08T06:00:11.848 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:00:11.848 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-08T06:00:11.874 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:00:11.874 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-08T06:00:11.941 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:00:11.941 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-08T06:00:12.008 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:00:12.008 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-08T06:00:12.075 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:00:12.075 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-08T06:00:12.101 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:00:12.101 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-08T06:00:12.168 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:00:12.168 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-08T06:00:12.232 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:12.232+0000 7f0db4c47040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T06:00:12.234 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:00:12.234 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-08T06:00:12.249 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:12.249+0000 7fe544e46040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T06:00:12.288 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:12.288+0000 7f0db4c47040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T06:00:12.300 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new b75bfb24-86a7-47d5-8a16-656c0f0d6f22 0 2024-03-08T06:00:12.302 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:12.302+0000 7fe544e46040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T06:00:12.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.440+0000 7f1391a47640 1 Processor -- start 2024-03-08T06:00:12.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.440+0000 7f1391a47640 1 -- start start 2024-03-08T06:00:12.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.440+0000 7f1391a47640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f138c15dbc0 0x7f138c0d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:12.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.440+0000 7f1391a47640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f138c0b5cf0 0x7f138c0b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:12.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.440+0000 7f1391a47640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f138c0d27d0 con 0x7f138c0b9840 2024-03-08T06:00:12.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.440+0000 7f1391a47640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f138c0d2ab0 con 0x7f138c15dbc0 2024-03-08T06:00:12.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.440+0000 7f1391a47640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f138c0d2d90 con 0x7f138c0b5cf0 2024-03-08T06:00:12.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.441+0000 7f138b7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f138c0b5cf0 0x7f138c0b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:12.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.441+0000 7f138b7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f138c0b5cf0 0x7f138c0b6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45710/0 (socket says 172.21.15.81:45710) 2024-03-08T06:00:12.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.441+0000 7f138b7fe640 1 -- 172.21.15.81:0/3582808817 learned_addr learned my addr 172.21.15.81:0/3582808817 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-08T06:00:12.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.441+0000 7f138affd640 1 --2- 172.21.15.81:0/3582808817 >> v2:172.21.15.76:3300/0 conn(0x7f138c15dbc0 0x7f138c0d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:12.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.441+0000 7f138b7fe640 1 -- 172.21.15.81:0/3582808817 >> v2:172.21.15.76:3300/0 conn(0x7f138c15dbc0 msgr2=0x7f138c0d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:12.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.441+0000 7f138b7fe640 1 --2- 172.21.15.81:0/3582808817 >> v2:172.21.15.76:3300/0 conn(0x7f138c15dbc0 0x7f138c0d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.441+0000 7f138b7fe640 1 -- 172.21.15.81:0/3582808817 >> v1:172.21.15.76:6789/0 conn(0x7f138c0b9840 legacy=0x7f138c0b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:12.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.441+0000 7f138b7fe640 1 -- 172.21.15.81:0/3582808817 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c0d3070 con 0x7f138c0b5cf0 2024-03-08T06:00:12.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.442+0000 7f138b7fe640 1 --2- 172.21.15.81:0/3582808817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138c0b5cf0 0x7f138c0b6c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f138c06d410 tx=0x7f138402bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5874aeb34721bbb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:12.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.442+0000 7f1389ffb640 1 -- 172.21.15.81:0/3582808817 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138402cb20 con 0x7f138c0b5cf0 2024-03-08T06:00:12.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.442+0000 7f1389ffb640 1 -- 172.21.15.81:0/3582808817 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138402ccc0 con 0x7f138c0b5cf0 2024-03-08T06:00:12.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.442+0000 7f1389ffb640 1 -- 172.21.15.81:0/3582808817 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384034610 con 0x7f138c0b5cf0 2024-03-08T06:00:12.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.442+0000 7f1391a47640 1 -- 172.21.15.81:0/3582808817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138c0b5cf0 msgr2=0x7f138c0b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:12.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.442+0000 7f1391a47640 1 --2- 172.21.15.81:0/3582808817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138c0b5cf0 0x7f138c0b6c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f138c06d410 tx=0x7f138402bf80 comp rx=0 tx=0).stop 2024-03-08T06:00:12.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.442+0000 7f1391a47640 1 -- 172.21.15.81:0/3582808817 shutdown_connections 2024-03-08T06:00:12.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.442+0000 7f1391a47640 1 --2- 172.21.15.81:0/3582808817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138c0b5cf0 0x7f138c0b6c90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.442+0000 7f1391a47640 1 --2- 172.21.15.81:0/3582808817 >> v2:172.21.15.76:3300/0 conn(0x7f138c15dbc0 0x7f138c0d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.442+0000 7f1391a47640 1 -- 172.21.15.81:0/3582808817 >> 172.21.15.81:0/3582808817 conn(0x7f138c0670d0 msgr2=0x7f138c0d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:12.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.443+0000 7f1391a47640 1 -- 172.21.15.81:0/3582808817 shutdown_connections 2024-03-08T06:00:12.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.443+0000 7f1391a47640 1 -- 172.21.15.81:0/3582808817 wait complete. 2024-03-08T06:00:12.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.444+0000 7f1391a47640 1 Processor -- start 2024-03-08T06:00:12.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.444+0000 7f1391a47640 1 -- start start 2024-03-08T06:00:12.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f1391a47640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f138c0b5cf0 0x7f138c149ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:12.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f1391a47640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138c0b9840 0x7f138c147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:12.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f1391a47640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f138c15dbc0 0x7f138c1f3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:12.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f1391a47640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c0d25b0 con 0x7f138c15dbc0 2024-03-08T06:00:12.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f1391a47640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c0d2430 con 0x7f138c0b9840 2024-03-08T06:00:12.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f1391a47640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138c15dfa0 con 0x7f138c0b5cf0 2024-03-08T06:00:12.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f138a7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138c0b9840 0x7f138c147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:12.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f138affd640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f138c0b5cf0 0x7f138c149ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:12.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f138a7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138c0b9840 0x7f138c147190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45726/0 (socket says 172.21.15.81:45726) 2024-03-08T06:00:12.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f138a7fc640 1 -- 172.21.15.81:0/1095366030 learned_addr learned my addr 172.21.15.81:0/1095366030 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-08T06:00:12.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f138b7fe640 1 --2- 172.21.15.81:0/1095366030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f138c15dbc0 0x7f138c1f3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:12.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f138affd640 1 -- 172.21.15.81:0/1095366030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138c0b9840 msgr2=0x7f138c147190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:12.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f138affd640 1 --2- 172.21.15.81:0/1095366030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138c0b9840 0x7f138c147190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f138affd640 1 -- 172.21.15.81:0/1095366030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f138c15dbc0 msgr2=0x7f138c1f3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:12.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f138affd640 1 --2- 172.21.15.81:0/1095366030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f138c15dbc0 0x7f138c1f3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f138affd640 1 -- 172.21.15.81:0/1095366030 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c1476d0 con 0x7f138c0b5cf0 2024-03-08T06:00:12.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f138a7fc640 1 --2- 172.21.15.81:0/1095366030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138c0b9840 0x7f138c147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:00:12.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f138affd640 1 --2- 172.21.15.81:0/1095366030 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f138c0b5cf0 0x7f138c149ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f137c00e870 tx=0x7f137c00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:12.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f1373fff640 1 -- 172.21.15.81:0/1095366030 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c017070 con 0x7f138c0b5cf0 2024-03-08T06:00:12.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f1373fff640 1 -- 172.21.15.81:0/1095366030 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137c00ee60 con 0x7f138c0b5cf0 2024-03-08T06:00:12.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f1373fff640 1 -- 172.21.15.81:0/1095366030 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c00cb60 con 0x7f138c0b5cf0 2024-03-08T06:00:12.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f1391a47640 1 -- 172.21.15.81:0/1095366030 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138c1f3940 con 0x7f138c0b5cf0 2024-03-08T06:00:12.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.445+0000 7f1391a47640 1 -- 172.21.15.81:0/1095366030 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138c147c50 con 0x7f138c0b5cf0 2024-03-08T06:00:12.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.446+0000 7f1373fff640 1 -- 172.21.15.81:0/1095366030 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f137c01a070 con 0x7f138c0b5cf0 2024-03-08T06:00:12.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.446+0000 7f1373fff640 1 -- 172.21.15.81:0/1095366030 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f137c013470 con 0x7f138c0b5cf0 2024-03-08T06:00:12.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.446+0000 7f1391a47640 1 -- 172.21.15.81:0/1095366030 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f138c04f030 con 0x7f138c0b5cf0 2024-03-08T06:00:12.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.450+0000 7f1373fff640 1 -- 172.21.15.81:0/1095366030 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f137c00cd50 con 0x7f138c0b5cf0 2024-03-08T06:00:12.522 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.522+0000 7f1391a47640 1 -- 172.21.15.81:0/1095366030 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b75bfb24-86a7-47d5-8a16-656c0f0d6f22", "id": 0} v 0) v1 -- 0x7f138c15b9a0 con 0x7f138c0b5cf0 2024-03-08T06:00:12.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.528+0000 7f1373fff640 1 -- 172.21.15.81:0/1095366030 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b75bfb24-86a7-47d5-8a16-656c0f0d6f22", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f137c005990 con 0x7f138c0b5cf0 2024-03-08T06:00:12.528 INFO:teuthology.orchestra.run.smithi081.stdout:0 2024-03-08T06:00:12.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.528+0000 7f1391a47640 1 -- 172.21.15.81:0/1095366030 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f138c0b5cf0 msgr2=0x7f138c149ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:12.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.528+0000 7f1391a47640 1 --2- 172.21.15.81:0/1095366030 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f138c0b5cf0 0x7f138c149ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f137c00e870 tx=0x7f137c00ed30 comp rx=0 tx=0).stop 2024-03-08T06:00:12.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.529+0000 7f1391a47640 1 -- 172.21.15.81:0/1095366030 shutdown_connections 2024-03-08T06:00:12.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.529+0000 7f1391a47640 1 --2- 172.21.15.81:0/1095366030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f138c15dbc0 0x7f138c1f3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.529+0000 7f1391a47640 1 --2- 172.21.15.81:0/1095366030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138c0b9840 0x7f138c147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.529+0000 7f1391a47640 1 --2- 172.21.15.81:0/1095366030 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f138c0b5cf0 0x7f138c149ce0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.529+0000 7f1391a47640 1 -- 172.21.15.81:0/1095366030 >> 172.21.15.81:0/1095366030 conn(0x7f138c0670d0 msgr2=0x7f138c0c7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:12.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.529+0000 7f1391a47640 1 -- 172.21.15.81:0/1095366030 shutdown_connections 2024-03-08T06:00:12.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.529+0000 7f1391a47640 1 -- 172.21.15.81:0/1095366030 wait complete. 2024-03-08T06:00:12.539 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 3bedb444-41a6-4622-b5b1-7ee426a7bd3d 1 2024-03-08T06:00:12.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.638+0000 7fc0fb2f4640 1 Processor -- start 2024-03-08T06:00:12.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.638+0000 7fc0fb2f4640 1 -- start start 2024-03-08T06:00:12.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.639+0000 7fc0fb2f4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc0f40cb320 0x7fc0f40ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:12.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.639+0000 7fc0fb2f4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0f40cb7b0 0x7fc0f40cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:12.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.639+0000 7fc0fb2f4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f40cda70 con 0x7fc0f415aab0 2024-03-08T06:00:12.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.639+0000 7fc0fb2f4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f406edb0 con 0x7fc0f40cb320 2024-03-08T06:00:12.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.639+0000 7fc0fb2f4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f40cd810 con 0x7fc0f40cb7b0 2024-03-08T06:00:12.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.639+0000 7fc0f9891640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0f40cb7b0 0x7fc0f40cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:12.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.639+0000 7fc0f9090640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc0f415aab0 0x7fc0f40cd100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:57944/0 (socket says 172.21.15.81:57944) 2024-03-08T06:00:12.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.639+0000 7fc0f9090640 1 -- 172.21.15.81:0/3588639613 learned_addr learned my addr 172.21.15.81:0/3588639613 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-03-08T06:00:12.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.640+0000 7fc0e7fff640 1 -- 172.21.15.81:0/3588639613 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634450812 0 0) 0x7fc0f40cda70 con 0x7fc0f415aab0 2024-03-08T06:00:12.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.640+0000 7fc0e7fff640 1 -- 172.21.15.81:0/3588639613 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0e0003520 con 0x7fc0f415aab0 2024-03-08T06:00:12.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.640+0000 7fc0f888f640 1 --2- 172.21.15.81:0/3588639613 >> v2:172.21.15.76:3300/0 conn(0x7fc0f40cb320 0x7fc0f40ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:12.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.640+0000 7fc0f9891640 1 -- 172.21.15.81:0/3588639613 >> v2:172.21.15.76:3300/0 conn(0x7fc0f40cb320 msgr2=0x7fc0f40ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:12.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.640+0000 7fc0f9891640 1 --2- 172.21.15.81:0/3588639613 >> v2:172.21.15.76:3300/0 conn(0x7fc0f40cb320 0x7fc0f40ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.640+0000 7fc0f9891640 1 -- 172.21.15.81:0/3588639613 >> v1:172.21.15.81:6789/0 conn(0x7fc0f415aab0 legacy=0x7fc0f40cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:12.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.640+0000 7fc0f9891640 1 -- 172.21.15.81:0/3588639613 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f40cdd50 con 0x7fc0f40cb7b0 2024-03-08T06:00:12.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.640+0000 7fc0f9891640 1 --2- 172.21.15.81:0/3588639613 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0f40cb7b0 0x7fc0f40cf2e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009dc0 tx=0x7fc0ec02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f34b12a5be4221ae server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:12.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.641+0000 7fc0e7fff640 1 -- 172.21.15.81:0/3588639613 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec039070 con 0x7fc0f40cb7b0 2024-03-08T06:00:12.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.641+0000 7fc0e7fff640 1 -- 172.21.15.81:0/3588639613 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ec02cc40 con 0x7fc0f40cb7b0 2024-03-08T06:00:12.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.641+0000 7fc0e7fff640 1 -- 172.21.15.81:0/3588639613 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec034440 con 0x7fc0f40cb7b0 2024-03-08T06:00:12.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.641+0000 7fc0fb2f4640 1 -- 172.21.15.81:0/3588639613 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0f40cb7b0 msgr2=0x7fc0f40cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:12.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.641+0000 7fc0fb2f4640 1 --2- 172.21.15.81:0/3588639613 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0f40cb7b0 0x7fc0f40cf2e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009dc0 tx=0x7fc0ec02be90 comp rx=0 tx=0).stop 2024-03-08T06:00:12.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.641+0000 7fc0fb2f4640 1 -- 172.21.15.81:0/3588639613 shutdown_connections 2024-03-08T06:00:12.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.641+0000 7fc0fb2f4640 1 --2- 172.21.15.81:0/3588639613 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0f40cb7b0 0x7fc0f40cf2e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.641+0000 7fc0fb2f4640 1 --2- 172.21.15.81:0/3588639613 >> v2:172.21.15.76:3300/0 conn(0x7fc0f40cb320 0x7fc0f40ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.641+0000 7fc0fb2f4640 1 -- 172.21.15.81:0/3588639613 >> 172.21.15.81:0/3588639613 conn(0x7fc0f4099a60 msgr2=0x7fc0f4099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:12.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.641+0000 7fc0fb2f4640 1 -- 172.21.15.81:0/3588639613 shutdown_connections 2024-03-08T06:00:12.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.641+0000 7fc0fb2f4640 1 -- 172.21.15.81:0/3588639613 wait complete. 2024-03-08T06:00:12.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.642+0000 7fc0fb2f4640 1 Processor -- start 2024-03-08T06:00:12.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.642+0000 7fc0fb2f4640 1 -- start start 2024-03-08T06:00:12.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.643+0000 7fc0fb2f4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0f40cb320 0x7fc0f40b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:12.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.643+0000 7fc0fb2f4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0f40cb7b0 0x7fc0f41f08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:12.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.643+0000 7fc0fb2f4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415aab0 0x7fc0f41f0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:12.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.643+0000 7fc0fb2f4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f415ae90 con 0x7fc0f415aab0 2024-03-08T06:00:12.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.643+0000 7fc0fb2f4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f40ce030 con 0x7fc0f40cb7b0 2024-03-08T06:00:12.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.643+0000 7fc0fb2f4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f415b010 con 0x7fc0f40cb320 2024-03-08T06:00:12.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.643+0000 7fc0f9090640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0f40cb320 0x7fc0f40b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:12.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.643+0000 7fc0f888f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0f40cb7b0 0x7fc0f41f08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:12.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.643+0000 7fc0f9090640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0f40cb320 0x7fc0f40b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.81:51802/0 (socket says 172.21.15.81:51802) 2024-03-08T06:00:12.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.643+0000 7fc0f9090640 1 -- 172.21.15.81:0/3626336491 learned_addr learned my addr 172.21.15.81:0/3626336491 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-08T06:00:12.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.643+0000 7fc0f9891640 1 --2- 172.21.15.81:0/3626336491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415aab0 0x7fc0f41f0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:12.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.644+0000 7fc0f9090640 1 -- 172.21.15.81:0/3626336491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0f40cb7b0 msgr2=0x7fc0f41f08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:12.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.644+0000 7fc0f9090640 1 --2- 172.21.15.81:0/3626336491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0f40cb7b0 0x7fc0f41f08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.644+0000 7fc0f9090640 1 -- 172.21.15.81:0/3626336491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415aab0 msgr2=0x7fc0f41f0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:12.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.644+0000 7fc0f9090640 1 --2- 172.21.15.81:0/3626336491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415aab0 0x7fc0f41f0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.644+0000 7fc0f9090640 1 -- 172.21.15.81:0/3626336491 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f40b4550 con 0x7fc0f40cb320 2024-03-08T06:00:12.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.644+0000 7fc0f9891640 1 --2- 172.21.15.81:0/3626336491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415aab0 0x7fc0f41f0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:12.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.644+0000 7fc0f888f640 1 --2- 172.21.15.81:0/3626336491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0f40cb7b0 0x7fc0f41f08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:00:12.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.644+0000 7fc0f9090640 1 --2- 172.21.15.81:0/3626336491 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0f40cb320 0x7fc0f40b4010 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800c460 tx=0x7fc0e800c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:12.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.644+0000 7fc0e5ffb640 1 -- 172.21.15.81:0/3626336491 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8017070 con 0x7fc0f40cb320 2024-03-08T06:00:12.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.644+0000 7fc0e5ffb640 1 -- 172.21.15.81:0/3626336491 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e8005ce0 con 0x7fc0f40cb320 2024-03-08T06:00:12.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.644+0000 7fc0fb2f4640 1 -- 172.21.15.81:0/3626336491 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f40c5030 con 0x7fc0f40cb320 2024-03-08T06:00:12.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.644+0000 7fc0e5ffb640 1 -- 172.21.15.81:0/3626336491 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8007290 con 0x7fc0f40cb320 2024-03-08T06:00:12.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.644+0000 7fc0e5ffb640 1 -- 172.21.15.81:0/3626336491 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc0e801a070 con 0x7fc0f40cb320 2024-03-08T06:00:12.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.644+0000 7fc0fb2f4640 1 -- 172.21.15.81:0/3626336491 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f40b3550 con 0x7fc0f40cb320 2024-03-08T06:00:12.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.644+0000 7fc0e5ffb640 1 -- 172.21.15.81:0/3626336491 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fc0e8012cf0 con 0x7fc0f40cb320 2024-03-08T06:00:12.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.645+0000 7fc0fb2f4640 1 -- 172.21.15.81:0/3626336491 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f404f030 con 0x7fc0f40cb320 2024-03-08T06:00:12.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.647+0000 7fc0e5ffb640 1 -- 172.21.15.81:0/3626336491 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc0e8012400 con 0x7fc0f40cb320 2024-03-08T06:00:12.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.719+0000 7fc0fb2f4640 1 -- 172.21.15.81:0/3626336491 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3bedb444-41a6-4622-b5b1-7ee426a7bd3d", "id": 1} v 0) v1 -- 0x7fc0f406e780 con 0x7fc0f40cb320 2024-03-08T06:00:12.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.725+0000 7fc0e5ffb640 1 -- 172.21.15.81:0/3626336491 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3bedb444-41a6-4622-b5b1-7ee426a7bd3d", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc0e80125e0 con 0x7fc0f40cb320 2024-03-08T06:00:12.725 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-03-08T06:00:12.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.726+0000 7fc0fb2f4640 1 -- 172.21.15.81:0/3626336491 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0f40cb320 msgr2=0x7fc0f40b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:12.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.726+0000 7fc0fb2f4640 1 --2- 172.21.15.81:0/3626336491 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0f40cb320 0x7fc0f40b4010 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800c460 tx=0x7fc0e800c920 comp rx=0 tx=0).stop 2024-03-08T06:00:12.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.726+0000 7fc0fb2f4640 1 -- 172.21.15.81:0/3626336491 shutdown_connections 2024-03-08T06:00:12.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.726+0000 7fc0fb2f4640 1 --2- 172.21.15.81:0/3626336491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415aab0 0x7fc0f41f0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.726+0000 7fc0fb2f4640 1 --2- 172.21.15.81:0/3626336491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0f40cb7b0 0x7fc0f41f08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.726+0000 7fc0fb2f4640 1 --2- 172.21.15.81:0/3626336491 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0f40cb320 0x7fc0f40b4010 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.726+0000 7fc0fb2f4640 1 -- 172.21.15.81:0/3626336491 >> 172.21.15.81:0/3626336491 conn(0x7fc0f4099a60 msgr2=0x7fc0f406ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:12.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.726+0000 7fc0fb2f4640 1 -- 172.21.15.81:0/3626336491 shutdown_connections 2024-03-08T06:00:12.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.726+0000 7fc0fb2f4640 1 -- 172.21.15.81:0/3626336491 wait complete. 2024-03-08T06:00:12.736 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 6cc45538-985d-4a82-ba35-0bc1019dcd5c 2 2024-03-08T06:00:12.825 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:12.825+0000 7fe544e46040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T06:00:12.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.837+0000 7f5986ea4640 1 Processor -- start 2024-03-08T06:00:12.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.837+0000 7f5986ea4640 1 -- start start 2024-03-08T06:00:12.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.838+0000 7f5986ea4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f59800cf4b0 0x7f598006f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:12.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.838+0000 7f5986ea4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59800cda70 con 0x7f598015aab0 2024-03-08T06:00:12.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.838+0000 7f5986ea4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59800cdd50 con 0x7f59800cb320 2024-03-08T06:00:12.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.838+0000 7f5986ea4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598006edb0 con 0x7f59800cf4b0 2024-03-08T06:00:12.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.838+0000 7f5985441640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f59800cf4b0 0x7f598006f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:12.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.838+0000 7f597ffff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f59800cb320 0x7f59800ceda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:57948/0 (socket says 172.21.15.81:57948) 2024-03-08T06:00:12.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.838+0000 7f597ffff640 1 -- 172.21.15.81:0/1859879404 learned_addr learned my addr 172.21.15.81:0/1859879404 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-03-08T06:00:12.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.838+0000 7f5985441640 1 -- 172.21.15.81:0/1859879404 >> v1:172.21.15.81:6789/0 conn(0x7f59800cb320 legacy=0x7f59800ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:12.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.838+0000 7f5985441640 1 -- 172.21.15.81:0/1859879404 >> v1:172.21.15.76:6789/0 conn(0x7f598015aab0 legacy=0x7f59800cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:12.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.838+0000 7f597f7fe640 1 -- 172.21.15.81:0/1859879404 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830672007 0 0) 0x7f59800cdd50 con 0x7f59800cb320 2024-03-08T06:00:12.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.838+0000 7f5985441640 1 -- 172.21.15.81:0/1859879404 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598006ea40 con 0x7f59800cf4b0 2024-03-08T06:00:12.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.839+0000 7f5985441640 1 --2- 172.21.15.81:0/1859879404 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f59800cf4b0 0x7f598006f240 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5978009df0 tx=0x7f597802e110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6f5857ea64baea3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:12.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.839+0000 7f597f7fe640 1 -- 172.21.15.81:0/1859879404 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597803b070 con 0x7f59800cf4b0 2024-03-08T06:00:12.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.839+0000 7f597f7fe640 1 -- 172.21.15.81:0/1859879404 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5978036440 con 0x7f59800cf4b0 2024-03-08T06:00:12.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.839+0000 7f597f7fe640 1 -- 172.21.15.81:0/1859879404 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5978036810 con 0x7f59800cf4b0 2024-03-08T06:00:12.840 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:12.840+0000 7f0db4c47040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T06:00:12.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.840+0000 7f5986ea4640 1 -- 172.21.15.81:0/1859879404 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f59800cf4b0 msgr2=0x7f598006f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:12.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.840+0000 7f5986ea4640 1 --2- 172.21.15.81:0/1859879404 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f59800cf4b0 0x7f598006f240 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5978009df0 tx=0x7f597802e110 comp rx=0 tx=0).stop 2024-03-08T06:00:12.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.840+0000 7f5986ea4640 1 -- 172.21.15.81:0/1859879404 shutdown_connections 2024-03-08T06:00:12.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.840+0000 7f5986ea4640 1 --2- 172.21.15.81:0/1859879404 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f59800cf4b0 0x7f598006f240 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.840+0000 7f5986ea4640 1 -- 172.21.15.81:0/1859879404 >> 172.21.15.81:0/1859879404 conn(0x7f5980099a60 msgr2=0x7f5980099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:12.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.840+0000 7f5986ea4640 1 -- 172.21.15.81:0/1859879404 shutdown_connections 2024-03-08T06:00:12.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.840+0000 7f5986ea4640 1 -- 172.21.15.81:0/1859879404 wait complete. 2024-03-08T06:00:12.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.841+0000 7f5986ea4640 1 Processor -- start 2024-03-08T06:00:12.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.841+0000 7f5986ea4640 1 -- start start 2024-03-08T06:00:12.844 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.842+0000 7f5986ea4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f59800cb320 0x7f5980156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:12.844 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.842+0000 7f5986ea4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59800cf4b0 0x7f59801f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:12.844 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.842+0000 7f5986ea4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f598015aab0 0x7f59800c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:12.844 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.842+0000 7f5986ea4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598015ae90 con 0x7f59800cf4b0 2024-03-08T06:00:12.844 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.842+0000 7f5986ea4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59800cd810 con 0x7f598015aab0 2024-03-08T06:00:12.844 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.842+0000 7f5986ea4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598015b010 con 0x7f59800cb320 2024-03-08T06:00:12.844 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.842+0000 7f5984c40640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f59800cb320 0x7f5980156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:12.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.842+0000 7f5984c40640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f59800cb320 0x7f5980156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.81:51830/0 (socket says 172.21.15.81:51830) 2024-03-08T06:00:12.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.842+0000 7f5984c40640 1 -- 172.21.15.81:0/4277074094 learned_addr learned my addr 172.21.15.81:0/4277074094 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-08T06:00:12.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.842+0000 7f5985441640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f598015aab0 0x7f59800c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:12.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.842+0000 7f597ffff640 1 --2- 172.21.15.81:0/4277074094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59800cf4b0 0x7f59801f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:12.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.842+0000 7f5984c40640 1 -- 172.21.15.81:0/4277074094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f598015aab0 msgr2=0x7f59800c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:12.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.842+0000 7f5984c40640 1 --2- 172.21.15.81:0/4277074094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f598015aab0 0x7f59800c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.842+0000 7f5984c40640 1 -- 172.21.15.81:0/4277074094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59800cf4b0 msgr2=0x7f59801f4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:12.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.842+0000 7f5984c40640 1 --2- 172.21.15.81:0/4277074094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59800cf4b0 0x7f59801f4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.843+0000 7f5984c40640 1 -- 172.21.15.81:0/4277074094 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59801f50c0 con 0x7f59800cb320 2024-03-08T06:00:12.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.843+0000 7f5985441640 1 --2- 172.21.15.81:0/4277074094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f598015aab0 0x7f59800c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:00:12.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.843+0000 7f5984c40640 1 --2- 172.21.15.81:0/4277074094 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f59800cb320 0x7f5980156000 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f597400c430 tx=0x7f597400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:12.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.843+0000 7f597d7fa640 1 -- 172.21.15.81:0/4277074094 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5974017070 con 0x7f59800cb320 2024-03-08T06:00:12.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.843+0000 7f597d7fa640 1 -- 172.21.15.81:0/4277074094 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59740052d0 con 0x7f59800cb320 2024-03-08T06:00:12.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.843+0000 7f5986ea4640 1 -- 172.21.15.81:0/4277074094 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59800ca0d0 con 0x7f59800cb320 2024-03-08T06:00:12.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.843+0000 7f597d7fa640 1 -- 172.21.15.81:0/4277074094 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5974003830 con 0x7f59800cb320 2024-03-08T06:00:12.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.843+0000 7f5986ea4640 1 -- 172.21.15.81:0/4277074094 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59800b39e0 con 0x7f59800cb320 2024-03-08T06:00:12.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.843+0000 7f597d7fa640 1 -- 172.21.15.81:0/4277074094 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f597401a070 con 0x7f59800cb320 2024-03-08T06:00:12.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.843+0000 7f597d7fa640 1 -- 172.21.15.81:0/4277074094 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f5974012420 con 0x7f59800cb320 2024-03-08T06:00:12.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.844+0000 7f5986ea4640 1 -- 172.21.15.81:0/4277074094 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598004f030 con 0x7f59800cb320 2024-03-08T06:00:12.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.846+0000 7f597d7fa640 1 -- 172.21.15.81:0/4277074094 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5974021070 con 0x7f59800cb320 2024-03-08T06:00:12.879 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:12.878+0000 7fe544e46040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T06:00:12.897 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:12.897+0000 7f0db4c47040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T06:00:12.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.918+0000 7f5986ea4640 1 -- 172.21.15.81:0/4277074094 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6cc45538-985d-4a82-ba35-0bc1019dcd5c", "id": 2} v 0) v1 -- 0x7f598006e780 con 0x7f59800cb320 2024-03-08T06:00:12.923 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:12.923+0000 7fe544e46040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T06:00:12.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.924+0000 7f597d7fa640 1 -- 172.21.15.81:0/4277074094 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6cc45538-985d-4a82-ba35-0bc1019dcd5c", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f59801f50c0 con 0x7f59800cb320 2024-03-08T06:00:12.924 INFO:teuthology.orchestra.run.smithi081.stdout:2 2024-03-08T06:00:12.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.924+0000 7f5986ea4640 1 -- 172.21.15.81:0/4277074094 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f59800cb320 msgr2=0x7f5980156000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:12.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.924+0000 7f5986ea4640 1 --2- 172.21.15.81:0/4277074094 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f59800cb320 0x7f5980156000 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f597400c430 tx=0x7f597400c8f0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.925+0000 7f5986ea4640 1 -- 172.21.15.81:0/4277074094 shutdown_connections 2024-03-08T06:00:12.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.925+0000 7f5986ea4640 1 --2- 172.21.15.81:0/4277074094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f598015aab0 0x7f59800c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.925+0000 7f5986ea4640 1 --2- 172.21.15.81:0/4277074094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59800cf4b0 0x7f59801f4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.925+0000 7f5986ea4640 1 --2- 172.21.15.81:0/4277074094 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f59800cb320 0x7f5980156000 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:12.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.925+0000 7f5986ea4640 1 -- 172.21.15.81:0/4277074094 >> 172.21.15.81:0/4277074094 conn(0x7f5980099a60 msgr2=0x7f59800d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:12.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.925+0000 7f5986ea4640 1 -- 172.21.15.81:0/4277074094 shutdown_connections 2024-03-08T06:00:12.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:12.925+0000 7f5986ea4640 1 -- 172.21.15.81:0/4277074094 wait complete. 2024-03-08T06:00:12.935 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 60266e20-1c28-4910-9963-62fce37868f3 3 2024-03-08T06:00:12.944 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:12.944+0000 7f0db4c47040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T06:00:12.977 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:12.976+0000 7fe544e46040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T06:00:13.005 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:13.005+0000 7f0db4c47040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T06:00:13.019 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:13.018+0000 7fe544e46040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T06:00:13.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.035+0000 7fe53d4b5640 1 Processor -- start 2024-03-08T06:00:13.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.035+0000 7fe53d4b5640 1 -- start start 2024-03-08T06:00:13.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.036+0000 7fe53d4b5640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe53815dbc0 0x7fe5380d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.036+0000 7fe53d4b5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe5380b5cf0 0x7fe5380b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.036+0000 7fe53d4b5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5380d27d0 con 0x7fe5380b9840 2024-03-08T06:00:13.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.036+0000 7fe53d4b5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5380d2ab0 con 0x7fe5380b5cf0 2024-03-08T06:00:13.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.036+0000 7fe53d4b5640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380d2d90 con 0x7fe53815dbc0 2024-03-08T06:00:13.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.036+0000 7fe5377fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe5380b5cf0 0x7fe5380b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.036+0000 7fe5377fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe5380b5cf0 0x7fe5380b6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45746/0 (socket says 172.21.15.81:45746) 2024-03-08T06:00:13.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.036+0000 7fe5367fc640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe5380b9840 0x7fe5380b55e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:57956/0 (socket says 172.21.15.81:57956) 2024-03-08T06:00:13.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.036+0000 7fe5377fe640 1 -- 172.21.15.81:0/228836497 learned_addr learned my addr 172.21.15.81:0/228836497 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-08T06:00:13.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.036+0000 7fe536ffd640 1 --2- 172.21.15.81:0/228836497 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe53815dbc0 0x7fe5380d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.036+0000 7fe536ffd640 1 -- 172.21.15.81:0/228836497 >> v2:172.21.15.81:3300/0 conn(0x7fe5380b5cf0 msgr2=0x7fe5380b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:13.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.036+0000 7fe536ffd640 1 --2- 172.21.15.81:0/228836497 >> v2:172.21.15.81:3300/0 conn(0x7fe5380b5cf0 0x7fe5380b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.036+0000 7fe536ffd640 1 -- 172.21.15.81:0/228836497 >> v1:172.21.15.81:6789/0 conn(0x7fe5380b9840 legacy=0x7fe5380b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.036+0000 7fe536ffd640 1 -- 172.21.15.81:0/228836497 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5380d3070 con 0x7fe53815dbc0 2024-03-08T06:00:13.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.036+0000 7fe5377fe640 1 --2- 172.21.15.81:0/228836497 >> v2:172.21.15.81:3300/0 conn(0x7fe5380b5cf0 0x7fe5380b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:00:13.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.037+0000 7fe536ffd640 1 --2- 172.21.15.81:0/228836497 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe53815dbc0 0x7fe5380d1ef0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe5280097a0 tx=0x7fe52802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=617055e7a66ff79c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:13.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.037+0000 7fe535ffb640 1 -- 172.21.15.81:0/228836497 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe528035020 con 0x7fe53815dbc0 2024-03-08T06:00:13.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.037+0000 7fe535ffb640 1 -- 172.21.15.81:0/228836497 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe528031810 con 0x7fe53815dbc0 2024-03-08T06:00:13.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.037+0000 7fe535ffb640 1 -- 172.21.15.81:0/228836497 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe528031be0 con 0x7fe53815dbc0 2024-03-08T06:00:13.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.037+0000 7fe53d4b5640 1 -- 172.21.15.81:0/228836497 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe53815dbc0 msgr2=0x7fe5380d1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.037+0000 7fe53d4b5640 1 --2- 172.21.15.81:0/228836497 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe53815dbc0 0x7fe5380d1ef0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe5280097a0 tx=0x7fe52802b8b0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.038+0000 7fe53d4b5640 1 -- 172.21.15.81:0/228836497 shutdown_connections 2024-03-08T06:00:13.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.038+0000 7fe53d4b5640 1 --2- 172.21.15.81:0/228836497 >> v2:172.21.15.81:3300/0 conn(0x7fe5380b5cf0 0x7fe5380b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.038+0000 7fe53d4b5640 1 --2- 172.21.15.81:0/228836497 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe53815dbc0 0x7fe5380d1ef0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.038+0000 7fe53d4b5640 1 -- 172.21.15.81:0/228836497 >> 172.21.15.81:0/228836497 conn(0x7fe5380670d0 msgr2=0x7fe5380d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:13.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.038+0000 7fe53d4b5640 1 -- 172.21.15.81:0/228836497 shutdown_connections 2024-03-08T06:00:13.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.038+0000 7fe53d4b5640 1 -- 172.21.15.81:0/228836497 wait complete. 2024-03-08T06:00:13.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.039+0000 7fe53d4b5640 1 Processor -- start 2024-03-08T06:00:13.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.039+0000 7fe53d4b5640 1 -- start start 2024-03-08T06:00:13.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe53d4b5640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5380b5cf0 0x7fe5380cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe53d4b5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5380b9840 0x7fe5380c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe53d4b5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe53815dbc0 0x7fe538146970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe53d4b5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380d25b0 con 0x7fe53815dbc0 2024-03-08T06:00:13.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe53d4b5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380d2430 con 0x7fe5380b9840 2024-03-08T06:00:13.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe53d4b5640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53815dfa0 con 0x7fe5380b5cf0 2024-03-08T06:00:13.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe536ffd640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5380b5cf0 0x7fe5380cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe5367fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5380b9840 0x7fe5380c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe536ffd640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5380b5cf0 0x7fe5380cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.81:51838/0 (socket says 172.21.15.81:51838) 2024-03-08T06:00:13.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe536ffd640 1 -- 172.21.15.81:0/2674089291 learned_addr learned my addr 172.21.15.81:0/2674089291 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-08T06:00:13.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe5377fe640 1 --2- 172.21.15.81:0/2674089291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe53815dbc0 0x7fe538146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe536ffd640 1 -- 172.21.15.81:0/2674089291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5380b9840 msgr2=0x7fe5380c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe536ffd640 1 --2- 172.21.15.81:0/2674089291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5380b9840 0x7fe5380c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe536ffd640 1 -- 172.21.15.81:0/2674089291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe53815dbc0 msgr2=0x7fe538146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe536ffd640 1 --2- 172.21.15.81:0/2674089291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe53815dbc0 0x7fe538146970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe536ffd640 1 -- 172.21.15.81:0/2674089291 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe538157ee0 con 0x7fe5380b5cf0 2024-03-08T06:00:13.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe5367fc640 1 --2- 172.21.15.81:0/2674089291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5380b9840 0x7fe5380c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:00:13.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe5377fe640 1 --2- 172.21.15.81:0/2674089291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe53815dbc0 0x7fe538146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:00:13.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe536ffd640 1 --2- 172.21.15.81:0/2674089291 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5380b5cf0 0x7fe5380cac30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe528000c00 tx=0x7fe5280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:13.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe51f7fe640 1 -- 172.21.15.81:0/2674089291 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe528040070 con 0x7fe5380b5cf0 2024-03-08T06:00:13.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe51f7fe640 1 -- 172.21.15.81:0/2674089291 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe528039cf0 con 0x7fe5380b5cf0 2024-03-08T06:00:13.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe51f7fe640 1 -- 172.21.15.81:0/2674089291 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe528035040 con 0x7fe5380b5cf0 2024-03-08T06:00:13.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.040+0000 7fe53d4b5640 1 -- 172.21.15.81:0/2674089291 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5381581a0 con 0x7fe5380b5cf0 2024-03-08T06:00:13.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.041+0000 7fe53d4b5640 1 -- 172.21.15.81:0/2674089291 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe538158760 con 0x7fe5380b5cf0 2024-03-08T06:00:13.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.041+0000 7fe51f7fe640 1 -- 172.21.15.81:0/2674089291 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe52804b060 con 0x7fe5380b5cf0 2024-03-08T06:00:13.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.041+0000 7fe51f7fe640 1 -- 172.21.15.81:0/2674089291 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fe52803ad60 con 0x7fe5380b5cf0 2024-03-08T06:00:13.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.041+0000 7fe53d4b5640 1 -- 172.21.15.81:0/2674089291 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe53804f030 con 0x7fe5380b5cf0 2024-03-08T06:00:13.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.043+0000 7fe51f7fe640 1 -- 172.21.15.81:0/2674089291 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe528052020 con 0x7fe5380b5cf0 2024-03-08T06:00:13.051 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:13.050+0000 7f0db4c47040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T06:00:13.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.116+0000 7fe53d4b5640 1 -- 172.21.15.81:0/2674089291 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd new", "uuid": "60266e20-1c28-4910-9963-62fce37868f3", "id": 3} v 0) v1 -- 0x7fe5380b5780 con 0x7fe5380b5cf0 2024-03-08T06:00:13.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.122+0000 7fe51f7fe640 1 -- 172.21.15.81:0/2674089291 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "60266e20-1c28-4910-9963-62fce37868f3", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe52803a640 con 0x7fe5380b5cf0 2024-03-08T06:00:13.122 INFO:teuthology.orchestra.run.smithi081.stdout:3 2024-03-08T06:00:13.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.122+0000 7fe53d4b5640 1 -- 172.21.15.81:0/2674089291 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5380b5cf0 msgr2=0x7fe5380cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.122+0000 7fe53d4b5640 1 --2- 172.21.15.81:0/2674089291 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5380b5cf0 0x7fe5380cac30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe528000c00 tx=0x7fe5280046c0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.122+0000 7fe53d4b5640 1 -- 172.21.15.81:0/2674089291 shutdown_connections 2024-03-08T06:00:13.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.122+0000 7fe53d4b5640 1 --2- 172.21.15.81:0/2674089291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe53815dbc0 0x7fe538146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.122+0000 7fe53d4b5640 1 --2- 172.21.15.81:0/2674089291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5380b9840 0x7fe5380c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.122+0000 7fe53d4b5640 1 --2- 172.21.15.81:0/2674089291 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5380b5cf0 0x7fe5380cac30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.122+0000 7fe53d4b5640 1 -- 172.21.15.81:0/2674089291 >> 172.21.15.81:0/2674089291 conn(0x7fe5380670d0 msgr2=0x7fe53815bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:13.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.123+0000 7fe53d4b5640 1 -- 172.21.15.81:0/2674089291 shutdown_connections 2024-03-08T06:00:13.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.123+0000 7fe53d4b5640 1 -- 172.21.15.81:0/2674089291 wait complete. 2024-03-08T06:00:13.133 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new f7dbb15c-028c-4e1f-94a4-4051041e3bdd 4 2024-03-08T06:00:13.135 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:13.135+0000 7fe544e46040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T06:00:13.170 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:13.170+0000 7f0db4c47040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T06:00:13.204 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:13.204+0000 7fe544e46040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T06:00:13.244 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:13.243+0000 7f0db4c47040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T06:00:13.249 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:13.249+0000 7fe544e46040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T06:00:13.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.269+0000 7fb68df99640 1 Processor -- start 2024-03-08T06:00:13.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.269+0000 7fb68df99640 1 -- start start 2024-03-08T06:00:13.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.270+0000 7fb68df99640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb6881594b0 0x7fb688160220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.270+0000 7fb68df99640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb68806b390 0x7fb68815c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.270+0000 7fb68df99640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6880cc820 con 0x7fb68815c0b0 2024-03-08T06:00:13.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.270+0000 7fb68df99640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68815cc70 con 0x7fb6881594b0 2024-03-08T06:00:13.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.270+0000 7fb68df99640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68815f740 con 0x7fb68806b390 2024-03-08T06:00:13.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.270+0000 7fb687fff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb68806b390 0x7fb68815c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.270+0000 7fb687fff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb68806b390 0x7fb68815c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.81:51848/0 (socket says 172.21.15.81:51848) 2024-03-08T06:00:13.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.270+0000 7fb687fff640 1 -- 172.21.15.81:0/2781514954 learned_addr learned my addr 172.21.15.81:0/2781514954 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-08T06:00:13.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.270+0000 7fb6877fe640 1 --2- 172.21.15.81:0/2781514954 >> v2:172.21.15.76:3300/0 conn(0x7fb6881594b0 0x7fb688160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.271+0000 7fb687fff640 1 -- 172.21.15.81:0/2781514954 >> v2:172.21.15.76:3300/0 conn(0x7fb6881594b0 msgr2=0x7fb688160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:13.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.271+0000 7fb687fff640 1 --2- 172.21.15.81:0/2781514954 >> v2:172.21.15.76:3300/0 conn(0x7fb6881594b0 0x7fb688160220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.271+0000 7fb687fff640 1 -- 172.21.15.81:0/2781514954 >> v1:172.21.15.76:6789/0 conn(0x7fb68815c0b0 legacy=0x7fb68806ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.271+0000 7fb687fff640 1 -- 172.21.15.81:0/2781514954 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68815fa20 con 0x7fb68806b390 2024-03-08T06:00:13.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.271+0000 7fb687fff640 1 --2- 172.21.15.81:0/2781514954 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb68806b390 0x7fb68815c540 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb680009e50 tx=0x7fb68002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b460655f77d9d8d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:13.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.272+0000 7fb6867fc640 1 -- 172.21.15.81:0/2781514954 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68002cb00 con 0x7fb68806b390 2024-03-08T06:00:13.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.272+0000 7fb6867fc640 1 -- 172.21.15.81:0/2781514954 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68002cca0 con 0x7fb68806b390 2024-03-08T06:00:13.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.272+0000 7fb6867fc640 1 -- 172.21.15.81:0/2781514954 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680034610 con 0x7fb68806b390 2024-03-08T06:00:13.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.272+0000 7fb68df99640 1 -- 172.21.15.81:0/2781514954 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb68806b390 msgr2=0x7fb68815c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.272+0000 7fb68df99640 1 --2- 172.21.15.81:0/2781514954 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb68806b390 0x7fb68815c540 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb680009e50 tx=0x7fb68002bf60 comp rx=0 tx=0).stop 2024-03-08T06:00:13.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.272+0000 7fb68df99640 1 -- 172.21.15.81:0/2781514954 shutdown_connections 2024-03-08T06:00:13.274 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.272+0000 7fb68df99640 1 --2- 172.21.15.81:0/2781514954 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb68806b390 0x7fb68815c540 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.274 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.272+0000 7fb68df99640 1 --2- 172.21.15.81:0/2781514954 >> v2:172.21.15.76:3300/0 conn(0x7fb6881594b0 0x7fb688160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.274 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.272+0000 7fb68df99640 1 -- 172.21.15.81:0/2781514954 >> 172.21.15.81:0/2781514954 conn(0x7fb688099a60 msgr2=0x7fb688099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:13.274 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.272+0000 7fb68df99640 1 -- 172.21.15.81:0/2781514954 shutdown_connections 2024-03-08T06:00:13.274 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.272+0000 7fb68df99640 1 -- 172.21.15.81:0/2781514954 wait complete. 2024-03-08T06:00:13.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.273+0000 7fb68df99640 1 Processor -- start 2024-03-08T06:00:13.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.274+0000 7fb68df99640 1 -- start start 2024-03-08T06:00:13.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.274+0000 7fb68df99640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb68806b390 0x7fb688158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.274+0000 7fb68df99640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb6881594b0 0x7fb6880d0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.274+0000 7fb68df99640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb68815c0b0 0x7fb68806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.274+0000 7fb68df99640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb688159890 con 0x7fb68815c0b0 2024-03-08T06:00:13.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.274+0000 7fb68df99640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68815cf50 con 0x7fb68806b390 2024-03-08T06:00:13.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.274+0000 7fb68df99640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb688159a10 con 0x7fb6881594b0 2024-03-08T06:00:13.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.274+0000 7fb6877fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb68806b390 0x7fb688158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.274+0000 7fb686ffd640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb6881594b0 0x7fb6880d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.274+0000 7fb6877fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb68806b390 0x7fb688158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45764/0 (socket says 172.21.15.81:45764) 2024-03-08T06:00:13.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.274+0000 7fb6877fe640 1 -- 172.21.15.81:0/2067233148 learned_addr learned my addr 172.21.15.81:0/2067233148 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-08T06:00:13.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.274+0000 7fb687fff640 1 --2- 172.21.15.81:0/2067233148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb68815c0b0 0x7fb68806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.275+0000 7fb686ffd640 1 -- 172.21.15.81:0/2067233148 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb68806b390 msgr2=0x7fb688158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.275+0000 7fb686ffd640 1 --2- 172.21.15.81:0/2067233148 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb68806b390 0x7fb688158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.275+0000 7fb686ffd640 1 -- 172.21.15.81:0/2067233148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb68815c0b0 msgr2=0x7fb68806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.275+0000 7fb686ffd640 1 --2- 172.21.15.81:0/2067233148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb68815c0b0 0x7fb68806ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.275+0000 7fb686ffd640 1 -- 172.21.15.81:0/2067233148 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6880d0ec0 con 0x7fb6881594b0 2024-03-08T06:00:13.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.275+0000 7fb6877fe640 1 --2- 172.21.15.81:0/2067233148 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb68806b390 0x7fb688158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:00:13.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.275+0000 7fb686ffd640 1 --2- 172.21.15.81:0/2067233148 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb6881594b0 0x7fb6880d0980 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb67800c340 tx=0x7fb67800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:13.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.275+0000 7fb66ffff640 1 -- 172.21.15.81:0/2067233148 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6780052d0 con 0x7fb6881594b0 2024-03-08T06:00:13.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.275+0000 7fb66ffff640 1 -- 172.21.15.81:0/2067233148 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb678005490 con 0x7fb6881594b0 2024-03-08T06:00:13.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.275+0000 7fb66ffff640 1 -- 172.21.15.81:0/2067233148 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678007260 con 0x7fb6881594b0 2024-03-08T06:00:13.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.275+0000 7fb68df99640 1 -- 172.21.15.81:0/2067233148 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6881591f0 con 0x7fb6881594b0 2024-03-08T06:00:13.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.275+0000 7fb68df99640 1 -- 172.21.15.81:0/2067233148 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6880c8190 con 0x7fb6881594b0 2024-03-08T06:00:13.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.275+0000 7fb66ffff640 1 -- 172.21.15.81:0/2067233148 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb678007400 con 0x7fb6881594b0 2024-03-08T06:00:13.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.275+0000 7fb66ffff640 1 -- 172.21.15.81:0/2067233148 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fb678011590 con 0x7fb6881594b0 2024-03-08T06:00:13.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.276+0000 7fb68df99640 1 -- 172.21.15.81:0/2067233148 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68804f030 con 0x7fb6881594b0 2024-03-08T06:00:13.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.278+0000 7fb66ffff640 1 -- 172.21.15.81:0/2067233148 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb678003a50 con 0x7fb6881594b0 2024-03-08T06:00:13.293 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:13.292+0000 7f0db4c47040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T06:00:13.302 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:13.302+0000 7fe544e46040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T06:00:13.346 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:13.346+0000 7f0db4c47040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T06:00:13.349 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:13.349+0000 7fe544e46040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T06:00:13.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.350+0000 7fb68df99640 1 -- 172.21.15.81:0/2067233148 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f7dbb15c-028c-4e1f-94a4-4051041e3bdd", "id": 4} v 0) v1 -- 0x7fb6880cd4a0 con 0x7fb6881594b0 2024-03-08T06:00:13.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.353+0000 7fb66ffff640 1 -- 172.21.15.81:0/2067233148 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f7dbb15c-028c-4e1f-94a4-4051041e3bdd", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb678003610 con 0x7fb6881594b0 2024-03-08T06:00:13.354 INFO:teuthology.orchestra.run.smithi081.stdout:4 2024-03-08T06:00:13.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.354+0000 7fb68df99640 1 -- 172.21.15.81:0/2067233148 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb6881594b0 msgr2=0x7fb6880d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.354+0000 7fb68df99640 1 --2- 172.21.15.81:0/2067233148 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb6881594b0 0x7fb6880d0980 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb67800c340 tx=0x7fb67800c800 comp rx=0 tx=0).stop 2024-03-08T06:00:13.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.354+0000 7fb68df99640 1 -- 172.21.15.81:0/2067233148 shutdown_connections 2024-03-08T06:00:13.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.354+0000 7fb68df99640 1 --2- 172.21.15.81:0/2067233148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb68815c0b0 0x7fb68806ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.354+0000 7fb68df99640 1 --2- 172.21.15.81:0/2067233148 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb6881594b0 0x7fb6880d0980 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.354+0000 7fb68df99640 1 --2- 172.21.15.81:0/2067233148 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb68806b390 0x7fb688158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.354+0000 7fb68df99640 1 -- 172.21.15.81:0/2067233148 >> 172.21.15.81:0/2067233148 conn(0x7fb688099a60 msgr2=0x7fb6880ae530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:13.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.354+0000 7fb68df99640 1 -- 172.21.15.81:0/2067233148 shutdown_connections 2024-03-08T06:00:13.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.354+0000 7fb68df99640 1 -- 172.21.15.81:0/2067233148 wait complete. 2024-03-08T06:00:13.365 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 694b04e6-f982-4600-a221-c1f0cd7d6d99 5 2024-03-08T06:00:13.397 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:13.397+0000 7f0db4c47040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T06:00:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.465+0000 7f9e91dd0640 1 Processor -- start 2024-03-08T06:00:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.465+0000 7f9e91dd0640 1 -- start start 2024-03-08T06:00:13.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.465+0000 7f9e91dd0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9e8c15aa80 0x7f9e8c06e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.465+0000 7f9e91dd0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8c15b930 con 0x7f9e8c0ce440 2024-03-08T06:00:13.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.465+0000 7f9e91dd0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8c15bc10 con 0x7f9e8c06eec0 2024-03-08T06:00:13.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.465+0000 7f9e91dd0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c15bef0 con 0x7f9e8c15aa80 2024-03-08T06:00:13.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.465+0000 7f9e8b7fe640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9e8c15aa80 0x7f9e8c06e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.465+0000 7f9e8affd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f9e8c06eec0 0x7f9e8c15ae60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:57958/0 (socket says 172.21.15.81:57958) 2024-03-08T06:00:13.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.466+0000 7f9e8affd640 1 -- 172.21.15.81:0/2509049317 learned_addr learned my addr 172.21.15.81:0/2509049317 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-03-08T06:00:13.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.466+0000 7f9e8b7fe640 1 -- 172.21.15.81:0/2509049317 >> v1:172.21.15.81:6789/0 conn(0x7f9e8c06eec0 legacy=0x7f9e8c15ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.466+0000 7f9e8a7fc640 1 -- 172.21.15.81:0/2509049317 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544617072 0 0) 0x7f9e8c15bc10 con 0x7f9e8c06eec0 2024-03-08T06:00:13.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.466+0000 7f9e8b7fe640 1 -- 172.21.15.81:0/2509049317 >> v1:172.21.15.76:6789/0 conn(0x7f9e8c0ce440 legacy=0x7f9e8c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.466+0000 7f9e8b7fe640 1 -- 172.21.15.81:0/2509049317 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8c0cd320 con 0x7f9e8c15aa80 2024-03-08T06:00:13.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.466+0000 7f9e8b7fe640 1 --2- 172.21.15.81:0/2509049317 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9e8c15aa80 0x7f9e8c06e980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0097d0 tx=0x7f9e7c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f549e79db90ac3e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:13.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.467+0000 7f9e8a7fc640 1 -- 172.21.15.81:0/2509049317 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c035020 con 0x7f9e8c15aa80 2024-03-08T06:00:13.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.467+0000 7f9e8a7fc640 1 -- 172.21.15.81:0/2509049317 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7c031890 con 0x7f9e8c15aa80 2024-03-08T06:00:13.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.467+0000 7f9e8a7fc640 1 -- 172.21.15.81:0/2509049317 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c031c60 con 0x7f9e8c15aa80 2024-03-08T06:00:13.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.467+0000 7f9e91dd0640 1 -- 172.21.15.81:0/2509049317 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9e8c15aa80 msgr2=0x7f9e8c06e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.467+0000 7f9e91dd0640 1 --2- 172.21.15.81:0/2509049317 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9e8c15aa80 0x7f9e8c06e980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0097d0 tx=0x7f9e7c02b930 comp rx=0 tx=0).stop 2024-03-08T06:00:13.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.467+0000 7f9e91dd0640 1 -- 172.21.15.81:0/2509049317 shutdown_connections 2024-03-08T06:00:13.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.467+0000 7f9e91dd0640 1 --2- 172.21.15.81:0/2509049317 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9e8c15aa80 0x7f9e8c06e980 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.467+0000 7f9e91dd0640 1 -- 172.21.15.81:0/2509049317 >> 172.21.15.81:0/2509049317 conn(0x7f9e8c099a60 msgr2=0x7f9e8c099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:13.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.467+0000 7f9e91dd0640 1 -- 172.21.15.81:0/2509049317 shutdown_connections 2024-03-08T06:00:13.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.467+0000 7f9e91dd0640 1 -- 172.21.15.81:0/2509049317 wait complete. 2024-03-08T06:00:13.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.468+0000 7f9e91dd0640 1 Processor -- start 2024-03-08T06:00:13.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.469+0000 7f9e91dd0640 1 -- start start 2024-03-08T06:00:13.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.469+0000 7f9e91dd0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e8c06eec0 0x7f9e8c0c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.469+0000 7f9e91dd0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9e8c0ce440 0x7f9e8c0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.469+0000 7f9e91dd0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e8c15aa80 0x7f9e8c1f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.469+0000 7f9e91dd0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c0ce0d0 con 0x7f9e8c15aa80 2024-03-08T06:00:13.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.469+0000 7f9e91dd0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c0cdf50 con 0x7f9e8c06eec0 2024-03-08T06:00:13.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.469+0000 7f9e91dd0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c0cec20 con 0x7f9e8c0ce440 2024-03-08T06:00:13.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.469+0000 7f9e8b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e8c06eec0 0x7f9e8c0c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.469+0000 7f9e8affd640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9e8c0ce440 0x7f9e8c0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.469+0000 7f9e8b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e8c06eec0 0x7f9e8c0c34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45772/0 (socket says 172.21.15.81:45772) 2024-03-08T06:00:13.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.469+0000 7f9e8b7fe640 1 -- 172.21.15.81:0/2713033177 learned_addr learned my addr 172.21.15.81:0/2713033177 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-08T06:00:13.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.470+0000 7f9e8bfff640 1 --2- 172.21.15.81:0/2713033177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e8c15aa80 0x7f9e8c1f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.470+0000 7f9e8affd640 1 -- 172.21.15.81:0/2713033177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e8c06eec0 msgr2=0x7f9e8c0c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.470+0000 7f9e8affd640 1 --2- 172.21.15.81:0/2713033177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e8c06eec0 0x7f9e8c0c34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.470+0000 7f9e8affd640 1 -- 172.21.15.81:0/2713033177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e8c15aa80 msgr2=0x7f9e8c1f7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.470+0000 7f9e8affd640 1 --2- 172.21.15.81:0/2713033177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e8c15aa80 0x7f9e8c1f7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.470+0000 7f9e8affd640 1 -- 172.21.15.81:0/2713033177 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8c0aee40 con 0x7f9e8c0ce440 2024-03-08T06:00:13.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.470+0000 7f9e8b7fe640 1 --2- 172.21.15.81:0/2713033177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e8c06eec0 0x7f9e8c0c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:00:13.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.470+0000 7f9e8affd640 1 --2- 172.21.15.81:0/2713033177 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9e8c0ce440 0x7f9e8c0ae900 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000c410 tx=0x7f9e8000c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:13.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.470+0000 7f9e73fff640 1 -- 172.21.15.81:0/2713033177 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e800074e0 con 0x7f9e8c0ce440 2024-03-08T06:00:13.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.470+0000 7f9e73fff640 1 -- 172.21.15.81:0/2713033177 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e80007680 con 0x7f9e8c0ce440 2024-03-08T06:00:13.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.470+0000 7f9e91dd0640 1 -- 172.21.15.81:0/2713033177 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8c1f82c0 con 0x7f9e8c0ce440 2024-03-08T06:00:13.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.470+0000 7f9e73fff640 1 -- 172.21.15.81:0/2713033177 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e80004a00 con 0x7f9e8c0ce440 2024-03-08T06:00:13.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.470+0000 7f9e91dd0640 1 -- 172.21.15.81:0/2713033177 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8c0af3c0 con 0x7f9e8c0ce440 2024-03-08T06:00:13.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.470+0000 7f9e73fff640 1 -- 172.21.15.81:0/2713033177 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9e80011070 con 0x7f9e8c0ce440 2024-03-08T06:00:13.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.470+0000 7f9e73fff640 1 -- 172.21.15.81:0/2713033177 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f9e80013940 con 0x7f9e8c0ce440 2024-03-08T06:00:13.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.471+0000 7f9e91dd0640 1 -- 172.21.15.81:0/2713033177 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8c070410 con 0x7f9e8c0ce440 2024-03-08T06:00:13.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.475+0000 7f9e73fff640 1 -- 172.21.15.81:0/2713033177 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9e8c070410 con 0x7f9e8c0ce440 2024-03-08T06:00:13.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.547+0000 7f9e91dd0640 1 -- 172.21.15.81:0/2713033177 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd new", "uuid": "694b04e6-f982-4600-a221-c1f0cd7d6d99", "id": 5} v 0) v1 -- 0x7f9e8c0cffa0 con 0x7f9e8c0ce440 2024-03-08T06:00:13.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.553+0000 7f9e73fff640 1 -- 172.21.15.81:0/2713033177 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "694b04e6-f982-4600-a221-c1f0cd7d6d99", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9e80013c40 con 0x7f9e8c0ce440 2024-03-08T06:00:13.554 INFO:teuthology.orchestra.run.smithi081.stdout:5 2024-03-08T06:00:13.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.554+0000 7f9e91dd0640 1 -- 172.21.15.81:0/2713033177 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9e8c0ce440 msgr2=0x7f9e8c0ae900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.554+0000 7f9e91dd0640 1 --2- 172.21.15.81:0/2713033177 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9e8c0ce440 0x7f9e8c0ae900 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000c410 tx=0x7f9e8000c8d0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.554+0000 7f9e91dd0640 1 -- 172.21.15.81:0/2713033177 shutdown_connections 2024-03-08T06:00:13.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.554+0000 7f9e91dd0640 1 --2- 172.21.15.81:0/2713033177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e8c15aa80 0x7f9e8c1f7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.554+0000 7f9e91dd0640 1 --2- 172.21.15.81:0/2713033177 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9e8c0ce440 0x7f9e8c0ae900 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.554+0000 7f9e91dd0640 1 --2- 172.21.15.81:0/2713033177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e8c06eec0 0x7f9e8c0c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.554+0000 7f9e91dd0640 1 -- 172.21.15.81:0/2713033177 >> 172.21.15.81:0/2713033177 conn(0x7f9e8c099a60 msgr2=0x7f9e8c069a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:13.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.554+0000 7f9e91dd0640 1 -- 172.21.15.81:0/2713033177 shutdown_connections 2024-03-08T06:00:13.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.554+0000 7f9e91dd0640 1 -- 172.21.15.81:0/2713033177 wait complete. 2024-03-08T06:00:13.565 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 58469229-276a-4a60-99bc-2ab5ee99d9b3 6 2024-03-08T06:00:13.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.706+0000 7fbb81e47640 1 Processor -- start 2024-03-08T06:00:13.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.706+0000 7fbb81e47640 1 -- start start 2024-03-08T06:00:13.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.707+0000 7fbb81e47640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbb7c15e120 0x7fbb7c0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.707+0000 7fbb81e47640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb7c15e500 0x7fbb7c0ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.707+0000 7fbb81e47640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7c0b1550 con 0x7fbb7c0cf4a0 2024-03-08T06:00:13.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.707+0000 7fbb81e47640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb7c0ae1d0 con 0x7fbb7c15e120 2024-03-08T06:00:13.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.707+0000 7fbb81e47640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb7c0ae4b0 con 0x7fbb7c15e500 2024-03-08T06:00:13.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.707+0000 7fbb7affd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb7c15e500 0x7fbb7c0ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.707+0000 7fbb7affd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb7c15e500 0x7fbb7c0ce900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45784/0 (socket says 172.21.15.81:45784) 2024-03-08T06:00:13.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.707+0000 7fbb7affd640 1 -- 172.21.15.81:0/925569197 learned_addr learned my addr 172.21.15.81:0/925569197 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-08T06:00:13.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.707+0000 7fbb7b7fe640 1 --2- 172.21.15.81:0/925569197 >> v2:172.21.15.76:3300/0 conn(0x7fbb7c15e120 0x7fbb7c0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.708+0000 7fbb7affd640 1 -- 172.21.15.81:0/925569197 >> v2:172.21.15.76:3300/0 conn(0x7fbb7c15e120 msgr2=0x7fbb7c0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:13.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.708+0000 7fbb7affd640 1 --2- 172.21.15.81:0/925569197 >> v2:172.21.15.76:3300/0 conn(0x7fbb7c15e120 0x7fbb7c0cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.708+0000 7fbb7affd640 1 -- 172.21.15.81:0/925569197 >> v1:172.21.15.76:6789/0 conn(0x7fbb7c0cf4a0 legacy=0x7fbb7c0b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.708+0000 7fbb7affd640 1 -- 172.21.15.81:0/925569197 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c0ae790 con 0x7fbb7c15e500 2024-03-08T06:00:13.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.708+0000 7fbb7affd640 1 --2- 172.21.15.81:0/925569197 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb7c15e500 0x7fbb7c0ce900 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0b3390 tx=0x7fbb6c02b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5f19df79bb10e0b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:13.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.708+0000 7fbb7a7fc640 1 -- 172.21.15.81:0/925569197 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c034070 con 0x7fbb7c15e500 2024-03-08T06:00:13.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.708+0000 7fbb7a7fc640 1 -- 172.21.15.81:0/925569197 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c030860 con 0x7fbb7c15e500 2024-03-08T06:00:13.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.708+0000 7fbb7a7fc640 1 -- 172.21.15.81:0/925569197 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c030c30 con 0x7fbb7c15e500 2024-03-08T06:00:13.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.709+0000 7fbb81e47640 1 -- 172.21.15.81:0/925569197 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb7c15e500 msgr2=0x7fbb7c0ce900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.709+0000 7fbb81e47640 1 --2- 172.21.15.81:0/925569197 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb7c15e500 0x7fbb7c0ce900 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0b3390 tx=0x7fbb6c02b9a0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.709+0000 7fbb81e47640 1 -- 172.21.15.81:0/925569197 shutdown_connections 2024-03-08T06:00:13.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.709+0000 7fbb81e47640 1 --2- 172.21.15.81:0/925569197 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb7c15e500 0x7fbb7c0ce900 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.709+0000 7fbb81e47640 1 --2- 172.21.15.81:0/925569197 >> v2:172.21.15.76:3300/0 conn(0x7fbb7c15e120 0x7fbb7c0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.709+0000 7fbb81e47640 1 -- 172.21.15.81:0/925569197 >> 172.21.15.81:0/925569197 conn(0x7fbb7c0d0890 msgr2=0x7fbb7c0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:13.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.709+0000 7fbb81e47640 1 -- 172.21.15.81:0/925569197 shutdown_connections 2024-03-08T06:00:13.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.709+0000 7fbb81e47640 1 -- 172.21.15.81:0/925569197 wait complete. 2024-03-08T06:00:13.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.710+0000 7fbb81e47640 1 Processor -- start 2024-03-08T06:00:13.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.710+0000 7fbb81e47640 1 -- start start 2024-03-08T06:00:13.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.711+0000 7fbb81e47640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb7c0cf4a0 0x7fbb7c14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.711+0000 7fbb81e47640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb7c15e120 0x7fbb7c0c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.711+0000 7fbb81e47640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb7c15e500 0x7fbb7c0c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.711+0000 7fbb81e47640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0b1da0 con 0x7fbb7c15e500 2024-03-08T06:00:13.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.711+0000 7fbb81e47640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0d1590 con 0x7fbb7c0cf4a0 2024-03-08T06:00:13.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.711+0000 7fbb81e47640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0b1f20 con 0x7fbb7c15e120 2024-03-08T06:00:13.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.711+0000 7fbb7b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb7c0cf4a0 0x7fbb7c14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.711+0000 7fbb7affd640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb7c15e120 0x7fbb7c0c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.711+0000 7fbb7b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb7c0cf4a0 0x7fbb7c14fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45798/0 (socket says 172.21.15.81:45798) 2024-03-08T06:00:13.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.711+0000 7fbb7b7fe640 1 -- 172.21.15.81:0/1148810489 learned_addr learned my addr 172.21.15.81:0/1148810489 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-08T06:00:13.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.711+0000 7fbb7bfff640 1 --2- 172.21.15.81:0/1148810489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb7c15e500 0x7fbb7c0c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.712+0000 7fbb7affd640 1 -- 172.21.15.81:0/1148810489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb7c0cf4a0 msgr2=0x7fbb7c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.712+0000 7fbb7affd640 1 --2- 172.21.15.81:0/1148810489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb7c0cf4a0 0x7fbb7c14fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.712+0000 7fbb7affd640 1 -- 172.21.15.81:0/1148810489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb7c15e500 msgr2=0x7fbb7c0c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.712+0000 7fbb7affd640 1 --2- 172.21.15.81:0/1148810489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb7c15e500 0x7fbb7c0c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.712+0000 7fbb7affd640 1 -- 172.21.15.81:0/1148810489 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c150240 con 0x7fbb7c15e120 2024-03-08T06:00:13.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.712+0000 7fbb7b7fe640 1 --2- 172.21.15.81:0/1148810489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb7c0cf4a0 0x7fbb7c14fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:00:13.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.712+0000 7fbb7affd640 1 --2- 172.21.15.81:0/1148810489 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb7c15e120 0x7fbb7c0c5eb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0b1720 tx=0x7fbb6c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:13.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.712+0000 7fbb63fff640 1 -- 172.21.15.81:0/1148810489 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c034040 con 0x7fbb7c15e120 2024-03-08T06:00:13.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.712+0000 7fbb63fff640 1 -- 172.21.15.81:0/1148810489 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c03ada0 con 0x7fbb7c15e120 2024-03-08T06:00:13.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.712+0000 7fbb63fff640 1 -- 172.21.15.81:0/1148810489 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c044650 con 0x7fbb7c15e120 2024-03-08T06:00:13.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.712+0000 7fbb81e47640 1 -- 172.21.15.81:0/1148810489 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7c0c4e10 con 0x7fbb7c15e120 2024-03-08T06:00:13.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.712+0000 7fbb81e47640 1 -- 172.21.15.81:0/1148810489 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c0c53d0 con 0x7fbb7c15e120 2024-03-08T06:00:13.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.712+0000 7fbb63fff640 1 -- 172.21.15.81:0/1148810489 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbb6c037070 con 0x7fbb7c15e120 2024-03-08T06:00:13.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.712+0000 7fbb63fff640 1 -- 172.21.15.81:0/1148810489 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fbb6c0447f0 con 0x7fbb7c15e120 2024-03-08T06:00:13.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.713+0000 7fbb81e47640 1 -- 172.21.15.81:0/1148810489 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7c04f030 con 0x7fbb7c15e120 2024-03-08T06:00:13.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.715+0000 7fbb63fff640 1 -- 172.21.15.81:0/1148810489 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbb6c03f070 con 0x7fbb7c15e120 2024-03-08T06:00:13.726 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:13.725+0000 7fe544e46040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T06:00:13.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.789+0000 7fbb81e47640 1 -- 172.21.15.81:0/1148810489 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd new", "uuid": "58469229-276a-4a60-99bc-2ab5ee99d9b3", "id": 6} v 0) v1 -- 0x7fbb7c0d0f30 con 0x7fbb7c15e120 2024-03-08T06:00:13.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.794+0000 7fbb63fff640 1 -- 172.21.15.81:0/1148810489 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "58469229-276a-4a60-99bc-2ab5ee99d9b3", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbb6c044ac0 con 0x7fbb7c15e120 2024-03-08T06:00:13.794 INFO:teuthology.orchestra.run.smithi081.stdout:6 2024-03-08T06:00:13.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.795+0000 7fbb81e47640 1 -- 172.21.15.81:0/1148810489 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb7c15e120 msgr2=0x7fbb7c0c5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.795+0000 7fbb81e47640 1 --2- 172.21.15.81:0/1148810489 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb7c15e120 0x7fbb7c0c5eb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0b1720 tx=0x7fbb6c0046c0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.795+0000 7fbb81e47640 1 -- 172.21.15.81:0/1148810489 shutdown_connections 2024-03-08T06:00:13.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.795+0000 7fbb81e47640 1 --2- 172.21.15.81:0/1148810489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb7c15e500 0x7fbb7c0c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.795+0000 7fbb81e47640 1 --2- 172.21.15.81:0/1148810489 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb7c15e120 0x7fbb7c0c5eb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.795+0000 7fbb81e47640 1 --2- 172.21.15.81:0/1148810489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb7c0cf4a0 0x7fbb7c14fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.795+0000 7fbb81e47640 1 -- 172.21.15.81:0/1148810489 >> 172.21.15.81:0/1148810489 conn(0x7fbb7c0d0890 msgr2=0x7fbb7c15d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:13.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.795+0000 7fbb81e47640 1 -- 172.21.15.81:0/1148810489 shutdown_connections 2024-03-08T06:00:13.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.795+0000 7fbb81e47640 1 -- 172.21.15.81:0/1148810489 wait complete. 2024-03-08T06:00:13.797 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:13.797+0000 7f0db4c47040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T06:00:13.806 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 685d3757-2cbd-4ff6-8ef1-8e391d9a64f5 7 2024-03-08T06:00:13.861 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:13.861+0000 7fe544e46040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T06:00:13.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.905+0000 7fcad2581640 1 Processor -- start 2024-03-08T06:00:13.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.905+0000 7fcad2581640 1 -- start start 2024-03-08T06:00:13.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcad2581640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcacc06ae40 0x7fcacc06b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcad2581640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcacc0d15b0 con 0x7fcacc161940 2024-03-08T06:00:13.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcad2581640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcacc06fdf0 con 0x7fcacc0afbf0 2024-03-08T06:00:13.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcad2581640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcacc0700d0 con 0x7fcacc06ae40 2024-03-08T06:00:13.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacbfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcacc0afbf0 0x7fcacc0d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:57962/0 (socket says 172.21.15.81:57962) 2024-03-08T06:00:13.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacbfff640 1 -- 172.21.15.81:0/3536136016 learned_addr learned my addr 172.21.15.81:0/3536136016 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-03-08T06:00:13.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacb7fe640 1 --2- 172.21.15.81:0/3536136016 >> v2:172.21.15.76:3300/0 conn(0x7fcacc06ae40 0x7fcacc06b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 -- 172.21.15.81:0/3536136016 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606213076 0 0) 0x7fcacc06fdf0 con 0x7fcacc0afbf0 2024-03-08T06:00:13.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 -- 172.21.15.81:0/3536136016 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcab4003520 con 0x7fcacc0afbf0 2024-03-08T06:00:13.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 -- 172.21.15.81:0/3536136016 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797892945 0 0) 0x7fcacc0d15b0 con 0x7fcacc161940 2024-03-08T06:00:13.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 -- 172.21.15.81:0/3536136016 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcacc06fdf0 con 0x7fcacc161940 2024-03-08T06:00:13.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 -- 172.21.15.81:0/3536136016 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3599888372 0 0) 0x7fcab4003520 con 0x7fcacc0afbf0 2024-03-08T06:00:13.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 -- 172.21.15.81:0/3536136016 >> v2:172.21.15.76:3300/0 conn(0x7fcacc06ae40 msgr2=0x7fcacc06b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:13.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 --2- 172.21.15.81:0/3536136016 >> v2:172.21.15.76:3300/0 conn(0x7fcacc06ae40 0x7fcacc06b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 -- 172.21.15.81:0/3536136016 >> v1:172.21.15.76:6789/0 conn(0x7fcacc161940 legacy=0x7fcacc0cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 -- 172.21.15.81:0/3536136016 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc0703b0 con 0x7fcacc0afbf0 2024-03-08T06:00:13.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 -- 172.21.15.81:0/3536136016 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fcac0001670 con 0x7fcacc0afbf0 2024-03-08T06:00:13.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 -- 172.21.15.81:0/3536136016 >> v1:172.21.15.81:6789/0 conn(0x7fcacc0afbf0 legacy=0x7fcacc0d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 --2- 172.21.15.81:0/3536136016 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcab4003bb0 0x7fcab4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 --2- 172.21.15.81:0/3536136016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcab4004800 0x7fcab4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 --2- 172.21.15.81:0/3536136016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab40051c0 0x7fcab400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 -- 172.21.15.81:0/3536136016 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0700d0 con 0x7fcab4004800 2024-03-08T06:00:13.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 -- 172.21.15.81:0/3536136016 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab400c0c0 con 0x7fcab40051c0 2024-03-08T06:00:13.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacaffd640 1 -- 172.21.15.81:0/3536136016 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab400c2c0 con 0x7fcab4003bb0 2024-03-08T06:00:13.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcacbfff640 1 --2- 172.21.15.81:0/3536136016 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcab4003bb0 0x7fcab4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.906+0000 7fcad0b1e640 1 --2- 172.21.15.81:0/3536136016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab40051c0 0x7fcab400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.907+0000 7fcacb7fe640 1 --2- 172.21.15.81:0/3536136016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcab4004800 0x7fcab4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.907+0000 7fcacbfff640 1 -- 172.21.15.81:0/3536136016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab40051c0 msgr2=0x7fcab400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:13.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.907+0000 7fcacbfff640 1 --2- 172.21.15.81:0/3536136016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab40051c0 0x7fcab400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.907+0000 7fcacbfff640 1 -- 172.21.15.81:0/3536136016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcab4004800 msgr2=0x7fcab4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:13.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.907+0000 7fcacbfff640 1 --2- 172.21.15.81:0/3536136016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcab4004800 0x7fcab4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.907+0000 7fcacbfff640 1 -- 172.21.15.81:0/3536136016 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab4003520 con 0x7fcab4003bb0 2024-03-08T06:00:13.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.907+0000 7fcad0b1e640 1 --2- 172.21.15.81:0/3536136016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab40051c0 0x7fcab400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:00:13.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.907+0000 7fcad0b1e640 1 -- 172.21.15.81:0/3536136016 reap_dead start 2024-03-08T06:00:13.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.908+0000 7fcacbfff640 1 --2- 172.21.15.81:0/3536136016 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcab4003bb0 0x7fcab4003fb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcac000c160 tx=0x7fcab4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23599ae6922b61b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:13.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.908+0000 7fcacaffd640 1 -- 172.21.15.81:0/3536136016 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac0007ca0 con 0x7fcab4003bb0 2024-03-08T06:00:13.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.908+0000 7fcacaffd640 1 -- 172.21.15.81:0/3536136016 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac0030070 con 0x7fcab4003bb0 2024-03-08T06:00:13.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.908+0000 7fcacaffd640 1 -- 172.21.15.81:0/3536136016 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac0005d70 con 0x7fcab4003bb0 2024-03-08T06:00:13.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.908+0000 7fcad2581640 1 -- 172.21.15.81:0/3536136016 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcab4003bb0 msgr2=0x7fcab4003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.908+0000 7fcad2581640 1 --2- 172.21.15.81:0/3536136016 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcab4003bb0 0x7fcab4003fb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcac000c160 tx=0x7fcab4005830 comp rx=0 tx=0).stop 2024-03-08T06:00:13.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.908+0000 7fcad2581640 1 -- 172.21.15.81:0/3536136016 shutdown_connections 2024-03-08T06:00:13.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.908+0000 7fcad2581640 1 --2- 172.21.15.81:0/3536136016 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcab4003bb0 0x7fcab4003fb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.908+0000 7fcad2581640 1 -- 172.21.15.81:0/3536136016 >> 172.21.15.81:0/3536136016 conn(0x7fcacc099a60 msgr2=0x7fcacc099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:13.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.908+0000 7fcad2581640 1 -- 172.21.15.81:0/3536136016 shutdown_connections 2024-03-08T06:00:13.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.908+0000 7fcad2581640 1 -- 172.21.15.81:0/3536136016 wait complete. 2024-03-08T06:00:13.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.909+0000 7fcad2581640 1 Processor -- start 2024-03-08T06:00:13.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.910+0000 7fcad2581640 1 -- start start 2024-03-08T06:00:13.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.910+0000 7fcad2581640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab4003bb0 0x7fcacc1f1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.910+0000 7fcad2581640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcacc1f4670 0x7fcacc0b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.910+0000 7fcad2581640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcacc0c9b60 0x7fcacc0c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:13.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.910+0000 7fcad2581640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0affd0 con 0x7fcacc0c9b60 2024-03-08T06:00:13.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.910+0000 7fcad2581640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc06fa80 con 0x7fcab4003bb0 2024-03-08T06:00:13.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.910+0000 7fcad2581640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0b0150 con 0x7fcacc1f4670 2024-03-08T06:00:13.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.910+0000 7fcacbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab4003bb0 0x7fcacc1f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.910+0000 7fcacb7fe640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcacc1f4670 0x7fcacc0b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcacb7fe640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcacc1f4670 0x7fcacc0b2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.81:51908/0 (socket says 172.21.15.81:51908) 2024-03-08T06:00:13.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcacbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab4003bb0 0x7fcacc1f1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45818/0 (socket says 172.21.15.81:45818) 2024-03-08T06:00:13.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcacb7fe640 1 -- 172.21.15.81:0/2922856973 learned_addr learned my addr 172.21.15.81:0/2922856973 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-08T06:00:13.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcad0b1e640 1 --2- 172.21.15.81:0/2922856973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcacc0c9b60 0x7fcacc0c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:13.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcacb7fe640 1 -- 172.21.15.81:0/2922856973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab4003bb0 msgr2=0x7fcacc1f1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcacb7fe640 1 --2- 172.21.15.81:0/2922856973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab4003bb0 0x7fcacc1f1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcacb7fe640 1 -- 172.21.15.81:0/2922856973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcacc0c9b60 msgr2=0x7fcacc0c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcacb7fe640 1 --2- 172.21.15.81:0/2922856973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcacc0c9b60 0x7fcacc0c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcacb7fe640 1 -- 172.21.15.81:0/2922856973 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc0b3070 con 0x7fcacc1f4670 2024-03-08T06:00:13.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcacb7fe640 1 --2- 172.21.15.81:0/2922856973 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcacc1f4670 0x7fcacc0b2b30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0119c0 tx=0x7fcabc011dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:13.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcac8ff9640 1 -- 172.21.15.81:0/2922856973 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc009590 con 0x7fcacc1f4670 2024-03-08T06:00:13.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcac8ff9640 1 -- 172.21.15.81:0/2922856973 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcabc00faf0 con 0x7fcacc1f4670 2024-03-08T06:00:13.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcad2581640 1 -- 172.21.15.81:0/2922856973 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcacc1f72f0 con 0x7fcacc1f4670 2024-03-08T06:00:13.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcac8ff9640 1 -- 172.21.15.81:0/2922856973 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc014440 con 0x7fcacc1f4670 2024-03-08T06:00:13.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcad2581640 1 -- 172.21.15.81:0/2922856973 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcacc1f78b0 con 0x7fcacc1f4670 2024-03-08T06:00:13.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcac8ff9640 1 -- 172.21.15.81:0/2922856973 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fcabc012040 con 0x7fcacc1f4670 2024-03-08T06:00:13.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.911+0000 7fcac8ff9640 1 -- 172.21.15.81:0/2922856973 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fcabc01dce0 con 0x7fcacc1f4670 2024-03-08T06:00:13.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.912+0000 7fcad2581640 1 -- 172.21.15.81:0/2922856973 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcacc0ce2b0 con 0x7fcacc1f4670 2024-03-08T06:00:13.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.916+0000 7fcac8ff9640 1 -- 172.21.15.81:0/2922856973 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fcabc019030 con 0x7fcacc1f4670 2024-03-08T06:00:13.940 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:13.940+0000 7f0db4c47040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T06:00:13.978 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:13.978+0000 7fe544e46040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T06:00:13.988 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.988+0000 7fcad2581640 1 -- 172.21.15.81:0/2922856973 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd new", "uuid": "685d3757-2cbd-4ff6-8ef1-8e391d9a64f5", "id": 7} v 0) v1 -- 0x7fcacc0cd4a0 con 0x7fcacc1f4670 2024-03-08T06:00:13.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.993+0000 7fcac8ff9640 1 -- 172.21.15.81:0/2922856973 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "685d3757-2cbd-4ff6-8ef1-8e391d9a64f5", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcabc01d750 con 0x7fcacc1f4670 2024-03-08T06:00:13.994 INFO:teuthology.orchestra.run.smithi081.stdout:7 2024-03-08T06:00:13.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.994+0000 7fcad2581640 1 -- 172.21.15.81:0/2922856973 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcacc1f4670 msgr2=0x7fcacc0b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:13.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.994+0000 7fcad2581640 1 --2- 172.21.15.81:0/2922856973 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcacc1f4670 0x7fcacc0b2b30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0119c0 tx=0x7fcabc011dc0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.994+0000 7fcad2581640 1 -- 172.21.15.81:0/2922856973 shutdown_connections 2024-03-08T06:00:13.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.994+0000 7fcad2581640 1 --2- 172.21.15.81:0/2922856973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcacc0c9b60 0x7fcacc0c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.994+0000 7fcad2581640 1 --2- 172.21.15.81:0/2922856973 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcacc1f4670 0x7fcacc0b2b30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.994+0000 7fcad2581640 1 --2- 172.21.15.81:0/2922856973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab4003bb0 0x7fcacc1f1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:13.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.994+0000 7fcad2581640 1 -- 172.21.15.81:0/2922856973 >> 172.21.15.81:0/2922856973 conn(0x7fcacc099a60 msgr2=0x7fcacc160c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:13.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.995+0000 7fcad2581640 1 -- 172.21.15.81:0/2922856973 shutdown_connections 2024-03-08T06:00:13.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-08T06:00:13.995+0000 7fcad2581640 1 -- 172.21.15.81:0/2922856973 wait complete. 2024-03-08T06:00:14.005 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-08T06:00:14.005 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-08T06:00:14.007 INFO:tasks.ceph.osd.0:Started 2024-03-08T06:00:14.008 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-08T06:00:14.008 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-08T06:00:14.010 INFO:tasks.ceph.osd.1:Started 2024-03-08T06:00:14.010 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-08T06:00:14.010 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-08T06:00:14.012 INFO:tasks.ceph.osd.2:Started 2024-03-08T06:00:14.012 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-08T06:00:14.012 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-08T06:00:14.014 INFO:tasks.ceph.osd.3:Started 2024-03-08T06:00:14.014 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-08T06:00:14.015 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-08T06:00:14.017 INFO:tasks.ceph.osd.4:Started 2024-03-08T06:00:14.017 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-08T06:00:14.017 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-08T06:00:14.019 INFO:tasks.ceph.osd.5:Started 2024-03-08T06:00:14.019 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-08T06:00:14.019 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-08T06:00:14.021 INFO:tasks.ceph.osd.6:Started 2024-03-08T06:00:14.021 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-08T06:00:14.021 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-08T06:00:14.023 INFO:tasks.ceph.osd.7:Started 2024-03-08T06:00:14.024 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:00:14.029 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:14.029+0000 7fe544e46040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T06:00:14.075 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:14.075+0000 7f0db4c47040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T06:00:14.132 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:14.132+0000 7f0db4c47040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T06:00:14.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.148+0000 7f28d13e9640 1 Processor -- start 2024-03-08T06:00:14.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.148+0000 7f28d13e9640 1 -- start start 2024-03-08T06:00:14.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.149+0000 7f28d13e9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f28cc15a9f0 0x7f28cc071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:14.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.149+0000 7f28d13e9640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28cc0cec20 0x7f28cc0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:14.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.149+0000 7f28d13e9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28cc0722e0 con 0x7f28cc0c98e0 2024-03-08T06:00:14.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.149+0000 7f28d13e9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28cc0ce790 con 0x7f28cc15a9f0 2024-03-08T06:00:14.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.149+0000 7f28d13e9640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0ae8d0 con 0x7f28cc0cec20 2024-03-08T06:00:14.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.149+0000 7f28caffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f28cc15a9f0 0x7f28cc071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:14.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.149+0000 7f28caffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f28cc15a9f0 0x7f28cc071da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37686/0 (socket says 172.21.15.76:37686) 2024-03-08T06:00:14.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.149+0000 7f28caffd640 1 -- 172.21.15.76:0/2656349468 learned_addr learned my addr 172.21.15.76:0/2656349468 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:14.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.149+0000 7f28cb7fe640 1 --2- 172.21.15.76:0/2656349468 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28cc0cec20 0x7f28cc0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:14.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.149+0000 7f28caffd640 1 -- 172.21.15.76:0/2656349468 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28cc0cec20 msgr2=0x7f28cc0cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:14.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.149+0000 7f28caffd640 1 --2- 172.21.15.76:0/2656349468 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28cc0cec20 0x7f28cc0cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:14.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.149+0000 7f28caffd640 1 -- 172.21.15.76:0/2656349468 >> v1:172.21.15.76:6789/0 conn(0x7f28cc0c98e0 legacy=0x7f28cc0c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:14.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.149+0000 7f28caffd640 1 -- 172.21.15.76:0/2656349468 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc0aebb0 con 0x7f28cc15a9f0 2024-03-08T06:00:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.150+0000 7f28caffd640 1 --2- 172.21.15.76:0/2656349468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc15a9f0 0x7f28cc071da0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f28c400bb10 tx=0x7f28c402e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50f2f457b96f117 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.150+0000 7f28c9ffb640 1 -- 172.21.15.76:0/2656349468 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c403b070 con 0x7f28cc15a9f0 2024-03-08T06:00:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.150+0000 7f28c9ffb640 1 -- 172.21.15.76:0/2656349468 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c4036440 con 0x7f28cc15a9f0 2024-03-08T06:00:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.150+0000 7f28c9ffb640 1 -- 172.21.15.76:0/2656349468 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4036810 con 0x7f28cc15a9f0 2024-03-08T06:00:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.151+0000 7f28d13e9640 1 -- 172.21.15.76:0/2656349468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc15a9f0 msgr2=0x7f28cc071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.151+0000 7f28d13e9640 1 --2- 172.21.15.76:0/2656349468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc15a9f0 0x7f28cc071da0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f28c400bb10 tx=0x7f28c402e150 comp rx=0 tx=0).stop 2024-03-08T06:00:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.151+0000 7f28d13e9640 1 -- 172.21.15.76:0/2656349468 shutdown_connections 2024-03-08T06:00:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.151+0000 7f28d13e9640 1 --2- 172.21.15.76:0/2656349468 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28cc0cec20 0x7f28cc0cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.151+0000 7f28d13e9640 1 --2- 172.21.15.76:0/2656349468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc15a9f0 0x7f28cc071da0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.151+0000 7f28d13e9640 1 -- 172.21.15.76:0/2656349468 >> 172.21.15.76:0/2656349468 conn(0x7f28cc0cf740 msgr2=0x7f28cc0cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.151+0000 7f28d13e9640 1 -- 172.21.15.76:0/2656349468 shutdown_connections 2024-03-08T06:00:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.151+0000 7f28d13e9640 1 -- 172.21.15.76:0/2656349468 wait complete. 2024-03-08T06:00:14.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.152+0000 7f28d13e9640 1 Processor -- start 2024-03-08T06:00:14.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.152+0000 7f28d13e9640 1 -- start start 2024-03-08T06:00:14.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28d13e9640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28cc0c98e0 0x7f28cc1f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:14.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28d13e9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc0cec20 0x7f28cc0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:14.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28d13e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28cc15a9f0 0x7f28cc15c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:14.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28d13e9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0ca620 con 0x7f28cc0cec20 2024-03-08T06:00:14.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28d13e9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0ca4a0 con 0x7f28cc15a9f0 2024-03-08T06:00:14.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28d13e9640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc15add0 con 0x7f28cc0c98e0 2024-03-08T06:00:14.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28ca7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc0cec20 0x7f28cc0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:14.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28ca7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc0cec20 0x7f28cc0b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37698/0 (socket says 172.21.15.76:37698) 2024-03-08T06:00:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28ca7fc640 1 -- 172.21.15.76:0/2784567483 learned_addr learned my addr 172.21.15.76:0/2784567483 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28caffd640 1 --2- 172.21.15.76:0/2784567483 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28cc0c98e0 0x7f28cc1f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28cb7fe640 1 --2- 172.21.15.76:0/2784567483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28cc15a9f0 0x7f28cc15c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28ca7fc640 1 -- 172.21.15.76:0/2784567483 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28cc0c98e0 msgr2=0x7f28cc1f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28ca7fc640 1 --2- 172.21.15.76:0/2784567483 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28cc0c98e0 0x7f28cc1f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28ca7fc640 1 -- 172.21.15.76:0/2784567483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28cc15a9f0 msgr2=0x7f28cc15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28ca7fc640 1 --2- 172.21.15.76:0/2784567483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28cc15a9f0 0x7f28cc15c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28ca7fc640 1 -- 172.21.15.76:0/2784567483 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc15cae0 con 0x7f28cc0cec20 2024-03-08T06:00:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.153+0000 7f28ca7fc640 1 --2- 172.21.15.76:0/2784567483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc0cec20 0x7f28cc0b3190 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f28c000c3c0 tx=0x7f28c000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.154+0000 7f28b37fe640 1 -- 172.21.15.76:0/2784567483 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0017070 con 0x7f28cc0cec20 2024-03-08T06:00:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.154+0000 7f28b37fe640 1 -- 172.21.15.76:0/2784567483 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c00052f0 con 0x7f28cc0cec20 2024-03-08T06:00:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.154+0000 7f28b37fe640 1 -- 172.21.15.76:0/2784567483 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0003850 con 0x7f28cc0cec20 2024-03-08T06:00:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.154+0000 7f28d13e9640 1 -- 172.21.15.76:0/2784567483 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28cc1f4a40 con 0x7f28cc0cec20 2024-03-08T06:00:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.154+0000 7f28d13e9640 1 -- 172.21.15.76:0/2784567483 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28cc1f4480 con 0x7f28cc0cec20 2024-03-08T06:00:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.154+0000 7f28b37fe640 1 -- 172.21.15.76:0/2784567483 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f28c001a070 con 0x7f28cc0cec20 2024-03-08T06:00:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.154+0000 7f28b37fe640 1 -- 172.21.15.76:0/2784567483 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f28c0011ca0 con 0x7f28cc0cec20 2024-03-08T06:00:14.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.155+0000 7f28d13e9640 1 -- 172.21.15.76:0/2784567483 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28cc04f030 con 0x7f28cc0cec20 2024-03-08T06:00:14.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.157+0000 7f28b37fe640 1 -- 172.21.15.76:0/2784567483 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f28c0021080 con 0x7f28cc0cec20 2024-03-08T06:00:14.173 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:14.173+0000 7fe544e46040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T06:00:14.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.218+0000 7f28d13e9640 1 -- 172.21.15.76:0/2784567483 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28cc0725a0 con 0x7f28cc0cec20 2024-03-08T06:00:14.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.219+0000 7f28b37fe640 1 -- 172.21.15.76:0/2784567483 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f28cc15cae0 con 0x7f28cc0cec20 2024-03-08T06:00:14.219 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:14.219 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":10,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:13.990205+0000","last_up_change":"0.000000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:00:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.219+0000 7f28d13e9640 1 -- 172.21.15.76:0/2784567483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc0cec20 msgr2=0x7f28cc0b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.219+0000 7f28d13e9640 1 --2- 172.21.15.76:0/2784567483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc0cec20 0x7f28cc0b3190 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f28c000c3c0 tx=0x7f28c000c880 comp rx=0 tx=0).stop 2024-03-08T06:00:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.220+0000 7f28d13e9640 1 -- 172.21.15.76:0/2784567483 shutdown_connections 2024-03-08T06:00:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.220+0000 7f28d13e9640 1 --2- 172.21.15.76:0/2784567483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28cc15a9f0 0x7f28cc15c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.220+0000 7f28d13e9640 1 --2- 172.21.15.76:0/2784567483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc0cec20 0x7f28cc0b3190 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.220+0000 7f28d13e9640 1 --2- 172.21.15.76:0/2784567483 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28cc0c98e0 0x7f28cc1f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.220+0000 7f28d13e9640 1 -- 172.21.15.76:0/2784567483 >> 172.21.15.76:0/2784567483 conn(0x7f28cc0cf740 msgr2=0x7f28cc0af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.220+0000 7f28d13e9640 1 -- 172.21.15.76:0/2784567483 shutdown_connections 2024-03-08T06:00:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:14.220+0000 7f28d13e9640 1 -- 172.21.15.76:0/2784567483 wait complete. 2024-03-08T06:00:14.230 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-08T06:00:14.230 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-08T06:00:14.244 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:14.243+0000 7f0db4c47040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T06:00:14.272 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:14.271+0000 7fe544e46040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T06:00:14.338 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:14.338+0000 7f0db4c47040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T06:00:14.426 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:14.426+0000 7fe544e46040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T06:00:14.469 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:14.468+0000 7fe544e46040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T06:00:14.493 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:14.493+0000 7f0db4c47040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T06:00:14.535 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:14.535+0000 7f0db4c47040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T06:00:14.613 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-03-08T06:00:14.613+0000 7f0f664d9640 -1 Falling back to public interface 2024-03-08T06:00:14.615 INFO:tasks.ceph.osd.4.smithi081.stderr:2024-03-08T06:00:14.614+0000 7f6244419640 -1 Falling back to public interface 2024-03-08T06:00:14.618 INFO:tasks.ceph.osd.5.smithi081.stderr:2024-03-08T06:00:14.618+0000 7f0899cae640 -1 Falling back to public interface 2024-03-08T06:00:14.621 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:14.621+0000 7fe544e46040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T06:00:14.623 INFO:tasks.ceph.osd.7.smithi081.stderr:2024-03-08T06:00:14.623+0000 7f2126147640 -1 Falling back to public interface 2024-03-08T06:00:14.625 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-03-08T06:00:14.624+0000 7f0653214640 -1 Falling back to public interface 2024-03-08T06:00:14.625 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-03-08T06:00:14.625+0000 7fbe3ba13640 -1 Falling back to public interface 2024-03-08T06:00:14.629 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-03-08T06:00:14.629+0000 7f0d3980e640 -1 Falling back to public interface 2024-03-08T06:00:14.637 INFO:tasks.ceph.osd.6.smithi081.stderr:2024-03-08T06:00:14.637+0000 7f460601f640 -1 Falling back to public interface 2024-03-08T06:00:14.689 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:14.689+0000 7f0db4c47040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T06:00:15.035 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-08T06:00:15.125 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.124+0000 7f61e801c640 1 Processor -- start 2024-03-08T06:00:15.125 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.124+0000 7f61e801c640 1 -- start start 2024-03-08T06:00:15.126 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.125+0000 7f61e801c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f61e00cef70 0x7f61e0161970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:15.126 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.125+0000 7f61e801c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f61e00b9500 0x7f61e00d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:15.126 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.125+0000 7f61e801c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e0162000 con 0x7f61e015dda0 2024-03-08T06:00:15.126 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.125+0000 7f61e801c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e01628a0 con 0x7f61e00cef70 2024-03-08T06:00:15.126 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.125+0000 7f61e801c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e01622e0 con 0x7f61e00b9500 2024-03-08T06:00:15.127 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.125+0000 7f61e55b7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f61e00cef70 0x7f61e0161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:15.127 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.125+0000 7f61e55b7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f61e00cef70 0x7f61e0161970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37714/0 (socket says 172.21.15.76:37714) 2024-03-08T06:00:15.127 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.125+0000 7f61e55b7640 1 -- 172.21.15.76:0/3437050715 learned_addr learned my addr 172.21.15.76:0/3437050715 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:15.127 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.125+0000 7f61e65b9640 1 --2- 172.21.15.76:0/3437050715 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f61e00b9500 0x7f61e00d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:15.127 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.126+0000 7f61e55b7640 1 -- 172.21.15.76:0/3437050715 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f61e00b9500 msgr2=0x7f61e00d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:15.127 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.126+0000 7f61e55b7640 1 --2- 172.21.15.76:0/3437050715 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f61e00b9500 0x7f61e00d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:15.127 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.126+0000 7f61e55b7640 1 -- 172.21.15.76:0/3437050715 >> v1:172.21.15.81:6789/0 conn(0x7f61e015dda0 legacy=0x7f61e00ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:15.127 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.126+0000 7f61e55b7640 1 -- 172.21.15.76:0/3437050715 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e01625c0 con 0x7f61e00cef70 2024-03-08T06:00:15.127 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.126+0000 7f61e65b9640 1 --2- 172.21.15.76:0/3437050715 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f61e00b9500 0x7f61e00d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:15.128 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.126+0000 7f61e55b7640 1 --2- 172.21.15.76:0/3437050715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61e00cef70 0x7f61e0161970 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009870 tx=0x7f61dc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe59cfa97b5ef0e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:15.128 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.126+0000 7f61e4db6640 1 -- 172.21.15.76:0/3437050715 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0317c0 con 0x7f61e00cef70 2024-03-08T06:00:15.128 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.126+0000 7f61e4db6640 1 -- 172.21.15.76:0/3437050715 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc037070 con 0x7f61e00cef70 2024-03-08T06:00:15.129 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.126+0000 7f61e4db6640 1 -- 172.21.15.76:0/3437050715 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc031b80 con 0x7f61e00cef70 2024-03-08T06:00:15.129 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.127+0000 7f61e801c640 1 -- 172.21.15.76:0/3437050715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61e00cef70 msgr2=0x7f61e0161970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:15.129 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.127+0000 7f61e801c640 1 --2- 172.21.15.76:0/3437050715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61e00cef70 0x7f61e0161970 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009870 tx=0x7f61dc02b860 comp rx=0 tx=0).stop 2024-03-08T06:00:15.129 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.127+0000 7f61e801c640 1 -- 172.21.15.76:0/3437050715 shutdown_connections 2024-03-08T06:00:15.129 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.127+0000 7f61e801c640 1 --2- 172.21.15.76:0/3437050715 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f61e00b9500 0x7f61e00d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:15.129 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.127+0000 7f61e801c640 1 --2- 172.21.15.76:0/3437050715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61e00cef70 0x7f61e0161970 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:15.129 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.127+0000 7f61e801c640 1 -- 172.21.15.76:0/3437050715 >> 172.21.15.76:0/3437050715 conn(0x7f61e006c560 msgr2=0x7f61e006c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:15.129 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.127+0000 7f61e801c640 1 -- 172.21.15.76:0/3437050715 shutdown_connections 2024-03-08T06:00:15.129 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.127+0000 7f61e801c640 1 -- 172.21.15.76:0/3437050715 wait complete. 2024-03-08T06:00:15.129 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.128+0000 7f61e801c640 1 Processor -- start 2024-03-08T06:00:15.129 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.128+0000 7f61e801c640 1 -- start start 2024-03-08T06:00:15.131 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.129+0000 7f61e801c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61e00b9500 0x7f61e00c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:15.131 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.129+0000 7f61e801c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61e00cef70 0x7f61e00cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:15.131 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.129+0000 7f61e801c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f61e015dda0 0x7f61e014e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:15.131 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.129+0000 7f61e801c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e015e180 con 0x7f61e00b9500 2024-03-08T06:00:15.132 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.129+0000 7f61e801c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e00b9990 con 0x7f61e00cef70 2024-03-08T06:00:15.132 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.129+0000 7f61e801c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e00d2a60 con 0x7f61e015dda0 2024-03-08T06:00:15.132 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.129+0000 7f61e5db8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61e00b9500 0x7f61e00c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:15.132 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.129+0000 7f61e5db8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61e00b9500 0x7f61e00c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37730/0 (socket says 172.21.15.76:37730) 2024-03-08T06:00:15.132 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.129+0000 7f61e5db8640 1 -- 172.21.15.76:0/1174029972 learned_addr learned my addr 172.21.15.76:0/1174029972 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:15.132 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.129+0000 7f61e65b9640 1 --2- 172.21.15.76:0/1174029972 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f61e015dda0 0x7f61e014e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:15.132 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.129+0000 7f61e55b7640 1 --2- 172.21.15.76:0/1174029972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61e00cef70 0x7f61e00cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:15.132 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.129+0000 7f61e5db8640 1 -- 172.21.15.76:0/1174029972 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f61e015dda0 msgr2=0x7f61e014e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:15.132 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.129+0000 7f61e5db8640 1 --2- 172.21.15.76:0/1174029972 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f61e015dda0 0x7f61e014e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:15.132 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.129+0000 7f61e5db8640 1 -- 172.21.15.76:0/1174029972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61e00cef70 msgr2=0x7f61e00cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:15.132 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.129+0000 7f61e5db8640 1 --2- 172.21.15.76:0/1174029972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61e00cef70 0x7f61e00cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:15.132 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.129+0000 7f61e5db8640 1 -- 172.21.15.76:0/1174029972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e00ce390 con 0x7f61e00b9500 2024-03-08T06:00:15.132 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.130+0000 7f61e5db8640 1 --2- 172.21.15.76:0/1174029972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61e00b9500 0x7f61e00c7f50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f61d000b3f0 tx=0x7f61d000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:15.132 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.130+0000 7f61d67fc640 1 -- 172.21.15.76:0/1174029972 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0015020 con 0x7f61e00b9500 2024-03-08T06:00:15.132 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.130+0000 7f61d67fc640 1 -- 172.21.15.76:0/1174029972 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d0011830 con 0x7f61e00b9500 2024-03-08T06:00:15.133 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.130+0000 7f61e801c640 1 -- 172.21.15.76:0/1174029972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e00ce670 con 0x7f61e00b9500 2024-03-08T06:00:15.133 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.130+0000 7f61d67fc640 1 -- 172.21.15.76:0/1174029972 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0011c20 con 0x7f61e00b9500 2024-03-08T06:00:15.133 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.130+0000 7f61e801c640 1 -- 172.21.15.76:0/1174029972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e00c8490 con 0x7f61e00b9500 2024-03-08T06:00:15.133 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.130+0000 7f61d67fc640 1 -- 172.21.15.76:0/1174029972 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f61d0018020 con 0x7f61e00b9500 2024-03-08T06:00:15.133 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.130+0000 7f61d67fc640 1 -- 172.21.15.76:0/1174029972 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f61d001ab10 con 0x7f61e00b9500 2024-03-08T06:00:15.133 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.131+0000 7f61e801c640 1 -- 172.21.15.76:0/1174029972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e00b8b80 con 0x7f61e00b9500 2024-03-08T06:00:15.133 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.133+0000 7f61d67fc640 1 -- 172.21.15.76:0/1174029972 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f61e00b8b80 con 0x7f61e00b9500 2024-03-08T06:00:15.196 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.196+0000 7f61e801c640 1 -- 172.21.15.76:0/1174029972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61e004efc0 con 0x7f61e00b9500 2024-03-08T06:00:15.210 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.197+0000 7f61d67fc640 1 -- 172.21.15.76:0/1174029972 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f61d001f030 con 0x7f61e00b9500 2024-03-08T06:00:15.210 INFO:teuthology.misc.health.smithi076.stdout: 2024-03-08T06:00:15.210 INFO:teuthology.misc.health.smithi076.stdout:{"epoch":10,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:13.990205+0000","last_up_change":"0.000000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:00:15.211 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.198+0000 7f61e801c640 1 -- 172.21.15.76:0/1174029972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61e00b9500 msgr2=0x7f61e00c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:15.212 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.198+0000 7f61e801c640 1 --2- 172.21.15.76:0/1174029972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61e00b9500 0x7f61e00c7f50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f61d000b3f0 tx=0x7f61d000b8b0 comp rx=0 tx=0).stop 2024-03-08T06:00:15.212 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.198+0000 7f61e801c640 1 -- 172.21.15.76:0/1174029972 shutdown_connections 2024-03-08T06:00:15.212 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.198+0000 7f61e801c640 1 --2- 172.21.15.76:0/1174029972 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f61e015dda0 0x7f61e014e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:15.212 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.198+0000 7f61e801c640 1 --2- 172.21.15.76:0/1174029972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61e00cef70 0x7f61e00cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:15.212 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.198+0000 7f61e801c640 1 --2- 172.21.15.76:0/1174029972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61e00b9500 0x7f61e00c7f50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:15.212 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.198+0000 7f61e801c640 1 -- 172.21.15.76:0/1174029972 >> 172.21.15.76:0/1174029972 conn(0x7f61e006c560 msgr2=0x7f61e00b8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:15.212 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.198+0000 7f61e801c640 1 -- 172.21.15.76:0/1174029972 shutdown_connections 2024-03-08T06:00:15.212 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:15.198+0000 7f61e801c640 1 -- 172.21.15.76:0/1174029972 wait complete. 2024-03-08T06:00:15.212 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-08T06:00:15.459 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:15.459+0000 7fe544e46040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T06:00:15.511 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:15.510+0000 7f0db4c47040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T06:00:15.733 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:15.732+0000 7fe544e46040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T06:00:15.785 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:15.784+0000 7fe544e46040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T06:00:15.786 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:15.785+0000 7f0db4c47040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T06:00:15.836 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:15.836+0000 7f0db4c47040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T06:00:15.898 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:15.898+0000 7fe544e46040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T06:00:15.954 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-08T06:00:15.953+0000 7f0db4c47040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T06:00:16.924 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:16.923+0000 7fe4e5ac2640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-08T06:00:16.924 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-03-08T06:00:16.923+0000 7fe4e5ac2640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-08T06:00:18.063 INFO:tasks.ceph.osd.4.smithi081.stderr:2024-03-08T06:00:18.063+0000 7f6244419640 -1 osd.4 0 log_to_monitors true 2024-03-08T06:00:18.084 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-03-08T06:00:18.084+0000 7f0f664d9640 -1 osd.1 0 log_to_monitors true 2024-03-08T06:00:18.114 INFO:tasks.ceph.osd.5.smithi081.stderr:2024-03-08T06:00:18.113+0000 7f0899cae640 -1 osd.5 0 log_to_monitors true 2024-03-08T06:00:18.144 INFO:tasks.ceph.osd.7.smithi081.stderr:2024-03-08T06:00:18.143+0000 7f2126147640 -1 osd.7 0 log_to_monitors true 2024-03-08T06:00:18.149 INFO:tasks.ceph.osd.6.smithi081.stderr:2024-03-08T06:00:18.148+0000 7f460601f640 -1 osd.6 0 log_to_monitors true 2024-03-08T06:00:18.158 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-03-08T06:00:18.158+0000 7f0653214640 -1 osd.3 0 log_to_monitors true 2024-03-08T06:00:18.160 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-03-08T06:00:18.160+0000 7f0d3980e640 -1 osd.2 0 log_to_monitors true 2024-03-08T06:00:18.163 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-03-08T06:00:18.162+0000 7fbe3ba13640 -1 osd.0 0 log_to_monitors true 2024-03-08T06:00:22.020 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-08T06:00:22.023 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-03-08T06:00:22.022+0000 7f0f6246f640 -1 osd.1 0 waiting for initial osdmap 2024-03-08T06:00:22.039 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-03-08T06:00:22.039+0000 7f0f5a28f640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T06:00:22.048 INFO:tasks.ceph.osd.5.smithi081.stderr:2024-03-08T06:00:22.046+0000 7f0896458640 -1 osd.5 0 waiting for initial osdmap 2024-03-08T06:00:22.055 INFO:tasks.ceph.osd.6.smithi081.stderr:2024-03-08T06:00:22.055+0000 7f4601a88640 -1 osd.6 0 waiting for initial osdmap 2024-03-08T06:00:22.062 INFO:tasks.ceph.osd.5.smithi081.stderr:2024-03-08T06:00:22.062+0000 7f088da64640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T06:00:22.067 INFO:tasks.ceph.osd.7.smithi081.stderr:2024-03-08T06:00:22.066+0000 7f21220dd640 -1 osd.7 0 waiting for initial osdmap 2024-03-08T06:00:22.067 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-03-08T06:00:22.067+0000 7fbe3737d640 -1 osd.0 0 waiting for initial osdmap 2024-03-08T06:00:22.069 INFO:tasks.ceph.osd.6.smithi081.stderr:2024-03-08T06:00:22.069+0000 7f45f98a8640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T06:00:22.076 INFO:tasks.ceph.osd.7.smithi081.stderr:2024-03-08T06:00:22.076+0000 7f2119efd640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T06:00:22.076 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-03-08T06:00:22.076+0000 7f0d350d4640 -1 osd.2 0 waiting for initial osdmap 2024-03-08T06:00:22.078 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-03-08T06:00:22.078+0000 7f064eb61640 -1 osd.3 0 waiting for initial osdmap 2024-03-08T06:00:22.084 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-03-08T06:00:22.084+0000 7fbe2f19d640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T06:00:22.094 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-03-08T06:00:22.094+0000 7f0d2cef4640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T06:00:22.094 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-03-08T06:00:22.094+0000 7f0646981640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T06:00:22.116 INFO:tasks.ceph.osd.4.smithi081.stderr:2024-03-08T06:00:22.116+0000 7f623fe7d640 -1 osd.4 0 waiting for initial osdmap 2024-03-08T06:00:22.130 INFO:tasks.ceph.osd.4.smithi081.stderr:2024-03-08T06:00:22.130+0000 7f6237c9d640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T06:00:22.135 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.134+0000 7f9dbeb93640 1 Processor -- start 2024-03-08T06:00:22.135 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.134+0000 7f9dbeb93640 1 -- start start 2024-03-08T06:00:22.136 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.135+0000 7f9dbeb93640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9db815dd30 0x7f9db80ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:22.136 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.135+0000 7f9dbeb93640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9db80cbfb0 0x7f9db80cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:22.136 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.135+0000 7f9dbeb93640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db80b0190 con 0x7f9db806e670 2024-03-08T06:00:22.137 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.135+0000 7f9dbeb93640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db81621d0 con 0x7f9db815dd30 2024-03-08T06:00:22.137 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.135+0000 7f9dbeb93640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db81626e0 con 0x7f9db80cbfb0 2024-03-08T06:00:22.137 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.135+0000 7f9dbc92f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9db815dd30 0x7f9db80ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:22.137 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.135+0000 7f9dbc92f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9db815dd30 0x7f9db80ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53896/0 (socket says 172.21.15.76:53896) 2024-03-08T06:00:22.137 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.135+0000 7f9dbc92f640 1 -- 172.21.15.76:0/1591354945 learned_addr learned my addr 172.21.15.76:0/1591354945 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:22.137 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.136+0000 7f9dbc92f640 1 -- 172.21.15.76:0/1591354945 >> v2:172.21.15.81:3300/0 conn(0x7f9db80cbfb0 msgr2=0x7f9db80cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:22.137 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.136+0000 7f9dbc92f640 1 --2- 172.21.15.76:0/1591354945 >> v2:172.21.15.81:3300/0 conn(0x7f9db80cbfb0 0x7f9db80cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:22.137 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.136+0000 7f9dbc92f640 1 -- 172.21.15.76:0/1591354945 >> v1:172.21.15.81:6789/0 conn(0x7f9db806e670 legacy=0x7f9db80b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:22.137 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.136+0000 7f9dbc92f640 1 -- 172.21.15.76:0/1591354945 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db8162400 con 0x7f9db815dd30 2024-03-08T06:00:22.139 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.136+0000 7f9dbc92f640 1 --2- 172.21.15.76:0/1591354945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db815dd30 0x7f9db80ccf90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9db8071020 tx=0x7f9db002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0fe1b3db192b34 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:22.139 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.136+0000 7f9db77fe640 1 -- 172.21.15.76:0/1591354945 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db002ca80 con 0x7f9db815dd30 2024-03-08T06:00:22.139 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.137+0000 7f9db77fe640 1 -- 172.21.15.76:0/1591354945 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db002cc20 con 0x7f9db815dd30 2024-03-08T06:00:22.139 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.137+0000 7f9db77fe640 1 -- 172.21.15.76:0/1591354945 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db00343f0 con 0x7f9db815dd30 2024-03-08T06:00:22.139 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.137+0000 7f9dbeb93640 1 -- 172.21.15.76:0/1591354945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db815dd30 msgr2=0x7f9db80ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:22.139 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.137+0000 7f9dbeb93640 1 --2- 172.21.15.76:0/1591354945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db815dd30 0x7f9db80ccf90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9db8071020 tx=0x7f9db002bee0 comp rx=0 tx=0).stop 2024-03-08T06:00:22.140 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.137+0000 7f9dbeb93640 1 -- 172.21.15.76:0/1591354945 shutdown_connections 2024-03-08T06:00:22.140 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.137+0000 7f9dbeb93640 1 --2- 172.21.15.76:0/1591354945 >> v2:172.21.15.81:3300/0 conn(0x7f9db80cbfb0 0x7f9db80cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:22.140 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.137+0000 7f9dbeb93640 1 --2- 172.21.15.76:0/1591354945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db815dd30 0x7f9db80ccf90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:22.140 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.137+0000 7f9dbeb93640 1 -- 172.21.15.76:0/1591354945 >> 172.21.15.76:0/1591354945 conn(0x7f9db8099a20 msgr2=0x7f9db8099e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:22.140 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.137+0000 7f9dbeb93640 1 -- 172.21.15.76:0/1591354945 shutdown_connections 2024-03-08T06:00:22.140 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.137+0000 7f9dbeb93640 1 -- 172.21.15.76:0/1591354945 wait complete. 2024-03-08T06:00:22.140 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.138+0000 7f9dbeb93640 1 Processor -- start 2024-03-08T06:00:22.140 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.138+0000 7f9dbeb93640 1 -- start start 2024-03-08T06:00:22.143 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.139+0000 7f9dbeb93640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db806e670 0x7f9db80c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:22.143 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.139+0000 7f9dbeb93640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db80cbfb0 0x7f9db81f7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:22.143 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.139+0000 7f9dbeb93640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db815dd30 0x7f9db81f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:22.143 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.139+0000 7f9dbeb93640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db80705b0 con 0x7f9db815dd30 2024-03-08T06:00:22.143 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.139+0000 7f9dbeb93640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db80702d0 con 0x7f9db806e670 2024-03-08T06:00:22.143 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.139+0000 7f9dbeb93640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db806b470 con 0x7f9db80cbfb0 2024-03-08T06:00:22.144 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.139+0000 7f9dbd130640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db815dd30 0x7f9db81f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:22.144 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.139+0000 7f9dbd130640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db815dd30 0x7f9db81f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53900/0 (socket says 172.21.15.76:53900) 2024-03-08T06:00:22.144 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.139+0000 7f9dbd130640 1 -- 172.21.15.76:0/928086040 learned_addr learned my addr 172.21.15.76:0/928086040 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:22.144 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.139+0000 7f9dbc92f640 1 --2- 172.21.15.76:0/928086040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db806e670 0x7f9db80c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:22.144 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.140+0000 7f9db7fff640 1 --2- 172.21.15.76:0/928086040 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db80cbfb0 0x7f9db81f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:22.144 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.140+0000 7f9dbd130640 1 -- 172.21.15.76:0/928086040 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db80cbfb0 msgr2=0x7f9db81f7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:22.144 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.140+0000 7f9dbd130640 1 --2- 172.21.15.76:0/928086040 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db80cbfb0 0x7f9db81f7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:22.144 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.140+0000 7f9dbd130640 1 -- 172.21.15.76:0/928086040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db806e670 msgr2=0x7f9db80c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:22.144 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.140+0000 7f9dbd130640 1 --2- 172.21.15.76:0/928086040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db806e670 0x7f9db80c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:22.144 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.140+0000 7f9dbd130640 1 -- 172.21.15.76:0/928086040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db815ebc0 con 0x7f9db815dd30 2024-03-08T06:00:22.144 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.140+0000 7f9db7fff640 1 --2- 172.21.15.76:0/928086040 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db80cbfb0 0x7f9db81f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:22.144 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.140+0000 7f9dbd130640 1 --2- 172.21.15.76:0/928086040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db815dd30 0x7f9db81f7680 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009fd0 tx=0x7f9da800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:22.145 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.140+0000 7f9db57fa640 1 -- 172.21.15.76:0/928086040 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8011830 con 0x7f9db815dd30 2024-03-08T06:00:22.145 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.140+0000 7f9db57fa640 1 -- 172.21.15.76:0/928086040 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da8016070 con 0x7f9db815dd30 2024-03-08T06:00:22.145 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.140+0000 7f9db57fa640 1 -- 172.21.15.76:0/928086040 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8011bf0 con 0x7f9db815dd30 2024-03-08T06:00:22.145 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.140+0000 7f9dbeb93640 1 -- 172.21.15.76:0/928086040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db81f7d10 con 0x7f9db815dd30 2024-03-08T06:00:22.145 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.140+0000 7f9dbeb93640 1 -- 172.21.15.76:0/928086040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db815f660 con 0x7f9db815dd30 2024-03-08T06:00:22.145 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.141+0000 7f9dbeb93640 1 -- 172.21.15.76:0/928086040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db80af9e0 con 0x7f9db815dd30 2024-03-08T06:00:22.145 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.141+0000 7f9db57fa640 1 -- 172.21.15.76:0/928086040 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f9da801b460 con 0x7f9db815dd30 2024-03-08T06:00:22.145 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.142+0000 7f9db57fa640 1 --2- 172.21.15.76:0/928086040 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9d9806b440 0x7f9d9806d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:22.145 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.142+0000 7f9dbc92f640 1 --2- 172.21.15.76:0/928086040 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9d9806b440 0x7f9d9806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:22.145 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.142+0000 7f9db57fa640 1 -- 172.21.15.76:0/928086040 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f9da8089020 con 0x7f9db815dd30 2024-03-08T06:00:22.145 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.142+0000 7f9dbc92f640 1 --2- 172.21.15.76:0/928086040 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9d9806b440 0x7f9d9806d900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9db0035000 tx=0x7f9db0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:22.147 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.146+0000 7f9db57fa640 1 -- 172.21.15.76:0/928086040 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9db80af9e0 con 0x7f9db815dd30 2024-03-08T06:00:22.317 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.317+0000 7f9dbeb93640 1 -- 172.21.15.76:0/928086040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9db80d0a70 con 0x7f9db815dd30 2024-03-08T06:00:22.318 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.317+0000 7f9db57fa640 1 -- 172.21.15.76:0/928086040 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f9da8058da0 con 0x7f9db815dd30 2024-03-08T06:00:22.318 INFO:teuthology.misc.health.smithi076.stdout: 2024-03-08T06:00:22.318 INFO:teuthology.misc.health.smithi076.stdout:{"epoch":12,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:19.926924+0000","last_up_change":"0.000000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:00:22.320 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.319+0000 7f9dbeb93640 1 -- 172.21.15.76:0/928086040 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9d9806b440 msgr2=0x7f9d9806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:22.320 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.319+0000 7f9dbeb93640 1 --2- 172.21.15.76:0/928086040 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9d9806b440 0x7f9d9806d900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9db0035000 tx=0x7f9db0030000 comp rx=0 tx=0).stop 2024-03-08T06:00:22.320 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.319+0000 7f9dbeb93640 1 -- 172.21.15.76:0/928086040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db815dd30 msgr2=0x7f9db81f7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:22.320 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.319+0000 7f9dbeb93640 1 --2- 172.21.15.76:0/928086040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db815dd30 0x7f9db81f7680 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009fd0 tx=0x7f9da800b8e0 comp rx=0 tx=0).stop 2024-03-08T06:00:22.321 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.319+0000 7f9dbeb93640 1 -- 172.21.15.76:0/928086040 shutdown_connections 2024-03-08T06:00:22.321 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.319+0000 7f9dbeb93640 1 --2- 172.21.15.76:0/928086040 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9d9806b440 0x7f9d9806d900 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:22.321 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.319+0000 7f9dbeb93640 1 --2- 172.21.15.76:0/928086040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db815dd30 0x7f9db81f7680 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:22.321 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.319+0000 7f9dbeb93640 1 --2- 172.21.15.76:0/928086040 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db80cbfb0 0x7f9db81f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:22.321 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.319+0000 7f9dbeb93640 1 --2- 172.21.15.76:0/928086040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db806e670 0x7f9db80c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:22.321 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.319+0000 7f9dbeb93640 1 -- 172.21.15.76:0/928086040 >> 172.21.15.76:0/928086040 conn(0x7f9db8099a20 msgr2=0x7f9db80ade40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:22.321 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.319+0000 7f9dbeb93640 1 -- 172.21.15.76:0/928086040 shutdown_connections 2024-03-08T06:00:22.321 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:22.319+0000 7f9dbeb93640 1 -- 172.21.15.76:0/928086040 wait complete. 2024-03-08T06:00:22.329 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-08T06:00:29.137 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-08T06:00:29.237 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.236+0000 7f09da781640 1 Processor -- start 2024-03-08T06:00:29.237 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.236+0000 7f09da781640 1 -- start start 2024-03-08T06:00:29.238 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.237+0000 7f09da781640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f09d41442a0 0x7f09d406e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.238 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.237+0000 7f09da781640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f09d4072120 0x7f09d40b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.238 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.237+0000 7f09da781640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f09d40ccd80 0x7f09d40cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.238 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.237+0000 7f09da781640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d40b4e80 con 0x7f09d41442a0 2024-03-08T06:00:29.238 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.237+0000 7f09da781640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d40cd710 con 0x7f09d4072120 2024-03-08T06:00:29.238 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.237+0000 7f09da781640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d406eff0 con 0x7f09d40ccd80 2024-03-08T06:00:29.238 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.237+0000 7f09d3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f09d41442a0 0x7f09d406e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.239 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.237+0000 7f09d3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f09d41442a0 0x7f09d406e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49262/0 (socket says 172.21.15.76:49262) 2024-03-08T06:00:29.239 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.237+0000 7f09d3fff640 1 -- 172.21.15.76:0/2987598422 learned_addr learned my addr 172.21.15.76:0/2987598422 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:29.239 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.237+0000 7f09d8d1e640 1 --2- 172.21.15.76:0/2987598422 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f09d40ccd80 0x7f09d40cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.239 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.237+0000 7f09d37fe640 1 --2- 172.21.15.76:0/2987598422 >> v2:172.21.15.81:3300/0 conn(0x7f09d4072120 0x7f09d40b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.239 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.238+0000 7f09d3fff640 1 -- 172.21.15.76:0/2987598422 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f09d40ccd80 msgr2=0x7f09d40cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:29.239 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.238+0000 7f09d3fff640 1 --2- 172.21.15.76:0/2987598422 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f09d40ccd80 0x7f09d40cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.239 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.238+0000 7f09d3fff640 1 -- 172.21.15.76:0/2987598422 >> v2:172.21.15.81:3300/0 conn(0x7f09d4072120 msgr2=0x7f09d40b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:29.239 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.238+0000 7f09d3fff640 1 --2- 172.21.15.76:0/2987598422 >> v2:172.21.15.81:3300/0 conn(0x7f09d4072120 0x7f09d40b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.239 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.238+0000 7f09d3fff640 1 -- 172.21.15.76:0/2987598422 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d4144680 con 0x7f09d41442a0 2024-03-08T06:00:29.240 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.238+0000 7f09d3fff640 1 --2- 172.21.15.76:0/2987598422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09d41442a0 0x7f09d406e680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f09d40b26c0 tx=0x7f09c402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54a29e41ae3332d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:29.240 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.239+0000 7f09d2ffd640 1 -- 172.21.15.76:0/2987598422 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c402e070 con 0x7f09d41442a0 2024-03-08T06:00:29.241 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.239+0000 7f09d2ffd640 1 -- 172.21.15.76:0/2987598422 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c4037070 con 0x7f09d41442a0 2024-03-08T06:00:29.241 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.239+0000 7f09d2ffd640 1 -- 172.21.15.76:0/2987598422 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4031a00 con 0x7f09d41442a0 2024-03-08T06:00:29.241 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.239+0000 7f09da781640 1 -- 172.21.15.76:0/2987598422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09d41442a0 msgr2=0x7f09d406e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:29.241 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.239+0000 7f09da781640 1 --2- 172.21.15.76:0/2987598422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09d41442a0 0x7f09d406e680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f09d40b26c0 tx=0x7f09c402b880 comp rx=0 tx=0).stop 2024-03-08T06:00:29.241 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.239+0000 7f09da781640 1 -- 172.21.15.76:0/2987598422 shutdown_connections 2024-03-08T06:00:29.241 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.239+0000 7f09da781640 1 --2- 172.21.15.76:0/2987598422 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f09d40ccd80 0x7f09d40cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.241 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.239+0000 7f09da781640 1 --2- 172.21.15.76:0/2987598422 >> v2:172.21.15.81:3300/0 conn(0x7f09d4072120 0x7f09d40b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.241 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.239+0000 7f09da781640 1 --2- 172.21.15.76:0/2987598422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09d41442a0 0x7f09d406e680 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.241 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.239+0000 7f09da781640 1 -- 172.21.15.76:0/2987598422 >> 172.21.15.76:0/2987598422 conn(0x7f09d40b1620 msgr2=0x7f09d40b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:29.241 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.239+0000 7f09da781640 1 -- 172.21.15.76:0/2987598422 shutdown_connections 2024-03-08T06:00:29.241 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.239+0000 7f09da781640 1 -- 172.21.15.76:0/2987598422 wait complete. 2024-03-08T06:00:29.244 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.240+0000 7f09da781640 1 Processor -- start 2024-03-08T06:00:29.244 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.241+0000 7f09da781640 1 -- start start 2024-03-08T06:00:29.244 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.241+0000 7f09da781640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09d4072120 0x7f09d4159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.244 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.241+0000 7f09da781640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f09d40ccd80 0x7f09d413a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.244 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.241+0000 7f09da781640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09d41442a0 0x7f09d40af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.244 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.241+0000 7f09da781640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d4071920 con 0x7f09d41442a0 2024-03-08T06:00:29.244 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.241+0000 7f09da781640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d4071640 con 0x7f09d4072120 2024-03-08T06:00:29.244 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.241+0000 7f09da781640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d4071c00 con 0x7f09d40ccd80 2024-03-08T06:00:29.244 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.242+0000 7f09d8d1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09d41442a0 0x7f09d40af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.244 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.242+0000 7f09d8d1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09d41442a0 0x7f09d40af410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49278/0 (socket says 172.21.15.76:49278) 2024-03-08T06:00:29.244 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.242+0000 7f09d8d1e640 1 -- 172.21.15.76:0/2528488015 learned_addr learned my addr 172.21.15.76:0/2528488015 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:29.244 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.242+0000 7f09d37fe640 1 --2- 172.21.15.76:0/2528488015 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f09d40ccd80 0x7f09d413a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.244 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.242+0000 7f09d3fff640 1 --2- 172.21.15.76:0/2528488015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09d4072120 0x7f09d4159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.244 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.242+0000 7f09d8d1e640 1 -- 172.21.15.76:0/2528488015 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f09d40ccd80 msgr2=0x7f09d413a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:29.245 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.242+0000 7f09d8d1e640 1 --2- 172.21.15.76:0/2528488015 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f09d40ccd80 0x7f09d413a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.245 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.242+0000 7f09d8d1e640 1 -- 172.21.15.76:0/2528488015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09d4072120 msgr2=0x7f09d4159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:29.245 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.242+0000 7f09d8d1e640 1 --2- 172.21.15.76:0/2528488015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09d4072120 0x7f09d4159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.245 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.242+0000 7f09d8d1e640 1 -- 172.21.15.76:0/2528488015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d415a180 con 0x7f09d41442a0 2024-03-08T06:00:29.245 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.242+0000 7f09d8d1e640 1 --2- 172.21.15.76:0/2528488015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09d41442a0 0x7f09d40af410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f09cc007e60 tx=0x7f09cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:29.245 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.242+0000 7f09d0ff9640 1 -- 172.21.15.76:0/2528488015 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc014420 con 0x7f09d41442a0 2024-03-08T06:00:29.245 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.242+0000 7f09d0ff9640 1 -- 172.21.15.76:0/2528488015 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09cc019070 con 0x7f09d41442a0 2024-03-08T06:00:29.245 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.242+0000 7f09da781640 1 -- 172.21.15.76:0/2528488015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d4139390 con 0x7f09d41442a0 2024-03-08T06:00:29.245 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.242+0000 7f09d0ff9640 1 -- 172.21.15.76:0/2528488015 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc014820 con 0x7f09d41442a0 2024-03-08T06:00:29.245 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.242+0000 7f09da781640 1 -- 172.21.15.76:0/2528488015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d4139930 con 0x7f09d41442a0 2024-03-08T06:00:29.245 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.243+0000 7f09da781640 1 -- 172.21.15.76:0/2528488015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d4068120 con 0x7f09d41442a0 2024-03-08T06:00:29.246 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.244+0000 7f09d0ff9640 1 -- 172.21.15.76:0/2528488015 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f09cc0149c0 con 0x7f09d41442a0 2024-03-08T06:00:29.246 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.244+0000 7f09d0ff9640 1 --2- 172.21.15.76:0/2528488015 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f09c006b4d0 0x7f09c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.246 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.244+0000 7f09d3fff640 1 --2- 172.21.15.76:0/2528488015 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f09c006b4d0 0x7f09c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.246 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.244+0000 7f09d0ff9640 1 -- 172.21.15.76:0/2528488015 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f09cc08a480 con 0x7f09d41442a0 2024-03-08T06:00:29.246 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.245+0000 7f09d3fff640 1 --2- 172.21.15.76:0/2528488015 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f09c006b4d0 0x7f09c006d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09c4036000 tx=0x7f09c403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:29.249 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.249+0000 7f09d0ff9640 1 -- 172.21.15.76:0/2528488015 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f09cc059090 con 0x7f09d41442a0 2024-03-08T06:00:29.422 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.422+0000 7f09da781640 1 -- 172.21.15.76:0/2528488015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09d404efc0 con 0x7f09d41442a0 2024-03-08T06:00:29.423 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.423+0000 7f09d0ff9640 1 -- 172.21.15.76:0/2528488015 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12800 (secure 0 0 0) 0x7f09cc05d130 con 0x7f09d41442a0 2024-03-08T06:00:29.423 INFO:teuthology.misc.health.smithi076.stdout: 2024-03-08T06:00:29.423 INFO:teuthology.misc.health.smithi076.stdout:{"epoch":17,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:27.125292+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:00:29.425 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.424+0000 7f09da781640 1 -- 172.21.15.76:0/2528488015 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f09c006b4d0 msgr2=0x7f09c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:29.425 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.424+0000 7f09da781640 1 --2- 172.21.15.76:0/2528488015 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f09c006b4d0 0x7f09c006d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09c4036000 tx=0x7f09c403d040 comp rx=0 tx=0).stop 2024-03-08T06:00:29.426 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.424+0000 7f09da781640 1 -- 172.21.15.76:0/2528488015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09d41442a0 msgr2=0x7f09d40af410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:29.426 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.424+0000 7f09da781640 1 --2- 172.21.15.76:0/2528488015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09d41442a0 0x7f09d40af410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f09cc007e60 tx=0x7f09cc00e4b0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.426 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.425+0000 7f09da781640 1 -- 172.21.15.76:0/2528488015 shutdown_connections 2024-03-08T06:00:29.426 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.425+0000 7f09da781640 1 --2- 172.21.15.76:0/2528488015 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f09c006b4d0 0x7f09c006d990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.426 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.425+0000 7f09da781640 1 --2- 172.21.15.76:0/2528488015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09d41442a0 0x7f09d40af410 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.426 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.425+0000 7f09da781640 1 --2- 172.21.15.76:0/2528488015 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f09d40ccd80 0x7f09d413a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.426 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.425+0000 7f09da781640 1 --2- 172.21.15.76:0/2528488015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09d4072120 0x7f09d4159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.426 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.425+0000 7f09da781640 1 -- 172.21.15.76:0/2528488015 >> 172.21.15.76:0/2528488015 conn(0x7f09d40b1620 msgr2=0x7f09d40b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:29.426 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.425+0000 7f09da781640 1 -- 172.21.15.76:0/2528488015 shutdown_connections 2024-03-08T06:00:29.426 INFO:teuthology.misc.health.smithi076.stderr:2024-03-08T06:00:29.425+0000 7f09da781640 1 -- 172.21.15.76:0/2528488015 wait complete. 2024-03-08T06:00:29.435 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-08T06:00:29.435 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-08T06:00:29.435 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-08T06:00:29.435 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-08T06:00:29.437 INFO:tasks.ceph.mds.a:Started 2024-03-08T06:00:29.437 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-08T06:00:29.437 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-08T06:00:29.439 INFO:tasks.ceph.mds.c:Started 2024-03-08T06:00:29.440 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-08T06:00:29.440 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-08T06:00:29.442 INFO:tasks.ceph.mds.b:Started 2024-03-08T06:00:29.442 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-08T06:00:29.443 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-08T06:00:29.445 INFO:tasks.ceph.mds.d:Started 2024-03-08T06:00:29.445 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-08T06:00:29.447 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:00:29.512 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-03-08T06:00:29.513 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2024-03-08T06:00:29.527 INFO:tasks.ceph.mds.b.smithi081.stdout:starting mds.b at 2024-03-08T06:00:29.527 INFO:tasks.ceph.mds.d.smithi081.stdout:starting mds.d at 2024-03-08T06:00:29.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.551+0000 7f2bb202c640 1 Processor -- start 2024-03-08T06:00:29.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.551+0000 7f2bb202c640 1 -- start start 2024-03-08T06:00:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.552+0000 7f2bb202c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2bac0ce320 0x7f2bac0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.552+0000 7f2bb202c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bac0ced70 con 0x7f2bac0d1960 2024-03-08T06:00:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.552+0000 7f2bb202c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bac0cf050 con 0x7f2bac06bd90 2024-03-08T06:00:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.552+0000 7f2bb202c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bac0cc7b0 con 0x7f2bac0ce320 2024-03-08T06:00:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.552+0000 7f2bab7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2bac0d1960 0x7f2bac0d1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46942/0 (socket says 172.21.15.76:46942) 2024-03-08T06:00:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.552+0000 7f2bab7fe640 1 -- 172.21.15.76:0/497368232 learned_addr learned my addr 172.21.15.76:0/497368232 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.552+0000 7f2baaffd640 1 --2- 172.21.15.76:0/497368232 >> v2:172.21.15.81:3300/0 conn(0x7f2bac0ce320 0x7f2bac0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.552+0000 7f2baa7fc640 1 -- 172.21.15.76:0/497368232 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499398983 0 0) 0x7f2bac0ced70 con 0x7f2bac0d1960 2024-03-08T06:00:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.552+0000 7f2baa7fc640 1 -- 172.21.15.76:0/497368232 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b94003520 con 0x7f2bac0d1960 2024-03-08T06:00:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2baa7fc640 1 -- 172.21.15.76:0/497368232 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 624877595 0 0) 0x7f2b94003520 con 0x7f2bac0d1960 2024-03-08T06:00:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2baa7fc640 1 -- 172.21.15.76:0/497368232 >> v2:172.21.15.81:3300/0 conn(0x7f2bac0ce320 msgr2=0x7f2bac0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2baa7fc640 1 --2- 172.21.15.76:0/497368232 >> v2:172.21.15.81:3300/0 conn(0x7f2bac0ce320 0x7f2bac0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2baa7fc640 1 -- 172.21.15.76:0/497368232 >> v1:172.21.15.81:6789/0 conn(0x7f2bac06bd90 legacy=0x7f2bac0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2baa7fc640 1 -- 172.21.15.76:0/497368232 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bac0cc160 con 0x7f2bac0d1960 2024-03-08T06:00:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2baa7fc640 1 -- 172.21.15.76:0/497368232 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f2b9c001670 con 0x7f2bac0d1960 2024-03-08T06:00:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2baa7fc640 1 -- 172.21.15.76:0/497368232 >> v1:172.21.15.76:6789/0 conn(0x7f2bac0d1960 legacy=0x7f2bac0d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2baa7fc640 1 --2- 172.21.15.76:0/497368232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b94003a20 0x7f2b94003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2baa7fc640 1 --2- 172.21.15.76:0/497368232 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b94004800 0x7f2b94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2baa7fc640 1 --2- 172.21.15.76:0/497368232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b940051c0 0x7f2b9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2baa7fc640 1 -- 172.21.15.76:0/497368232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac0cc7b0 con 0x7f2b94003a20 2024-03-08T06:00:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2baa7fc640 1 -- 172.21.15.76:0/497368232 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9400c0f0 con 0x7f2b940051c0 2024-03-08T06:00:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2baa7fc640 1 -- 172.21.15.76:0/497368232 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9400c2f0 con 0x7f2b94004800 2024-03-08T06:00:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2bab7fe640 1 --2- 172.21.15.76:0/497368232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b94003a20 0x7f2b94003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2baaffd640 1 --2- 172.21.15.76:0/497368232 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b94004800 0x7f2b94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2babfff640 1 --2- 172.21.15.76:0/497368232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b940051c0 0x7f2b9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2bab7fe640 1 -- 172.21.15.76:0/497368232 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b94004800 msgr2=0x7f2b94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2bab7fe640 1 --2- 172.21.15.76:0/497368232 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b94004800 0x7f2b94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2bab7fe640 1 -- 172.21.15.76:0/497368232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b940051c0 msgr2=0x7f2b9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2bab7fe640 1 --2- 172.21.15.76:0/497368232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b940051c0 0x7f2b9400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2babfff640 1 -- 172.21.15.76:0/497368232 reap_dead start 2024-03-08T06:00:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.553+0000 7f2bab7fe640 1 -- 172.21.15.76:0/497368232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b94003520 con 0x7f2b94003a20 2024-03-08T06:00:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.554+0000 7f2bab7fe640 1 --2- 172.21.15.76:0/497368232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b94003a20 0x7f2b94003e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00c240 tx=0x7f2b94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956ef582f9b82ab server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.554+0000 7f2baa7fc640 1 -- 172.21.15.76:0/497368232 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c007d10 con 0x7f2b94003a20 2024-03-08T06:00:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.554+0000 7f2baa7fc640 1 -- 172.21.15.76:0/497368232 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9c030070 con 0x7f2b94003a20 2024-03-08T06:00:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.554+0000 7f2baa7fc640 1 -- 172.21.15.76:0/497368232 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c02cdc0 con 0x7f2b94003a20 2024-03-08T06:00:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.554+0000 7f2bb202c640 1 -- 172.21.15.76:0/497368232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b94003a20 msgr2=0x7f2b94003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.554+0000 7f2bb202c640 1 --2- 172.21.15.76:0/497368232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b94003a20 0x7f2b94003e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00c240 tx=0x7f2b94005830 comp rx=0 tx=0).stop 2024-03-08T06:00:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.554+0000 7f2bb202c640 1 -- 172.21.15.76:0/497368232 shutdown_connections 2024-03-08T06:00:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.554+0000 7f2bb202c640 1 --2- 172.21.15.76:0/497368232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b94003a20 0x7f2b94003e40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.554+0000 7f2bb202c640 1 -- 172.21.15.76:0/497368232 >> 172.21.15.76:0/497368232 conn(0x7f2bac0b34e0 msgr2=0x7f2bac0b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.555+0000 7f2bb202c640 1 -- 172.21.15.76:0/497368232 shutdown_connections 2024-03-08T06:00:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.555+0000 7f2bb202c640 1 -- 172.21.15.76:0/497368232 wait complete. 2024-03-08T06:00:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.556+0000 7f2bb202c640 1 Processor -- start 2024-03-08T06:00:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.556+0000 7f2bb202c640 1 -- start start 2024-03-08T06:00:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2bb202c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b94003a20 0x7f2bac15f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2bb202c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bac0c6c70 0x7f2bac0c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2bb202c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bac1f6800 0x7f2bac1f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2bb202c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac0cdf80 con 0x7f2bac0c6c70 2024-03-08T06:00:29.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2bb202c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac0cde00 con 0x7f2bac1f6800 2024-03-08T06:00:29.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2bb202c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac15dea0 con 0x7f2b94003a20 2024-03-08T06:00:29.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2baaffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bac0c6c70 0x7f2bac0c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2baaffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bac0c6c70 0x7f2bac0c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49344/0 (socket says 172.21.15.76:49344) 2024-03-08T06:00:29.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2baaffd640 1 -- 172.21.15.76:0/1313277500 learned_addr learned my addr 172.21.15.76:0/1313277500 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:29.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2babfff640 1 --2- 172.21.15.76:0/1313277500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bac1f6800 0x7f2bac1f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2bab7fe640 1 --2- 172.21.15.76:0/1313277500 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b94003a20 0x7f2bac15f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2baaffd640 1 -- 172.21.15.76:0/1313277500 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b94003a20 msgr2=0x7f2bac15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:29.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2baaffd640 1 --2- 172.21.15.76:0/1313277500 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b94003a20 0x7f2bac15f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2baaffd640 1 -- 172.21.15.76:0/1313277500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bac1f6800 msgr2=0x7f2bac1f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:29.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2baaffd640 1 --2- 172.21.15.76:0/1313277500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bac1f6800 0x7f2bac1f6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2baaffd640 1 -- 172.21.15.76:0/1313277500 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bac1f71c0 con 0x7f2bac0c6c70 2024-03-08T06:00:29.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2baaffd640 1 --2- 172.21.15.76:0/1313277500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bac0c6c70 0x7f2bac0c70d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0011980 tx=0x7f2ba0011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:29.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2b93fff640 1 -- 172.21.15.76:0/1313277500 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba000faf0 con 0x7f2bac0c6c70 2024-03-08T06:00:29.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2b93fff640 1 -- 172.21.15.76:0/1313277500 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba000fc90 con 0x7f2bac0c6c70 2024-03-08T06:00:29.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2b93fff640 1 -- 172.21.15.76:0/1313277500 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0014440 con 0x7f2bac0c6c70 2024-03-08T06:00:29.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2bb202c640 1 -- 172.21.15.76:0/1313277500 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bac1e7860 con 0x7f2bac0c6c70 2024-03-08T06:00:29.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.557+0000 7f2bb202c640 1 -- 172.21.15.76:0/1313277500 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bac0c7aa0 con 0x7f2bac0c6c70 2024-03-08T06:00:29.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.558+0000 7f2bb202c640 1 -- 172.21.15.76:0/1313277500 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bac04f030 con 0x7f2bac0c6c70 2024-03-08T06:00:29.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.559+0000 7f2b93fff640 1 -- 172.21.15.76:0/1313277500 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ba0012040 con 0x7f2bac0c6c70 2024-03-08T06:00:29.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.559+0000 7f2b93fff640 1 --2- 172.21.15.76:0/1313277500 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2b9806b500 0x7f2b9806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.559+0000 7f2b93fff640 1 -- 172.21.15.76:0/1313277500 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2ba008a340 con 0x7f2bac0c6c70 2024-03-08T06:00:29.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.559+0000 7f2bab7fe640 1 --2- 172.21.15.76:0/1313277500 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2b9806b500 0x7f2b9806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.559+0000 7f2bab7fe640 1 --2- 172.21.15.76:0/1313277500 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2b9806b500 0x7f2b9806d9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c034000 tx=0x7f2b9c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:29.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.563+0000 7f2b93fff640 1 -- 172.21.15.76:0/1313277500 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ba0019030 con 0x7f2bac0c6c70 2024-03-08T06:00:29.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.735+0000 7f2bb202c640 1 -- 172.21.15.76:0/1313277500 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bac0d0550 con 0x7f2bac0c6c70 2024-03-08T06:00:29.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.736+0000 7f2b93fff640 1 -- 172.21.15.76:0/1313277500 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12800 (secure 0 0 0) 0x7f2ba0058ec0 con 0x7f2bac0c6c70 2024-03-08T06:00:29.736 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:29.736 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":17,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:27.125292+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:00:29.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.738+0000 7f2bb202c640 1 -- 172.21.15.76:0/1313277500 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2b9806b500 msgr2=0x7f2b9806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:29.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.738+0000 7f2bb202c640 1 --2- 172.21.15.76:0/1313277500 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2b9806b500 0x7f2b9806d9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c034000 tx=0x7f2b9c009030 comp rx=0 tx=0).stop 2024-03-08T06:00:29.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.738+0000 7f2bb202c640 1 -- 172.21.15.76:0/1313277500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bac0c6c70 msgr2=0x7f2bac0c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:29.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.738+0000 7f2bb202c640 1 --2- 172.21.15.76:0/1313277500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bac0c6c70 0x7f2bac0c70d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0011980 tx=0x7f2ba0011e40 comp rx=0 tx=0).stop 2024-03-08T06:00:29.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.738+0000 7f2bb202c640 1 -- 172.21.15.76:0/1313277500 shutdown_connections 2024-03-08T06:00:29.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.738+0000 7f2bb202c640 1 --2- 172.21.15.76:0/1313277500 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2b9806b500 0x7f2b9806d9c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.738+0000 7f2bb202c640 1 --2- 172.21.15.76:0/1313277500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bac1f6800 0x7f2bac1f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.738+0000 7f2bb202c640 1 --2- 172.21.15.76:0/1313277500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bac0c6c70 0x7f2bac0c70d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.738+0000 7f2bb202c640 1 --2- 172.21.15.76:0/1313277500 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b94003a20 0x7f2bac15f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.738+0000 7f2bb202c640 1 -- 172.21.15.76:0/1313277500 >> 172.21.15.76:0/1313277500 conn(0x7f2bac0b34e0 msgr2=0x7f2bac0b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:29.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.738+0000 7f2bb202c640 1 -- 172.21.15.76:0/1313277500 shutdown_connections 2024-03-08T06:00:29.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.738+0000 7f2bb202c640 1 -- 172.21.15.76:0/1313277500 wait complete. 2024-03-08T06:00:29.749 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:00:29.749 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:00:29.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.852+0000 7f433ec07640 1 Processor -- start 2024-03-08T06:00:29.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.852+0000 7f433ec07640 1 -- start start 2024-03-08T06:00:29.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.853+0000 7f433ec07640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f43380d2150 0x7f43380cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.853+0000 7f433ec07640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f43380d0800 0x7f43380b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.853+0000 7f433ec07640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43380cfa80 con 0x7f43380ce690 2024-03-08T06:00:29.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.853+0000 7f433ec07640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43380cefb0 con 0x7f43380d2150 2024-03-08T06:00:29.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.853+0000 7f433ec07640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43380b1000 con 0x7f43380d0800 2024-03-08T06:00:29.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.853+0000 7f432ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f43380d2150 0x7f43380cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.853+0000 7f433c9a3640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f43380ce690 0x7f43380b0840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46952/0 (socket says 172.21.15.76:46952) 2024-03-08T06:00:29.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.853+0000 7f433c9a3640 1 -- 172.21.15.76:0/3997146720 learned_addr learned my addr 172.21.15.76:0/3997146720 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:29.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.853+0000 7f433d1a4640 1 --2- 172.21.15.76:0/3997146720 >> v2:172.21.15.81:3300/0 conn(0x7f43380d0800 0x7f43380b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.853+0000 7f432ffff640 1 -- 172.21.15.76:0/3997146720 >> v2:172.21.15.81:3300/0 conn(0x7f43380d0800 msgr2=0x7f43380b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:29.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.853+0000 7f432ffff640 1 --2- 172.21.15.76:0/3997146720 >> v2:172.21.15.81:3300/0 conn(0x7f43380d0800 0x7f43380b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.853+0000 7f432ffff640 1 -- 172.21.15.76:0/3997146720 >> v1:172.21.15.76:6789/0 conn(0x7f43380ce690 legacy=0x7f43380b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:29.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.853+0000 7f432ffff640 1 -- 172.21.15.76:0/3997146720 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43380cfd20 con 0x7f43380d2150 2024-03-08T06:00:29.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.853+0000 7f432f7fe640 1 -- 172.21.15.76:0/3997146720 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037776003 0 0) 0x7f43380cfa80 con 0x7f43380ce690 2024-03-08T06:00:29.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.854+0000 7f432ffff640 1 --2- 172.21.15.76:0/3997146720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43380d2150 0x7f43380cea70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f43380cc030 tx=0x7f432802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d34f6c20b953b56 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:29.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.854+0000 7f432f7fe640 1 -- 172.21.15.76:0/3997146720 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328035020 con 0x7f43380d2150 2024-03-08T06:00:29.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.854+0000 7f432f7fe640 1 -- 172.21.15.76:0/3997146720 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4328031830 con 0x7f43380d2150 2024-03-08T06:00:29.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.854+0000 7f432f7fe640 1 -- 172.21.15.76:0/3997146720 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328031c00 con 0x7f43380d2150 2024-03-08T06:00:29.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.855+0000 7f433ec07640 1 -- 172.21.15.76:0/3997146720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43380d2150 msgr2=0x7f43380cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:29.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.855+0000 7f433ec07640 1 --2- 172.21.15.76:0/3997146720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43380d2150 0x7f43380cea70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f43380cc030 tx=0x7f432802b8d0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.855+0000 7f433ec07640 1 -- 172.21.15.76:0/3997146720 shutdown_connections 2024-03-08T06:00:29.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.855+0000 7f433ec07640 1 --2- 172.21.15.76:0/3997146720 >> v2:172.21.15.81:3300/0 conn(0x7f43380d0800 0x7f43380b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.855+0000 7f433ec07640 1 --2- 172.21.15.76:0/3997146720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43380d2150 0x7f43380cea70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.855+0000 7f433ec07640 1 -- 172.21.15.76:0/3997146720 >> 172.21.15.76:0/3997146720 conn(0x7f433806ae60 msgr2=0x7f433806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:29.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.855+0000 7f433ec07640 1 -- 172.21.15.76:0/3997146720 shutdown_connections 2024-03-08T06:00:29.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.855+0000 7f433ec07640 1 -- 172.21.15.76:0/3997146720 wait complete. 2024-03-08T06:00:29.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.856+0000 7f433ec07640 1 Processor -- start 2024-03-08T06:00:29.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.856+0000 7f433ec07640 1 -- start start 2024-03-08T06:00:29.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433ec07640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43380ce690 0x7f43380cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433ec07640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43380d0800 0x7f4338155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433ec07640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43380d2150 0x7f43380c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433ec07640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433815be20 con 0x7f43380d2150 2024-03-08T06:00:29.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433ec07640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43380d02e0 con 0x7f43380ce690 2024-03-08T06:00:29.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433ec07640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433815bfa0 con 0x7f43380d0800 2024-03-08T06:00:29.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433d1a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43380d2150 0x7f43380c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433d1a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43380d2150 0x7f43380c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49368/0 (socket says 172.21.15.76:49368) 2024-03-08T06:00:29.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433d1a4640 1 -- 172.21.15.76:0/293636420 learned_addr learned my addr 172.21.15.76:0/293636420 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:29.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f432ffff640 1 --2- 172.21.15.76:0/293636420 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43380d0800 0x7f4338155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433c9a3640 1 --2- 172.21.15.76:0/293636420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43380ce690 0x7f43380cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433d1a4640 1 -- 172.21.15.76:0/293636420 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43380d0800 msgr2=0x7f4338155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:29.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433d1a4640 1 --2- 172.21.15.76:0/293636420 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43380d0800 0x7f4338155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433d1a4640 1 -- 172.21.15.76:0/293636420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43380ce690 msgr2=0x7f43380cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:29.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433d1a4640 1 --2- 172.21.15.76:0/293636420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43380ce690 0x7f43380cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:29.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433d1a4640 1 -- 172.21.15.76:0/293636420 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43380c5520 con 0x7f43380d2150 2024-03-08T06:00:29.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f432ffff640 1 --2- 172.21.15.76:0/293636420 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43380d0800 0x7f4338155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:29.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433c9a3640 1 --2- 172.21.15.76:0/293636420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43380ce690 0x7f43380cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:29.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433d1a4640 1 --2- 172.21.15.76:0/293636420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43380d2150 0x7f43380c7d10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4330007e40 tx=0x7f433000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:29.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f432d7fa640 1 -- 172.21.15.76:0/293636420 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330014620 con 0x7f43380d2150 2024-03-08T06:00:29.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f432d7fa640 1 -- 172.21.15.76:0/293636420 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4330019070 con 0x7f43380d2150 2024-03-08T06:00:29.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433ec07640 1 -- 172.21.15.76:0/293636420 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43380c5800 con 0x7f43380d2150 2024-03-08T06:00:29.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f432d7fa640 1 -- 172.21.15.76:0/293636420 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330014a20 con 0x7f43380d2150 2024-03-08T06:00:29.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.857+0000 7f433ec07640 1 -- 172.21.15.76:0/293636420 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43380c5dc0 con 0x7f43380d2150 2024-03-08T06:00:29.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.858+0000 7f433ec07640 1 -- 172.21.15.76:0/293636420 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433804f030 con 0x7f43380d2150 2024-03-08T06:00:29.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.859+0000 7f432d7fa640 1 -- 172.21.15.76:0/293636420 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4330014bc0 con 0x7f43380d2150 2024-03-08T06:00:29.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.859+0000 7f432d7fa640 1 --2- 172.21.15.76:0/293636420 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f432406b500 0x7f432406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:29.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.859+0000 7f433c9a3640 1 --2- 172.21.15.76:0/293636420 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f432406b500 0x7f432406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:29.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.859+0000 7f432d7fa640 1 -- 172.21.15.76:0/293636420 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f433008a740 con 0x7f43380d2150 2024-03-08T06:00:29.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.860+0000 7f433c9a3640 1 --2- 172.21.15.76:0/293636420 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f432406b500 0x7f432406d9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4334000f60 tx=0x7f4334009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:29.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:29.864+0000 7f432d7fa640 1 -- 172.21.15.76:0/293636420 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43300599a0 con 0x7f43380d2150 2024-03-08T06:00:30.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.035+0000 7f433ec07640 1 -- 172.21.15.76:0/293636420 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43380c4af0 con 0x7f43380d2150 2024-03-08T06:00:30.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.035+0000 7f432d7fa640 1 -- 172.21.15.76:0/293636420 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43300592a0 con 0x7f43380d2150 2024-03-08T06:00:30.035 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:00:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.037+0000 7f433ec07640 1 -- 172.21.15.76:0/293636420 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f432406b500 msgr2=0x7f432406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.037+0000 7f433ec07640 1 --2- 172.21.15.76:0/293636420 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f432406b500 0x7f432406d9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4334000f60 tx=0x7f4334009040 comp rx=0 tx=0).stop 2024-03-08T06:00:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.037+0000 7f433ec07640 1 -- 172.21.15.76:0/293636420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43380d2150 msgr2=0x7f43380c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.037+0000 7f433ec07640 1 --2- 172.21.15.76:0/293636420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43380d2150 0x7f43380c7d10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4330007e40 tx=0x7f433000e4b0 comp rx=0 tx=0).stop 2024-03-08T06:00:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.037+0000 7f433ec07640 1 -- 172.21.15.76:0/293636420 shutdown_connections 2024-03-08T06:00:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.037+0000 7f433ec07640 1 --2- 172.21.15.76:0/293636420 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f432406b500 0x7f432406d9c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.037+0000 7f433ec07640 1 --2- 172.21.15.76:0/293636420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43380d2150 0x7f43380c7d10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.037+0000 7f433ec07640 1 --2- 172.21.15.76:0/293636420 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43380d0800 0x7f4338155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.037+0000 7f433ec07640 1 --2- 172.21.15.76:0/293636420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43380ce690 0x7f43380cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.037+0000 7f433ec07640 1 -- 172.21.15.76:0/293636420 >> 172.21.15.76:0/293636420 conn(0x7f433806ae60 msgr2=0x7f43380b6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.037+0000 7f433ec07640 1 -- 172.21.15.76:0/293636420 shutdown_connections 2024-03-08T06:00:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.037+0000 7f433ec07640 1 -- 172.21.15.76:0/293636420 wait complete. 2024-03-08T06:00:30.048 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:00:30.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.148+0000 7ff74ed81640 1 Processor -- start 2024-03-08T06:00:30.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.148+0000 7ff74ed81640 1 -- start start 2024-03-08T06:00:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.148+0000 7ff74ed81640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7480cc510 0x7ff7480b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff74ed81640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff74815c1e0 con 0x7ff748159700 2024-03-08T06:00:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff74ed81640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff74815c4c0 con 0x7ff74806b390 2024-03-08T06:00:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff74ed81640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74815c7a0 con 0x7ff7480cc510 2024-03-08T06:00:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff74cb1d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff748159700 0x7ff74806ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46964/0 (socket says 172.21.15.76:46964) 2024-03-08T06:00:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff74cb1d640 1 -- 172.21.15.76:0/2259585503 learned_addr learned my addr 172.21.15.76:0/2259585503 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff7477fe640 1 -- 172.21.15.76:0/2259585503 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550669031 0 0) 0x7ff74815c1e0 con 0x7ff748159700 2024-03-08T06:00:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff7477fe640 1 -- 172.21.15.76:0/2259585503 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff738003520 con 0x7ff748159700 2024-03-08T06:00:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff74d31e640 1 --2- 172.21.15.76:0/2259585503 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7480cc510 0x7ff7480b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff7477fe640 1 -- 172.21.15.76:0/2259585503 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2885768002 0 0) 0x7ff738003520 con 0x7ff748159700 2024-03-08T06:00:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff7477fe640 1 -- 172.21.15.76:0/2259585503 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7480cc510 msgr2=0x7ff7480b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff7477fe640 1 --2- 172.21.15.76:0/2259585503 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7480cc510 0x7ff7480b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff7477fe640 1 -- 172.21.15.76:0/2259585503 >> v1:172.21.15.81:6789/0 conn(0x7ff74806b390 legacy=0x7ff7480b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff7477fe640 1 -- 172.21.15.76:0/2259585503 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74815ca80 con 0x7ff748159700 2024-03-08T06:00:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff7477fe640 1 -- 172.21.15.76:0/2259585503 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7ff73c001670 con 0x7ff748159700 2024-03-08T06:00:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff7477fe640 1 -- 172.21.15.76:0/2259585503 >> v1:172.21.15.76:6789/0 conn(0x7ff748159700 legacy=0x7ff74806ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff7477fe640 1 --2- 172.21.15.76:0/2259585503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff738003a50 0x7ff738003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff7477fe640 1 --2- 172.21.15.76:0/2259585503 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7480cc510 0x7ff738004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff7477fe640 1 --2- 172.21.15.76:0/2259585503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff738004d40 0x7ff7380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff7477fe640 1 -- 172.21.15.76:0/2259585503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74815c7a0 con 0x7ff738004d40 2024-03-08T06:00:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff7477fe640 1 -- 172.21.15.76:0/2259585503 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73800bad0 con 0x7ff738003a50 2024-03-08T06:00:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff7477fe640 1 -- 172.21.15.76:0/2259585503 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73800bcd0 con 0x7ff7480cc510 2024-03-08T06:00:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff74d31e640 1 --2- 172.21.15.76:0/2259585503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff738004d40 0x7ff7380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff74cb1d640 1 --2- 172.21.15.76:0/2259585503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff738003a50 0x7ff738003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff747fff640 1 --2- 172.21.15.76:0/2259585503 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7480cc510 0x7ff738004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff74d31e640 1 -- 172.21.15.76:0/2259585503 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7480cc510 msgr2=0x7ff738004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff74d31e640 1 --2- 172.21.15.76:0/2259585503 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7480cc510 0x7ff738004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff74d31e640 1 -- 172.21.15.76:0/2259585503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff738003a50 msgr2=0x7ff738003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.149+0000 7ff74d31e640 1 --2- 172.21.15.76:0/2259585503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff738003a50 0x7ff738003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.150+0000 7ff74d31e640 1 -- 172.21.15.76:0/2259585503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff738003520 con 0x7ff738004d40 2024-03-08T06:00:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.150+0000 7ff747fff640 1 --2- 172.21.15.76:0/2259585503 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7480cc510 0x7ff738004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:00:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.150+0000 7ff74d31e640 1 --2- 172.21.15.76:0/2259585503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff738004d40 0x7ff7380051c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff740009f90 tx=0x7ff738005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b22be473f6a052 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.150+0000 7ff7477fe640 1 -- 172.21.15.76:0/2259585503 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740039070 con 0x7ff738004d40 2024-03-08T06:00:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.150+0000 7ff7477fe640 1 -- 172.21.15.76:0/2259585503 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74002cbd0 con 0x7ff738004d40 2024-03-08T06:00:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.150+0000 7ff7477fe640 1 -- 172.21.15.76:0/2259585503 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740009c10 con 0x7ff738004d40 2024-03-08T06:00:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.151+0000 7ff74ed81640 1 -- 172.21.15.76:0/2259585503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff738004d40 msgr2=0x7ff7380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.151+0000 7ff74ed81640 1 --2- 172.21.15.76:0/2259585503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff738004d40 0x7ff7380051c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff740009f90 tx=0x7ff738005830 comp rx=0 tx=0).stop 2024-03-08T06:00:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.151+0000 7ff74d31e640 1 -- 172.21.15.76:0/2259585503 reap_dead start 2024-03-08T06:00:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.151+0000 7ff74ed81640 1 -- 172.21.15.76:0/2259585503 shutdown_connections 2024-03-08T06:00:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.151+0000 7ff74ed81640 1 -- 172.21.15.76:0/2259585503 >> 172.21.15.76:0/2259585503 conn(0x7ff748099a60 msgr2=0x7ff7480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.151+0000 7ff74ed81640 1 -- 172.21.15.76:0/2259585503 shutdown_connections 2024-03-08T06:00:30.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.151+0000 7ff74ed81640 1 -- 172.21.15.76:0/2259585503 wait complete. 2024-03-08T06:00:30.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.152+0000 7ff74ed81640 1 Processor -- start 2024-03-08T06:00:30.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.152+0000 7ff74ed81640 1 -- start start 2024-03-08T06:00:30.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff74ed81640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff738004d40 0x7ff748157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:30.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff74ed81640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7480aea00 0x7ff7481577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:30.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff74ed81640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff748157cf0 0x7ff7480c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:30.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff74ed81640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480b2d70 con 0x7ff738004d40 2024-03-08T06:00:30.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff74ed81640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480b21b0 con 0x7ff748157cf0 2024-03-08T06:00:30.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff74ed81640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480b2ef0 con 0x7ff7480aea00 2024-03-08T06:00:30.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff74cb1d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff738004d40 0x7ff748157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:30.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff74cb1d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff738004d40 0x7ff748157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49388/0 (socket says 172.21.15.76:49388) 2024-03-08T06:00:30.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff74cb1d640 1 -- 172.21.15.76:0/1095926576 learned_addr learned my addr 172.21.15.76:0/1095926576 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:30.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff74d31e640 1 --2- 172.21.15.76:0/1095926576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff748157cf0 0x7ff7480c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:30.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff747fff640 1 --2- 172.21.15.76:0/1095926576 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7480aea00 0x7ff7481577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:30.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff74cb1d640 1 -- 172.21.15.76:0/1095926576 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7480aea00 msgr2=0x7ff7481577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:30.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff74cb1d640 1 --2- 172.21.15.76:0/1095926576 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7480aea00 0x7ff7481577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:30.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff74cb1d640 1 -- 172.21.15.76:0/1095926576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff748157cf0 msgr2=0x7ff7480c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:30.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff74cb1d640 1 --2- 172.21.15.76:0/1095926576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff748157cf0 0x7ff7480c6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:30.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff74cb1d640 1 -- 172.21.15.76:0/1095926576 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff748158570 con 0x7ff738004d40 2024-03-08T06:00:30.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff747fff640 1 --2- 172.21.15.76:0/1095926576 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7480aea00 0x7ff7481577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:00:30.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.153+0000 7ff74cb1d640 1 --2- 172.21.15.76:0/1095926576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff738004d40 0x7ff748157170 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0072c0 tx=0x7ff73c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:30.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.154+0000 7ff7457fa640 1 -- 172.21.15.76:0/1095926576 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c01a070 con 0x7ff738004d40 2024-03-08T06:00:30.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.154+0000 7ff7457fa640 1 -- 172.21.15.76:0/1095926576 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c00f960 con 0x7ff738004d40 2024-03-08T06:00:30.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.154+0000 7ff74ed81640 1 -- 172.21.15.76:0/1095926576 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7480ad860 con 0x7ff738004d40 2024-03-08T06:00:30.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.154+0000 7ff7457fa640 1 -- 172.21.15.76:0/1095926576 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c00fd50 con 0x7ff738004d40 2024-03-08T06:00:30.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.154+0000 7ff74ed81640 1 -- 172.21.15.76:0/1095926576 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7480ade20 con 0x7ff738004d40 2024-03-08T06:00:30.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.155+0000 7ff74ed81640 1 -- 172.21.15.76:0/1095926576 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff74804f030 con 0x7ff738004d40 2024-03-08T06:00:30.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.155+0000 7ff7457fa640 1 -- 172.21.15.76:0/1095926576 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff73c01d070 con 0x7ff738004d40 2024-03-08T06:00:30.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.156+0000 7ff7457fa640 1 --2- 172.21.15.76:0/1095926576 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff72c06b4d0 0x7ff72c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:30.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.156+0000 7ff7457fa640 1 -- 172.21.15.76:0/1095926576 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff73c089880 con 0x7ff738004d40 2024-03-08T06:00:30.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.156+0000 7ff747fff640 1 --2- 172.21.15.76:0/1095926576 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff72c06b4d0 0x7ff72c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:30.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.156+0000 7ff747fff640 1 --2- 172.21.15.76:0/1095926576 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff72c06b4d0 0x7ff72c06d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff7481f7150 tx=0x7ff734009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:30.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.160+0000 7ff7457fa640 1 -- 172.21.15.76:0/1095926576 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff73c058400 con 0x7ff738004d40 2024-03-08T06:00:30.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.366+0000 7ff74ed81640 1 -- 172.21.15.76:0/1095926576 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff74806ba80 con 0x7ff738004d40 2024-03-08T06:00:30.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.366+0000 7ff7457fa640 1 -- 172.21.15.76:0/1095926576 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7ff73c05c4a0 con 0x7ff738004d40 2024-03-08T06:00:30.367 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:30.367 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4249563566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4249563566},{"type":"v1","addr":"172.21.15.76:6835","nonce":4249563566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-08T06:00:30.367 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 2 2024-03-08T06:00:30.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.368+0000 7ff74ed81640 1 -- 172.21.15.76:0/1095926576 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff72c06b4d0 msgr2=0x7ff72c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:30.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.368+0000 7ff74ed81640 1 --2- 172.21.15.76:0/1095926576 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff72c06b4d0 0x7ff72c06d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff7481f7150 tx=0x7ff734009040 comp rx=0 tx=0).stop 2024-03-08T06:00:30.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.368+0000 7ff74ed81640 1 -- 172.21.15.76:0/1095926576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff738004d40 msgr2=0x7ff748157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:30.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.368+0000 7ff74ed81640 1 --2- 172.21.15.76:0/1095926576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff738004d40 0x7ff748157170 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0072c0 tx=0x7ff73c007780 comp rx=0 tx=0).stop 2024-03-08T06:00:30.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.368+0000 7ff74ed81640 1 -- 172.21.15.76:0/1095926576 shutdown_connections 2024-03-08T06:00:30.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.368+0000 7ff74ed81640 1 --2- 172.21.15.76:0/1095926576 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff72c06b4d0 0x7ff72c06d990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:30.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.368+0000 7ff74ed81640 1 --2- 172.21.15.76:0/1095926576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff748157cf0 0x7ff7480c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:30.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.368+0000 7ff74ed81640 1 --2- 172.21.15.76:0/1095926576 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7480aea00 0x7ff7481577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:30.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.368+0000 7ff74ed81640 1 --2- 172.21.15.76:0/1095926576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff738004d40 0x7ff748157170 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:30.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.368+0000 7ff74ed81640 1 -- 172.21.15.76:0/1095926576 >> 172.21.15.76:0/1095926576 conn(0x7ff748099a60 msgr2=0x7ff748159cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:30.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.368+0000 7ff74ed81640 1 -- 172.21.15.76:0/1095926576 shutdown_connections 2024-03-08T06:00:30.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:30.368+0000 7ff74ed81640 1 -- 172.21.15.76:0/1095926576 wait complete. 2024-03-08T06:00:32.381 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:00:32.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.490+0000 7f3b2f09a640 1 Processor -- start 2024-03-08T06:00:32.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.490+0000 7f3b2f09a640 1 -- start start 2024-03-08T06:00:32.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.491+0000 7f3b2f09a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3b280af9c0 0x7f3b280caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:32.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.491+0000 7f3b2f09a640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2806f5f0 con 0x7f3b2806ac80 2024-03-08T06:00:32.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.491+0000 7f3b2f09a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b280d0150 con 0x7f3b280cfa00 2024-03-08T06:00:32.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.491+0000 7f3b2f09a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b280d0430 con 0x7f3b280af9c0 2024-03-08T06:00:32.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.491+0000 7f3b2ce36640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3b2806ac80 0x7f3b2806b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46970/0 (socket says 172.21.15.76:46970) 2024-03-08T06:00:32.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.491+0000 7f3b2ce36640 1 -- 172.21.15.76:0/4190518002 learned_addr learned my addr 172.21.15.76:0/4190518002 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:32.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.491+0000 7f3b2d637640 1 --2- 172.21.15.76:0/4190518002 >> v2:172.21.15.81:3300/0 conn(0x7f3b280af9c0 0x7f3b280caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:32.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.491+0000 7f3b277fe640 1 -- 172.21.15.76:0/4190518002 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318296592 0 0) 0x7f3b2806f5f0 con 0x7f3b2806ac80 2024-03-08T06:00:32.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.491+0000 7f3b277fe640 1 -- 172.21.15.76:0/4190518002 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b18003520 con 0x7f3b2806ac80 2024-03-08T06:00:32.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.491+0000 7f3b277fe640 1 -- 172.21.15.76:0/4190518002 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290900047 0 0) 0x7f3b280d0150 con 0x7f3b280cfa00 2024-03-08T06:00:32.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.491+0000 7f3b277fe640 1 -- 172.21.15.76:0/4190518002 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b2806f5f0 con 0x7f3b280cfa00 2024-03-08T06:00:32.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.491+0000 7f3b2d637640 1 -- 172.21.15.76:0/4190518002 >> v1:172.21.15.81:6789/0 conn(0x7f3b280cfa00 legacy=0x7f3b2806eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:32.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.491+0000 7f3b2d637640 1 -- 172.21.15.76:0/4190518002 >> v1:172.21.15.76:6789/0 conn(0x7f3b2806ac80 legacy=0x7f3b2806b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:32.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.491+0000 7f3b2d637640 1 -- 172.21.15.76:0/4190518002 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2806e6e0 con 0x7f3b280af9c0 2024-03-08T06:00:32.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.492+0000 7f3b2d637640 1 --2- 172.21.15.76:0/4190518002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b280af9c0 0x7f3b280caea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009d90 tx=0x7f3b2002bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b1a883e0964d207 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:32.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.492+0000 7f3b277fe640 1 -- 172.21.15.76:0/4190518002 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20039070 con 0x7f3b280af9c0 2024-03-08T06:00:32.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.492+0000 7f3b277fe640 1 -- 172.21.15.76:0/4190518002 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2002cd00 con 0x7f3b280af9c0 2024-03-08T06:00:32.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.492+0000 7f3b277fe640 1 -- 172.21.15.76:0/4190518002 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20034660 con 0x7f3b280af9c0 2024-03-08T06:00:32.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.493+0000 7f3b2f09a640 1 -- 172.21.15.76:0/4190518002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b280af9c0 msgr2=0x7f3b280caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:32.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.493+0000 7f3b2f09a640 1 --2- 172.21.15.76:0/4190518002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b280af9c0 0x7f3b280caea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009d90 tx=0x7f3b2002bf50 comp rx=0 tx=0).stop 2024-03-08T06:00:32.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.493+0000 7f3b2f09a640 1 -- 172.21.15.76:0/4190518002 shutdown_connections 2024-03-08T06:00:32.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.493+0000 7f3b2f09a640 1 --2- 172.21.15.76:0/4190518002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b280af9c0 0x7f3b280caea0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:32.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.493+0000 7f3b2f09a640 1 -- 172.21.15.76:0/4190518002 >> 172.21.15.76:0/4190518002 conn(0x7f3b28099a60 msgr2=0x7f3b280d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:32.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.493+0000 7f3b2f09a640 1 -- 172.21.15.76:0/4190518002 shutdown_connections 2024-03-08T06:00:32.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.493+0000 7f3b2f09a640 1 -- 172.21.15.76:0/4190518002 wait complete. 2024-03-08T06:00:32.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.494+0000 7f3b2f09a640 1 Processor -- start 2024-03-08T06:00:32.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.494+0000 7f3b2f09a640 1 -- start start 2024-03-08T06:00:32.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2f09a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2806ac80 0x7f3b28157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:32.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2f09a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b280af9c0 0x7f3b280b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2f09a640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b280cfa00 0x7f3b280ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2f09a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280afe50 con 0x7f3b2806ac80 2024-03-08T06:00:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2f09a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280cfe60 con 0x7f3b280af9c0 2024-03-08T06:00:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2f09a640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2815e280 con 0x7f3b280cfa00 2024-03-08T06:00:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2ce36640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2806ac80 0x7f3b28157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2ce36640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2806ac80 0x7f3b28157e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49402/0 (socket says 172.21.15.76:49402) 2024-03-08T06:00:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2ce36640 1 -- 172.21.15.76:0/43925319 learned_addr learned my addr 172.21.15.76:0/43925319 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2d637640 1 --2- 172.21.15.76:0/43925319 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b280cfa00 0x7f3b280ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b27fff640 1 --2- 172.21.15.76:0/43925319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b280af9c0 0x7f3b280b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2ce36640 1 -- 172.21.15.76:0/43925319 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b280cfa00 msgr2=0x7f3b280ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2ce36640 1 --2- 172.21.15.76:0/43925319 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b280cfa00 0x7f3b280ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2ce36640 1 -- 172.21.15.76:0/43925319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b280af9c0 msgr2=0x7f3b280b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2ce36640 1 --2- 172.21.15.76:0/43925319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b280af9c0 0x7f3b280b2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2ce36640 1 -- 172.21.15.76:0/43925319 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b281583a0 con 0x7f3b2806ac80 2024-03-08T06:00:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2d637640 1 --2- 172.21.15.76:0/43925319 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b280cfa00 0x7f3b280ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:32.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.495+0000 7f3b2ce36640 1 --2- 172.21.15.76:0/43925319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2806ac80 0x7f3b28157e60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00c4d0 tx=0x7f3b1c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:32.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.496+0000 7f3b257fa640 1 -- 172.21.15.76:0/43925319 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c017070 con 0x7f3b2806ac80 2024-03-08T06:00:32.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.496+0000 7f3b257fa640 1 -- 172.21.15.76:0/43925319 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c005ce0 con 0x7f3b2806ac80 2024-03-08T06:00:32.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.496+0000 7f3b2f09a640 1 -- 172.21.15.76:0/43925319 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b281f6ed0 con 0x7f3b2806ac80 2024-03-08T06:00:32.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.496+0000 7f3b257fa640 1 -- 172.21.15.76:0/43925319 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c007270 con 0x7f3b2806ac80 2024-03-08T06:00:32.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.496+0000 7f3b2f09a640 1 -- 172.21.15.76:0/43925319 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b281f7490 con 0x7f3b2806ac80 2024-03-08T06:00:32.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.497+0000 7f3b2f09a640 1 -- 172.21.15.76:0/43925319 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b280676e0 con 0x7f3b2806ac80 2024-03-08T06:00:32.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.497+0000 7f3b257fa640 1 -- 172.21.15.76:0/43925319 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b1c01a070 con 0x7f3b2806ac80 2024-03-08T06:00:32.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.497+0000 7f3b257fa640 1 --2- 172.21.15.76:0/43925319 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3b0c06b500 0x7f3b0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:32.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.497+0000 7f3b257fa640 1 -- 172.21.15.76:0/43925319 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3b1c089130 con 0x7f3b2806ac80 2024-03-08T06:00:32.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.498+0000 7f3b27fff640 1 --2- 172.21.15.76:0/43925319 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3b0c06b500 0x7f3b0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:32.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.498+0000 7f3b27fff640 1 --2- 172.21.15.76:0/43925319 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3b0c06b500 0x7f3b0c06d9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3b14007fd0 tx=0x7f3b14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:32.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.502+0000 7f3b257fa640 1 -- 172.21.15.76:0/43925319 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b280676e0 con 0x7f3b2806ac80 2024-03-08T06:00:32.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.710+0000 7f3b2f09a640 1 -- 172.21.15.76:0/43925319 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b280cd040 con 0x7f3b2806ac80 2024-03-08T06:00:32.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.711+0000 7f3b257fa640 1 -- 172.21.15.76:0/43925319 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7f3b1c057cb0 con 0x7f3b2806ac80 2024-03-08T06:00:32.711 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:32.711 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4249563566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4249563566},{"type":"v1","addr":"172.21.15.76:6835","nonce":4249563566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1514641617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1514641617},{"type":"v1","addr":"172.21.15.76:6837","nonce":1514641617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6834/1332442802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1332442802},{"type":"v1","addr":"172.21.15.81:6834","nonce":1332442802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2357927814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":2357927814},{"type":"v1","addr":"172.21.15.81:6835","nonce":2357927814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-08T06:00:32.712 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 3 2024-03-08T06:00:32.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.713+0000 7f3b2f09a640 1 -- 172.21.15.76:0/43925319 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3b0c06b500 msgr2=0x7f3b0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:32.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.713+0000 7f3b2f09a640 1 --2- 172.21.15.76:0/43925319 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3b0c06b500 0x7f3b0c06d9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3b14007fd0 tx=0x7f3b14009040 comp rx=0 tx=0).stop 2024-03-08T06:00:32.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.713+0000 7f3b2f09a640 1 -- 172.21.15.76:0/43925319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2806ac80 msgr2=0x7f3b28157e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:32.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.713+0000 7f3b2f09a640 1 --2- 172.21.15.76:0/43925319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2806ac80 0x7f3b28157e60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00c4d0 tx=0x7f3b1c00c990 comp rx=0 tx=0).stop 2024-03-08T06:00:32.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.713+0000 7f3b2f09a640 1 -- 172.21.15.76:0/43925319 shutdown_connections 2024-03-08T06:00:32.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.713+0000 7f3b2f09a640 1 --2- 172.21.15.76:0/43925319 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3b0c06b500 0x7f3b0c06d9c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:32.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.713+0000 7f3b2f09a640 1 --2- 172.21.15.76:0/43925319 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b280cfa00 0x7f3b280ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:32.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.713+0000 7f3b2f09a640 1 --2- 172.21.15.76:0/43925319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b280af9c0 0x7f3b280b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:32.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.713+0000 7f3b2f09a640 1 --2- 172.21.15.76:0/43925319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2806ac80 0x7f3b28157e60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:32.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.713+0000 7f3b2f09a640 1 -- 172.21.15.76:0/43925319 >> 172.21.15.76:0/43925319 conn(0x7f3b28099a60 msgr2=0x7f3b2806f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:32.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.713+0000 7f3b2f09a640 1 -- 172.21.15.76:0/43925319 shutdown_connections 2024-03-08T06:00:32.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.713+0000 7f3b2f09a640 1 -- 172.21.15.76:0/43925319 wait complete. 2024-03-08T06:00:32.724 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:00:32.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.824+0000 7f1af1d2c640 1 Processor -- start 2024-03-08T06:00:32.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.824+0000 7f1af1d2c640 1 -- start start 2024-03-08T06:00:32.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.824+0000 7f1af1d2c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1aec06ac80 0x7f1aec0cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:32.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.824+0000 7f1af1d2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1aec0af9c0 0x7f1aec06b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:32.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.825+0000 7f1af1d2c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aec06b5a0 con 0x7f1aec06eee0 2024-03-08T06:00:32.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.825+0000 7f1af1d2c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aec0d0150 con 0x7f1aec0af9c0 2024-03-08T06:00:32.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.825+0000 7f1af1d2c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aec0d0430 con 0x7f1aec06ac80 2024-03-08T06:00:32.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.825+0000 7f1ae3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1aec0af9c0 0x7f1aec06b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:32.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.825+0000 7f1aebfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1aec06eee0 0x7f1aec0caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46986/0 (socket says 172.21.15.76:46986) 2024-03-08T06:00:32.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.825+0000 7f1aebfff640 1 -- 172.21.15.76:0/1632891330 learned_addr learned my addr 172.21.15.76:0/1632891330 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:32.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.825+0000 7f1aeb7fe640 1 --2- 172.21.15.76:0/1632891330 >> v2:172.21.15.81:3300/0 conn(0x7f1aec06ac80 0x7f1aec0cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:32.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.825+0000 7f1ae3fff640 1 -- 172.21.15.76:0/1632891330 >> v2:172.21.15.81:3300/0 conn(0x7f1aec06ac80 msgr2=0x7f1aec0cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:32.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.825+0000 7f1ae3fff640 1 --2- 172.21.15.76:0/1632891330 >> v2:172.21.15.81:3300/0 conn(0x7f1aec06ac80 0x7f1aec0cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:32.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.825+0000 7f1ae3fff640 1 -- 172.21.15.76:0/1632891330 >> v1:172.21.15.76:6789/0 conn(0x7f1aec06eee0 legacy=0x7f1aec0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:32.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.825+0000 7f1ae3fff640 1 -- 172.21.15.76:0/1632891330 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec06e6e0 con 0x7f1aec0af9c0 2024-03-08T06:00:32.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.825+0000 7f1aeaffd640 1 -- 172.21.15.76:0/1632891330 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 576756324 0 0) 0x7f1aec06b5a0 con 0x7f1aec06eee0 2024-03-08T06:00:32.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.825+0000 7f1aeb7fe640 1 --2- 172.21.15.76:0/1632891330 >> v2:172.21.15.81:3300/0 conn(0x7f1aec06ac80 0x7f1aec0cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:00:32.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.826+0000 7f1ae3fff640 1 --2- 172.21.15.76:0/1632891330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aec0af9c0 0x7f1aec06b060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0ce360 tx=0x7f1ad402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a10cf3582ebc02 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:32.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.826+0000 7f1aeaffd640 1 -- 172.21.15.76:0/1632891330 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4035020 con 0x7f1aec0af9c0 2024-03-08T06:00:32.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.826+0000 7f1aeaffd640 1 -- 172.21.15.76:0/1632891330 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad4031830 con 0x7f1aec0af9c0 2024-03-08T06:00:32.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.826+0000 7f1aeaffd640 1 -- 172.21.15.76:0/1632891330 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4031c00 con 0x7f1aec0af9c0 2024-03-08T06:00:32.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.826+0000 7f1af1d2c640 1 -- 172.21.15.76:0/1632891330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aec0af9c0 msgr2=0x7f1aec06b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:32.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.826+0000 7f1af1d2c640 1 --2- 172.21.15.76:0/1632891330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aec0af9c0 0x7f1aec06b060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0ce360 tx=0x7f1ad402b8d0 comp rx=0 tx=0).stop 2024-03-08T06:00:32.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.826+0000 7f1af1d2c640 1 -- 172.21.15.76:0/1632891330 shutdown_connections 2024-03-08T06:00:32.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.826+0000 7f1af1d2c640 1 --2- 172.21.15.76:0/1632891330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aec0af9c0 0x7f1aec06b060 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:32.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.826+0000 7f1af1d2c640 1 --2- 172.21.15.76:0/1632891330 >> v2:172.21.15.81:3300/0 conn(0x7f1aec06ac80 0x7f1aec0cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:32.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.826+0000 7f1af1d2c640 1 -- 172.21.15.76:0/1632891330 >> 172.21.15.76:0/1632891330 conn(0x7f1aec099a60 msgr2=0x7f1aec0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:32.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.826+0000 7f1af1d2c640 1 -- 172.21.15.76:0/1632891330 shutdown_connections 2024-03-08T06:00:32.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.826+0000 7f1af1d2c640 1 -- 172.21.15.76:0/1632891330 wait complete. 2024-03-08T06:00:32.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.828+0000 7f1af1d2c640 1 Processor -- start 2024-03-08T06:00:32.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.828+0000 7f1af1d2c640 1 -- start start 2024-03-08T06:00:32.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.828+0000 7f1af1d2c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aec06ac80 0x7f1aec1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:32.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.828+0000 7f1af1d2c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1aec06eee0 0x7f1aec0ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:32.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.828+0000 7f1af1d2c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1aec0af9c0 0x7f1aec0ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:32.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.828+0000 7f1af1d2c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec15e280 con 0x7f1aec06ac80 2024-03-08T06:00:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.828+0000 7f1af1d2c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec0afe20 con 0x7f1aec06eee0 2024-03-08T06:00:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.828+0000 7f1af1d2c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec0cb5b0 con 0x7f1aec0af9c0 2024-03-08T06:00:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.828+0000 7f1aeb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aec06ac80 0x7f1aec1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.828+0000 7f1aeb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aec06ac80 0x7f1aec1576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49418/0 (socket says 172.21.15.76:49418) 2024-03-08T06:00:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.828+0000 7f1aeb7fe640 1 -- 172.21.15.76:0/4204929974 learned_addr learned my addr 172.21.15.76:0/4204929974 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.829+0000 7f1ae3fff640 1 --2- 172.21.15.76:0/4204929974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1aec06eee0 0x7f1aec0ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.829+0000 7f1aeb7fe640 1 -- 172.21.15.76:0/4204929974 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1aec0af9c0 msgr2=0x7f1aec0ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.829+0000 7f1aeb7fe640 1 --2- 172.21.15.76:0/4204929974 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1aec0af9c0 0x7f1aec0ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.829+0000 7f1aeb7fe640 1 -- 172.21.15.76:0/4204929974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1aec06eee0 msgr2=0x7f1aec0ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.829+0000 7f1aeb7fe640 1 --2- 172.21.15.76:0/4204929974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1aec06eee0 0x7f1aec0ca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.829+0000 7f1aeb7fe640 1 -- 172.21.15.76:0/4204929974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec160e10 con 0x7f1aec06ac80 2024-03-08T06:00:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.829+0000 7f1aeb7fe640 1 --2- 172.21.15.76:0/4204929974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aec06ac80 0x7f1aec1576e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00e7e0 tx=0x7f1adc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.829+0000 7f1ae8ff9640 1 -- 172.21.15.76:0/4204929974 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc00f040 con 0x7f1aec06ac80 2024-03-08T06:00:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.829+0000 7f1ae8ff9640 1 -- 172.21.15.76:0/4204929974 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1adc004030 con 0x7f1aec06ac80 2024-03-08T06:00:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.829+0000 7f1af1d2c640 1 -- 172.21.15.76:0/4204929974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aec1610f0 con 0x7f1aec06ac80 2024-03-08T06:00:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.829+0000 7f1ae8ff9640 1 -- 172.21.15.76:0/4204929974 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc004400 con 0x7f1aec06ac80 2024-03-08T06:00:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.829+0000 7f1af1d2c640 1 -- 172.21.15.76:0/4204929974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aec157c20 con 0x7f1aec06ac80 2024-03-08T06:00:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.830+0000 7f1af1d2c640 1 -- 172.21.15.76:0/4204929974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aec04f030 con 0x7f1aec06ac80 2024-03-08T06:00:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.830+0000 7f1ae8ff9640 1 -- 172.21.15.76:0/4204929974 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1adc00c6d0 con 0x7f1aec06ac80 2024-03-08T06:00:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.831+0000 7f1ae8ff9640 1 --2- 172.21.15.76:0/4204929974 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1acc06b500 0x7f1acc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.831+0000 7f1ae8ff9640 1 -- 172.21.15.76:0/4204929974 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1adc089c80 con 0x7f1aec06ac80 2024-03-08T06:00:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.831+0000 7f1ae3fff640 1 --2- 172.21.15.76:0/4204929974 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1acc06b500 0x7f1acc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.831+0000 7f1ae3fff640 1 --2- 172.21.15.76:0/4204929974 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1acc06b500 0x7f1acc06d9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4036000 tx=0x7f1ad4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:32.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:32.835+0000 7f1ae8ff9640 1 -- 172.21.15.76:0/4204929974 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1adc017020 con 0x7f1aec06ac80 2024-03-08T06:00:33.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.005+0000 7f1af1d2c640 1 -- 172.21.15.76:0/4204929974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1aec161760 con 0x7f1aec06ac80 2024-03-08T06:00:33.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.006+0000 7f1ae8ff9640 1 -- 172.21.15.76:0/4204929974 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12800 (secure 0 0 0) 0x7f1adc058e50 con 0x7f1aec06ac80 2024-03-08T06:00:33.006 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:33.007 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":17,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:27.125292+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:00:33.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.008+0000 7f1af1d2c640 1 -- 172.21.15.76:0/4204929974 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1acc06b500 msgr2=0x7f1acc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.008+0000 7f1af1d2c640 1 --2- 172.21.15.76:0/4204929974 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1acc06b500 0x7f1acc06d9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4036000 tx=0x7f1ad4008030 comp rx=0 tx=0).stop 2024-03-08T06:00:33.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.008+0000 7f1af1d2c640 1 -- 172.21.15.76:0/4204929974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aec06ac80 msgr2=0x7f1aec1576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.008+0000 7f1af1d2c640 1 --2- 172.21.15.76:0/4204929974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aec06ac80 0x7f1aec1576e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00e7e0 tx=0x7f1adc00eca0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.008+0000 7f1af1d2c640 1 -- 172.21.15.76:0/4204929974 shutdown_connections 2024-03-08T06:00:33.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.008+0000 7f1af1d2c640 1 --2- 172.21.15.76:0/4204929974 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1acc06b500 0x7f1acc06d9c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.008+0000 7f1af1d2c640 1 --2- 172.21.15.76:0/4204929974 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1aec0af9c0 0x7f1aec0ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.008+0000 7f1af1d2c640 1 --2- 172.21.15.76:0/4204929974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1aec06eee0 0x7f1aec0ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.008+0000 7f1af1d2c640 1 --2- 172.21.15.76:0/4204929974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aec06ac80 0x7f1aec1576e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.008+0000 7f1af1d2c640 1 -- 172.21.15.76:0/4204929974 >> 172.21.15.76:0/4204929974 conn(0x7f1aec099a60 msgr2=0x7f1aec0cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:33.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.008+0000 7f1af1d2c640 1 -- 172.21.15.76:0/4204929974 shutdown_connections 2024-03-08T06:00:33.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.008+0000 7f1af1d2c640 1 -- 172.21.15.76:0/4204929974 wait complete. 2024-03-08T06:00:33.019 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:00:33.019 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:00:33.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.121+0000 7fc00af5c640 1 Processor -- start 2024-03-08T06:00:33.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.121+0000 7fc00af5c640 1 -- start start 2024-03-08T06:00:33.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.122+0000 7fc00af5c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0040ce5e0 0x7fc00406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.122+0000 7fc00af5c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc0040ce9c0 0x7fc004162020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.122+0000 7fc00af5c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc004161840 con 0x7fc004162560 2024-03-08T06:00:33.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.122+0000 7fc00af5c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc004161b90 con 0x7fc0040ce9c0 2024-03-08T06:00:33.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.122+0000 7fc00af5c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00415eb30 con 0x7fc0040ce5e0 2024-03-08T06:00:33.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.122+0000 7fc008cf8640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0040ce5e0 0x7fc00406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.122+0000 7fc003fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc0040ce9c0 0x7fc004162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.122+0000 7fc008cf8640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0040ce5e0 0x7fc00406d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:44804/0 (socket says 172.21.15.76:44804) 2024-03-08T06:00:33.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.122+0000 7fc0094f9640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc004162560 0x7fc00415dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:47626/0 (socket says 172.21.15.76:47626) 2024-03-08T06:00:33.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.122+0000 7fc008cf8640 1 -- 172.21.15.76:0/2572946479 learned_addr learned my addr 172.21.15.76:0/2572946479 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:33.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.122+0000 7fc008cf8640 1 -- 172.21.15.76:0/2572946479 >> v2:172.21.15.81:3300/0 conn(0x7fc0040ce9c0 msgr2=0x7fc004162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:33.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.122+0000 7fc008cf8640 1 --2- 172.21.15.76:0/2572946479 >> v2:172.21.15.81:3300/0 conn(0x7fc0040ce9c0 0x7fc004162020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.122+0000 7fc008cf8640 1 -- 172.21.15.76:0/2572946479 >> v1:172.21.15.81:6789/0 conn(0x7fc004162560 legacy=0x7fc00415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.122+0000 7fc008cf8640 1 -- 172.21.15.76:0/2572946479 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00415ee10 con 0x7fc0040ce5e0 2024-03-08T06:00:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.123+0000 7fc008cf8640 1 --2- 172.21.15.76:0/2572946479 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0040ce5e0 0x7fc00406d8a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009870 tx=0x7fbff802ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aea7fcec73142b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.123+0000 7fc0037fe640 1 -- 172.21.15.76:0/2572946479 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff802f840 con 0x7fc0040ce5e0 2024-03-08T06:00:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.123+0000 7fc0037fe640 1 -- 172.21.15.76:0/2572946479 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff802f9e0 con 0x7fc0040ce5e0 2024-03-08T06:00:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.123+0000 7fc0037fe640 1 -- 172.21.15.76:0/2572946479 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff802fdb0 con 0x7fc0040ce5e0 2024-03-08T06:00:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.124+0000 7fc00af5c640 1 -- 172.21.15.76:0/2572946479 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0040ce5e0 msgr2=0x7fc00406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.124+0000 7fc00af5c640 1 --2- 172.21.15.76:0/2572946479 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0040ce5e0 0x7fc00406d8a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009870 tx=0x7fbff802ba20 comp rx=0 tx=0).stop 2024-03-08T06:00:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.124+0000 7fc00af5c640 1 -- 172.21.15.76:0/2572946479 shutdown_connections 2024-03-08T06:00:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.124+0000 7fc00af5c640 1 --2- 172.21.15.76:0/2572946479 >> v2:172.21.15.81:3300/0 conn(0x7fc0040ce9c0 0x7fc004162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.124+0000 7fc00af5c640 1 --2- 172.21.15.76:0/2572946479 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0040ce5e0 0x7fc00406d8a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.124+0000 7fc00af5c640 1 -- 172.21.15.76:0/2572946479 >> 172.21.15.76:0/2572946479 conn(0x7fc00406ae60 msgr2=0x7fc00406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.124+0000 7fc00af5c640 1 -- 172.21.15.76:0/2572946479 shutdown_connections 2024-03-08T06:00:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.124+0000 7fc00af5c640 1 -- 172.21.15.76:0/2572946479 wait complete. 2024-03-08T06:00:33.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.125+0000 7fc00af5c640 1 Processor -- start 2024-03-08T06:00:33.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.125+0000 7fc00af5c640 1 -- start start 2024-03-08T06:00:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc00af5c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0040ce5e0 0x7fc00415cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc00af5c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0040ce9c0 0x7fc0040b6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc00af5c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc004162560 0x7fc0040c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc00af5c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00415e6c0 con 0x7fc0040ce9c0 2024-03-08T06:00:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc00af5c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040cee20 con 0x7fc004162560 2024-03-08T06:00:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc00af5c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00415e840 con 0x7fc0040ce5e0 2024-03-08T06:00:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc003fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0040ce9c0 0x7fc0040b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc003fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0040ce9c0 0x7fc0040b6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49422/0 (socket says 172.21.15.76:49422) 2024-03-08T06:00:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc003fff640 1 -- 172.21.15.76:0/3791936987 learned_addr learned my addr 172.21.15.76:0/3791936987 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc008cf8640 1 --2- 172.21.15.76:0/3791936987 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0040ce5e0 0x7fc00415cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc0094f9640 1 --2- 172.21.15.76:0/3791936987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc004162560 0x7fc0040c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc003fff640 1 -- 172.21.15.76:0/3791936987 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0040ce5e0 msgr2=0x7fc00415cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc003fff640 1 --2- 172.21.15.76:0/3791936987 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0040ce5e0 0x7fc00415cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc003fff640 1 -- 172.21.15.76:0/3791936987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc004162560 msgr2=0x7fc0040c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc003fff640 1 --2- 172.21.15.76:0/3791936987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc004162560 0x7fc0040c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc003fff640 1 -- 172.21.15.76:0/3791936987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0040c5bf0 con 0x7fc0040ce9c0 2024-03-08T06:00:33.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc003fff640 1 --2- 172.21.15.76:0/3791936987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0040ce9c0 0x7fc0040b6890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbff000e870 tx=0x7fbff000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:33.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc0017fa640 1 -- 172.21.15.76:0/3791936987 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0017070 con 0x7fc0040ce9c0 2024-03-08T06:00:33.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc0017fa640 1 -- 172.21.15.76:0/3791936987 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff000c8e0 con 0x7fc0040ce9c0 2024-03-08T06:00:33.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc00af5c640 1 -- 172.21.15.76:0/3791936987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0040c5ed0 con 0x7fc0040ce9c0 2024-03-08T06:00:33.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.126+0000 7fc0017fa640 1 -- 172.21.15.76:0/3791936987 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff000ccd0 con 0x7fc0040ce9c0 2024-03-08T06:00:33.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.127+0000 7fc00af5c640 1 -- 172.21.15.76:0/3791936987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0040c6490 con 0x7fc0040ce9c0 2024-03-08T06:00:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.127+0000 7fc00af5c640 1 -- 172.21.15.76:0/3791936987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00404f030 con 0x7fc0040ce9c0 2024-03-08T06:00:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.128+0000 7fc0017fa640 1 -- 172.21.15.76:0/3791936987 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbff001a070 con 0x7fc0040ce9c0 2024-03-08T06:00:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.128+0000 7fc0017fa640 1 --2- 172.21.15.76:0/3791936987 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbfe806b4d0 0x7fbfe806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.128+0000 7fc008cf8640 1 --2- 172.21.15.76:0/3791936987 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbfe806b4d0 0x7fbfe806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.128+0000 7fc0017fa640 1 -- 172.21.15.76:0/3791936987 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbff0089fe0 con 0x7fc0040ce9c0 2024-03-08T06:00:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.129+0000 7fc008cf8640 1 --2- 172.21.15.76:0/3791936987 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbfe806b4d0 0x7fbfe806d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbff8036000 tx=0x7fbff8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:33.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.133+0000 7fc0017fa640 1 -- 172.21.15.76:0/3791936987 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbff0058b60 con 0x7fc0040ce9c0 2024-03-08T06:00:33.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.306+0000 7fc00af5c640 1 -- 172.21.15.76:0/3791936987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc00415e330 con 0x7fc0040ce9c0 2024-03-08T06:00:33.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.306+0000 7fc0017fa640 1 -- 172.21.15.76:0/3791936987 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbff005cc00 con 0x7fc0040ce9c0 2024-03-08T06:00:33.307 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:00:33.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.308+0000 7fc00af5c640 1 -- 172.21.15.76:0/3791936987 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbfe806b4d0 msgr2=0x7fbfe806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.308+0000 7fc00af5c640 1 --2- 172.21.15.76:0/3791936987 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbfe806b4d0 0x7fbfe806d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbff8036000 tx=0x7fbff8008030 comp rx=0 tx=0).stop 2024-03-08T06:00:33.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.308+0000 7fc00af5c640 1 -- 172.21.15.76:0/3791936987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0040ce9c0 msgr2=0x7fc0040b6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.308+0000 7fc00af5c640 1 --2- 172.21.15.76:0/3791936987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0040ce9c0 0x7fc0040b6890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbff000e870 tx=0x7fbff000ed30 comp rx=0 tx=0).stop 2024-03-08T06:00:33.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.308+0000 7fc00af5c640 1 -- 172.21.15.76:0/3791936987 shutdown_connections 2024-03-08T06:00:33.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.308+0000 7fc00af5c640 1 --2- 172.21.15.76:0/3791936987 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbfe806b4d0 0x7fbfe806d990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.308+0000 7fc00af5c640 1 --2- 172.21.15.76:0/3791936987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc004162560 0x7fc0040c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.308+0000 7fc00af5c640 1 --2- 172.21.15.76:0/3791936987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0040ce9c0 0x7fc0040b6890 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.308+0000 7fc00af5c640 1 --2- 172.21.15.76:0/3791936987 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc0040ce5e0 0x7fc00415cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.308+0000 7fc00af5c640 1 -- 172.21.15.76:0/3791936987 >> 172.21.15.76:0/3791936987 conn(0x7fc00406ae60 msgr2=0x7fc0040d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:33.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.309+0000 7fc00af5c640 1 -- 172.21.15.76:0/3791936987 shutdown_connections 2024-03-08T06:00:33.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.309+0000 7fc00af5c640 1 -- 172.21.15.76:0/3791936987 wait complete. 2024-03-08T06:00:33.320 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-08T06:00:33.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.420+0000 7f201f325640 1 Processor -- start 2024-03-08T06:00:33.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.420+0000 7f201f325640 1 -- start start 2024-03-08T06:00:33.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.421+0000 7f201f325640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f201815dff0 0x7f20180b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.421+0000 7f201f325640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20181623e0 con 0x7f201815dc10 2024-03-08T06:00:33.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.421+0000 7f201f325640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20181626c0 con 0x7f20180b6e70 2024-03-08T06:00:33.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.421+0000 7f201f325640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20180d2300 con 0x7f201815dff0 2024-03-08T06:00:33.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.421+0000 7f201d0c1640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f201815dc10 0x7f2018161990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46994/0 (socket says 172.21.15.76:46994) 2024-03-08T06:00:33.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.421+0000 7f201d0c1640 1 -- 172.21.15.76:0/2922859807 learned_addr learned my addr 172.21.15.76:0/2922859807 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:33.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.421+0000 7f201c8c0640 1 --2- 172.21.15.76:0/2922859807 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f201815dff0 0x7f20180b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.421+0000 7f200ffff640 1 -- 172.21.15.76:0/2922859807 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268526352 0 0) 0x7f20181623e0 con 0x7f201815dc10 2024-03-08T06:00:33.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.421+0000 7f200ffff640 1 -- 172.21.15.76:0/2922859807 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2004003520 con 0x7f201815dc10 2024-03-08T06:00:33.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.421+0000 7f200ffff640 1 -- 172.21.15.76:0/2922859807 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2519274900 0 0) 0x7f20181626c0 con 0x7f20180b6e70 2024-03-08T06:00:33.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.421+0000 7f200ffff640 1 -- 172.21.15.76:0/2922859807 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20181623e0 con 0x7f20180b6e70 2024-03-08T06:00:33.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.421+0000 7f201c8c0640 1 -- 172.21.15.76:0/2922859807 >> v1:172.21.15.81:6789/0 conn(0x7f20180b6e70 legacy=0x7f20180d2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.421+0000 7f201c8c0640 1 -- 172.21.15.76:0/2922859807 >> v1:172.21.15.76:6789/0 conn(0x7f201815dc10 legacy=0x7f2018161990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.421+0000 7f201c8c0640 1 -- 172.21.15.76:0/2922859807 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20181620a0 con 0x7f201815dff0 2024-03-08T06:00:33.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.422+0000 7f201c8c0640 1 --2- 172.21.15.76:0/2922859807 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f201815dff0 0x7f20180b56c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f20080097d0 tx=0x7f200802b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=474e11bf0fa72df3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:33.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.422+0000 7f200ffff640 1 -- 172.21.15.76:0/2922859807 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008031770 con 0x7f201815dff0 2024-03-08T06:00:33.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.422+0000 7f200ffff640 1 -- 172.21.15.76:0/2922859807 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2008037070 con 0x7f201815dff0 2024-03-08T06:00:33.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.422+0000 7f200ffff640 1 -- 172.21.15.76:0/2922859807 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008031b30 con 0x7f201815dff0 2024-03-08T06:00:33.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.423+0000 7f201f325640 1 -- 172.21.15.76:0/2922859807 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f201815dff0 msgr2=0x7f20180b56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.423+0000 7f201f325640 1 --2- 172.21.15.76:0/2922859807 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f201815dff0 0x7f20180b56c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f20080097d0 tx=0x7f200802b810 comp rx=0 tx=0).stop 2024-03-08T06:00:33.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.423+0000 7f201f325640 1 -- 172.21.15.76:0/2922859807 shutdown_connections 2024-03-08T06:00:33.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.423+0000 7f201f325640 1 --2- 172.21.15.76:0/2922859807 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f201815dff0 0x7f20180b56c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.423+0000 7f201f325640 1 -- 172.21.15.76:0/2922859807 >> 172.21.15.76:0/2922859807 conn(0x7f20180d06c0 msgr2=0x7f20180d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:33.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.423+0000 7f201f325640 1 -- 172.21.15.76:0/2922859807 shutdown_connections 2024-03-08T06:00:33.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.423+0000 7f201f325640 1 -- 172.21.15.76:0/2922859807 wait complete. 2024-03-08T06:00:33.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.424+0000 7f201f325640 1 Processor -- start 2024-03-08T06:00:33.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.424+0000 7f201f325640 1 -- start start 2024-03-08T06:00:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201f325640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20180b6e70 0x7f20181533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201f325640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f201815dc10 0x7f2018158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201f325640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f201815dff0 0x7f20180c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201f325640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20180b5d80 con 0x7f201815dff0 2024-03-08T06:00:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201f325640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20180b5c00 con 0x7f20180b6e70 2024-03-08T06:00:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201f325640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20180b5f00 con 0x7f201815dc10 2024-03-08T06:00:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201d8c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f201815dff0 0x7f20180c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201d8c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f201815dff0 0x7f20180c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49438/0 (socket says 172.21.15.76:49438) 2024-03-08T06:00:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201d8c2640 1 -- 172.21.15.76:0/4149650670 learned_addr learned my addr 172.21.15.76:0/4149650670 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201d0c1640 1 --2- 172.21.15.76:0/4149650670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20180b6e70 0x7f20181533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201c8c0640 1 --2- 172.21.15.76:0/4149650670 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f201815dc10 0x7f2018158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201d8c2640 1 -- 172.21.15.76:0/4149650670 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f201815dc10 msgr2=0x7f2018158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201d8c2640 1 --2- 172.21.15.76:0/4149650670 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f201815dc10 0x7f2018158dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201d8c2640 1 -- 172.21.15.76:0/4149650670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20180b6e70 msgr2=0x7f20181533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201d8c2640 1 --2- 172.21.15.76:0/4149650670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20180b6e70 0x7f20181533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201d8c2640 1 -- 172.21.15.76:0/4149650670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2018153930 con 0x7f201815dff0 2024-03-08T06:00:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.425+0000 7f201d8c2640 1 --2- 172.21.15.76:0/4149650670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f201815dff0 0x7f20180c8780 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f201000c710 tx=0x7f201000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.426+0000 7f200dffb640 1 -- 172.21.15.76:0/4149650670 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010003b70 con 0x7f201815dff0 2024-03-08T06:00:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.426+0000 7f200dffb640 1 -- 172.21.15.76:0/4149650670 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2010003d10 con 0x7f201815dff0 2024-03-08T06:00:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.426+0000 7f201f325640 1 -- 172.21.15.76:0/4149650670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20180c79c0 con 0x7f201815dff0 2024-03-08T06:00:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.426+0000 7f200dffb640 1 -- 172.21.15.76:0/4149650670 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201000aa50 con 0x7f201815dff0 2024-03-08T06:00:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.426+0000 7f201f325640 1 -- 172.21.15.76:0/4149650670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20180c7f80 con 0x7f201815dff0 2024-03-08T06:00:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.427+0000 7f201f325640 1 -- 172.21.15.76:0/4149650670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20180ca2b0 con 0x7f201815dff0 2024-03-08T06:00:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.427+0000 7f200dffb640 1 -- 172.21.15.76:0/4149650670 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f201000abf0 con 0x7f201815dff0 2024-03-08T06:00:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.427+0000 7f200dffb640 1 --2- 172.21.15.76:0/4149650670 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1ffc06b500 0x7f1ffc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.428+0000 7f200dffb640 1 -- 172.21.15.76:0/4149650670 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2010089b70 con 0x7f201815dff0 2024-03-08T06:00:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.428+0000 7f201d0c1640 1 --2- 172.21.15.76:0/4149650670 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1ffc06b500 0x7f1ffc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.428+0000 7f201d0c1640 1 --2- 172.21.15.76:0/4149650670 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1ffc06b500 0x7f1ffc06d9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2014000f60 tx=0x7f2014007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:33.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.432+0000 7f200dffb640 1 -- 172.21.15.76:0/4149650670 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f201000ce20 con 0x7f201815dff0 2024-03-08T06:00:33.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.607+0000 7f201f325640 1 -- 172.21.15.76:0/4149650670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f201804f030 con 0x7f201815dff0 2024-03-08T06:00:33.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.607+0000 7f200dffb640 1 -- 172.21.15.76:0/4149650670 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2010058780 con 0x7f201815dff0 2024-03-08T06:00:33.607 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:33.607 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2024-03-08T06:00:33.608 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:00:33.608 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2024-03-08T06:00:33.608 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2024-03-08T06:00:33.608 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:00:33.608 INFO:teuthology.orchestra.run.smithi076.stdout:] 2024-03-08T06:00:33.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.609+0000 7f201f325640 1 -- 172.21.15.76:0/4149650670 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1ffc06b500 msgr2=0x7f1ffc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.609+0000 7f201f325640 1 --2- 172.21.15.76:0/4149650670 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1ffc06b500 0x7f1ffc06d9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2014000f60 tx=0x7f2014007d30 comp rx=0 tx=0).stop 2024-03-08T06:00:33.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.609+0000 7f201f325640 1 -- 172.21.15.76:0/4149650670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f201815dff0 msgr2=0x7f20180c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.609+0000 7f201f325640 1 --2- 172.21.15.76:0/4149650670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f201815dff0 0x7f20180c8780 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f201000c710 tx=0x7f201000cbd0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.609+0000 7f201f325640 1 -- 172.21.15.76:0/4149650670 shutdown_connections 2024-03-08T06:00:33.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.609+0000 7f201f325640 1 --2- 172.21.15.76:0/4149650670 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1ffc06b500 0x7f1ffc06d9c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.609+0000 7f201f325640 1 --2- 172.21.15.76:0/4149650670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f201815dff0 0x7f20180c8780 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.609+0000 7f201f325640 1 --2- 172.21.15.76:0/4149650670 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f201815dc10 0x7f2018158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.609+0000 7f201f325640 1 --2- 172.21.15.76:0/4149650670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20180b6e70 0x7f20181533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.609+0000 7f201f325640 1 -- 172.21.15.76:0/4149650670 >> 172.21.15.76:0/4149650670 conn(0x7f20180d06c0 msgr2=0x7f201815b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:33.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.609+0000 7f201f325640 1 -- 172.21.15.76:0/4149650670 shutdown_connections 2024-03-08T06:00:33.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.609+0000 7f201f325640 1 -- 172.21.15.76:0/4149650670 wait complete. 2024-03-08T06:00:33.620 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-08T06:00:33.620 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-08T06:00:33.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.724+0000 7f43025d7640 1 Processor -- start 2024-03-08T06:00:33.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.724+0000 7f43025d7640 1 -- start start 2024-03-08T06:00:33.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.725+0000 7f43025d7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f42fc15df20 0x7f42fc0d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.725+0000 7f43025d7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f42fc0d0a40 0x7f42fc0d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.725+0000 7f43025d7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42fc0cf020 con 0x7f42fc15db40 2024-03-08T06:00:33.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.725+0000 7f43025d7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42fc0d0ed0 con 0x7f42fc0d0a40 2024-03-08T06:00:33.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.725+0000 7f43025d7640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42fc0cf300 con 0x7f42fc15df20 2024-03-08T06:00:33.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.725+0000 7f4300b74640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f42fc0d0a40 0x7f42fc0d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.725+0000 7f4300b74640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f42fc0d0a40 0x7f42fc0d28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49446/0 (socket says 172.21.15.76:49446) 2024-03-08T06:00:33.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.725+0000 7f4300b74640 1 -- 172.21.15.76:0/1533993572 learned_addr learned my addr 172.21.15.76:0/1533993572 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:33.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.725+0000 7f42fbfff640 1 --1- 172.21.15.76:0/1533993572 >> v1:172.21.15.76:6789/0 conn(0x7f42fc15db40 0x7f42fc1622b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47006/0 (socket says 172.21.15.76:47006) 2024-03-08T06:00:33.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.725+0000 7f42fb7fe640 1 --2- 172.21.15.76:0/1533993572 >> v2:172.21.15.81:3300/0 conn(0x7f42fc15df20 0x7f42fc0d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.726+0000 7f4300b74640 1 -- 172.21.15.76:0/1533993572 >> v2:172.21.15.81:3300/0 conn(0x7f42fc15df20 msgr2=0x7f42fc0d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:33.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.726+0000 7f4300b74640 1 --2- 172.21.15.76:0/1533993572 >> v2:172.21.15.81:3300/0 conn(0x7f42fc15df20 0x7f42fc0d0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.726+0000 7f4300b74640 1 -- 172.21.15.76:0/1533993572 >> v1:172.21.15.76:6789/0 conn(0x7f42fc15db40 legacy=0x7f42fc1622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.726+0000 7f4300b74640 1 -- 172.21.15.76:0/1533993572 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc0d1e10 con 0x7f42fc0d0a40 2024-03-08T06:00:33.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.726+0000 7f42faffd640 1 -- 172.21.15.76:0/1533993572 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564252347 0 0) 0x7f42fc0cf020 con 0x7f42fc15db40 2024-03-08T06:00:33.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.726+0000 7f4300b74640 1 --2- 172.21.15.76:0/1533993572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42fc0d0a40 0x7f42fc0d28f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0b66c0 tx=0x7f42f402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68dd2a297c83204e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:33.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.726+0000 7f42faffd640 1 -- 172.21.15.76:0/1533993572 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f402cb20 con 0x7f42fc0d0a40 2024-03-08T06:00:33.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.726+0000 7f42faffd640 1 -- 172.21.15.76:0/1533993572 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f402ccc0 con 0x7f42fc0d0a40 2024-03-08T06:00:33.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.726+0000 7f42faffd640 1 -- 172.21.15.76:0/1533993572 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f4034610 con 0x7f42fc0d0a40 2024-03-08T06:00:33.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.727+0000 7f43025d7640 1 -- 172.21.15.76:0/1533993572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42fc0d0a40 msgr2=0x7f42fc0d28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.727+0000 7f43025d7640 1 --2- 172.21.15.76:0/1533993572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42fc0d0a40 0x7f42fc0d28f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0b66c0 tx=0x7f42f402bf80 comp rx=0 tx=0).stop 2024-03-08T06:00:33.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.727+0000 7f43025d7640 1 -- 172.21.15.76:0/1533993572 shutdown_connections 2024-03-08T06:00:33.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.727+0000 7f43025d7640 1 --2- 172.21.15.76:0/1533993572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42fc0d0a40 0x7f42fc0d28f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.727+0000 7f43025d7640 1 --2- 172.21.15.76:0/1533993572 >> v2:172.21.15.81:3300/0 conn(0x7f42fc15df20 0x7f42fc0d0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.727+0000 7f43025d7640 1 -- 172.21.15.76:0/1533993572 >> 172.21.15.76:0/1533993572 conn(0x7f42fc066fc0 msgr2=0x7f42fc0b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:33.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.727+0000 7f43025d7640 1 -- 172.21.15.76:0/1533993572 shutdown_connections 2024-03-08T06:00:33.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.727+0000 7f43025d7640 1 -- 172.21.15.76:0/1533993572 wait complete. 2024-03-08T06:00:33.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.728+0000 7f43025d7640 1 Processor -- start 2024-03-08T06:00:33.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.728+0000 7f43025d7640 1 -- start start 2024-03-08T06:00:33.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.729+0000 7f43025d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42fc0d0a40 0x7f42fc1f1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.729+0000 7f43025d7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42fc15db40 0x7f42fc1f4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.729+0000 7f43025d7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f42fc15df20 0x7f42fc1f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.729+0000 7f43025d7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc0b6e10 con 0x7f42fc0d0a40 2024-03-08T06:00:33.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.729+0000 7f43025d7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc0d23d0 con 0x7f42fc15db40 2024-03-08T06:00:33.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.729+0000 7f43025d7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc0ce670 con 0x7f42fc15df20 2024-03-08T06:00:33.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.729+0000 7f42fbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42fc0d0a40 0x7f42fc1f1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.729+0000 7f42fbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42fc0d0a40 0x7f42fc1f1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49458/0 (socket says 172.21.15.76:49458) 2024-03-08T06:00:33.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.729+0000 7f42fbfff640 1 -- 172.21.15.76:0/2992090499 learned_addr learned my addr 172.21.15.76:0/2992090499 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:33.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.729+0000 7f42fb7fe640 1 --2- 172.21.15.76:0/2992090499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42fc15db40 0x7f42fc1f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.729+0000 7f4300b74640 1 --2- 172.21.15.76:0/2992090499 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f42fc15df20 0x7f42fc1f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.729+0000 7f42fbfff640 1 -- 172.21.15.76:0/2992090499 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f42fc15df20 msgr2=0x7f42fc1f4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.729+0000 7f42fbfff640 1 --2- 172.21.15.76:0/2992090499 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f42fc15df20 0x7f42fc1f4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.729+0000 7f42fbfff640 1 -- 172.21.15.76:0/2992090499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42fc15db40 msgr2=0x7f42fc1f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:33.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.729+0000 7f42fbfff640 1 --2- 172.21.15.76:0/2992090499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42fc15db40 0x7f42fc1f4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:33.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.729+0000 7f42fbfff640 1 -- 172.21.15.76:0/2992090499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc1f4dc0 con 0x7f42fc0d0a40 2024-03-08T06:00:33.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.730+0000 7f42fbfff640 1 --2- 172.21.15.76:0/2992090499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42fc0d0a40 0x7f42fc1f1390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00c4d0 tx=0x7f42ec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:33.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.730+0000 7f42f8ff9640 1 -- 172.21.15.76:0/2992090499 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec018070 con 0x7f42fc0d0a40 2024-03-08T06:00:33.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.730+0000 7f42f8ff9640 1 -- 172.21.15.76:0/2992090499 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec007500 con 0x7f42fc0d0a40 2024-03-08T06:00:33.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.730+0000 7f42f8ff9640 1 -- 172.21.15.76:0/2992090499 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec004a20 con 0x7f42fc0d0a40 2024-03-08T06:00:33.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.730+0000 7f43025d7640 1 -- 172.21.15.76:0/2992090499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42fc15e6c0 con 0x7f42fc0d0a40 2024-03-08T06:00:33.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.730+0000 7f43025d7640 1 -- 172.21.15.76:0/2992090499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42fc15ec80 con 0x7f42fc0d0a40 2024-03-08T06:00:33.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.731+0000 7f43025d7640 1 -- 172.21.15.76:0/2992090499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42fc072ee0 con 0x7f42fc0d0a40 2024-03-08T06:00:33.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.731+0000 7f42f8ff9640 1 -- 172.21.15.76:0/2992090499 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42ec01b070 con 0x7f42fc0d0a40 2024-03-08T06:00:33.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.732+0000 7f42f8ff9640 1 --2- 172.21.15.76:0/2992090499 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f42e806b4d0 0x7f42e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:33.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.732+0000 7f42f8ff9640 1 -- 172.21.15.76:0/2992090499 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f42ec0891c0 con 0x7f42fc0d0a40 2024-03-08T06:00:33.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.732+0000 7f42fb7fe640 1 --2- 172.21.15.76:0/2992090499 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f42e806b4d0 0x7f42e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:33.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.732+0000 7f42fb7fe640 1 --2- 172.21.15.76:0/2992090499 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f42e806b4d0 0x7f42e806d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f42f0006fd0 tx=0x7f42f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:33.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.736+0000 7f42f8ff9640 1 -- 172.21.15.76:0/2992090499 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42fc072ee0 con 0x7f42fc0d0a40 2024-03-08T06:00:33.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:33.907+0000 7f43025d7640 1 -- 172.21.15.76:0/2992090499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f42fc161350 con 0x7f42fc0d0a40 2024-03-08T06:00:34.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.145+0000 7f42f8ff9640 1 -- 172.21.15.76:0/2992090499 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f42fc1f4dc0 con 0x7f42fc0d0a40 2024-03-08T06:00:34.146 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2024-03-08T06:00:34.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.148+0000 7f43025d7640 1 -- 172.21.15.76:0/2992090499 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f42e806b4d0 msgr2=0x7f42e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:34.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.148+0000 7f43025d7640 1 --2- 172.21.15.76:0/2992090499 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f42e806b4d0 0x7f42e806d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f42f0006fd0 tx=0x7f42f0008040 comp rx=0 tx=0).stop 2024-03-08T06:00:34.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.148+0000 7f43025d7640 1 -- 172.21.15.76:0/2992090499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42fc0d0a40 msgr2=0x7f42fc1f1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:34.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.148+0000 7f43025d7640 1 --2- 172.21.15.76:0/2992090499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42fc0d0a40 0x7f42fc1f1390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00c4d0 tx=0x7f42ec00c990 comp rx=0 tx=0).stop 2024-03-08T06:00:34.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.148+0000 7f43025d7640 1 -- 172.21.15.76:0/2992090499 shutdown_connections 2024-03-08T06:00:34.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.148+0000 7f43025d7640 1 --2- 172.21.15.76:0/2992090499 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f42e806b4d0 0x7f42e806d990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:34.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.148+0000 7f43025d7640 1 --2- 172.21.15.76:0/2992090499 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f42fc15df20 0x7f42fc1f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:34.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.148+0000 7f43025d7640 1 --2- 172.21.15.76:0/2992090499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42fc15db40 0x7f42fc1f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:34.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.148+0000 7f43025d7640 1 --2- 172.21.15.76:0/2992090499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42fc0d0a40 0x7f42fc1f1390 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:34.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.148+0000 7f43025d7640 1 -- 172.21.15.76:0/2992090499 >> 172.21.15.76:0/2992090499 conn(0x7f42fc066fc0 msgr2=0x7f42fc156fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:34.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.149+0000 7f43025d7640 1 -- 172.21.15.76:0/2992090499 shutdown_connections 2024-03-08T06:00:34.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.149+0000 7f43025d7640 1 -- 172.21.15.76:0/2992090499 wait complete. 2024-03-08T06:00:34.163 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-08T06:00:34.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.269+0000 7f2b6bfe2640 1 Processor -- start 2024-03-08T06:00:34.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.270+0000 7f2b6bfe2640 1 -- start start 2024-03-08T06:00:34.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.270+0000 7f2b6bfe2640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b640d07a0 0x7f2b640cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:34.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.270+0000 7f2b6bfe2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b640ccaa0 con 0x7f2b6409a2c0 2024-03-08T06:00:34.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.270+0000 7f2b6bfe2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b640ccd80 con 0x7f2b6415bba0 2024-03-08T06:00:34.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.270+0000 7f2b6bfe2640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b640cd3d0 con 0x7f2b640d07a0 2024-03-08T06:00:34.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b6957d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2b6409a2c0 0x7f2b640d0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47016/0 (socket says 172.21.15.76:47016) 2024-03-08T06:00:34.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b6957d640 1 -- 172.21.15.76:0/1939895494 learned_addr learned my addr 172.21.15.76:0/1939895494 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:34.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b6a57f640 1 --2- 172.21.15.76:0/1939895494 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b640d07a0 0x7f2b640cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:34.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 -- 172.21.15.76:0/1939895494 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922610887 0 0) 0x7f2b640ccaa0 con 0x7f2b6409a2c0 2024-03-08T06:00:34.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 -- 172.21.15.76:0/1939895494 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b50003520 con 0x7f2b6409a2c0 2024-03-08T06:00:34.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 -- 172.21.15.76:0/1939895494 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1992794291 0 0) 0x7f2b640ccd80 con 0x7f2b6415bba0 2024-03-08T06:00:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 -- 172.21.15.76:0/1939895494 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b640ccaa0 con 0x7f2b6415bba0 2024-03-08T06:00:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 -- 172.21.15.76:0/1939895494 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3622595986 0 0) 0x7f2b50003520 con 0x7f2b6409a2c0 2024-03-08T06:00:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 -- 172.21.15.76:0/1939895494 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b640d07a0 msgr2=0x7f2b640cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 --2- 172.21.15.76:0/1939895494 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b640d07a0 0x7f2b640cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 -- 172.21.15.76:0/1939895494 >> v1:172.21.15.81:6789/0 conn(0x7f2b6415bba0 legacy=0x7f2b64099bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 -- 172.21.15.76:0/1939895494 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b640cd060 con 0x7f2b6409a2c0 2024-03-08T06:00:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 -- 172.21.15.76:0/1939895494 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f2b54001670 con 0x7f2b6409a2c0 2024-03-08T06:00:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 -- 172.21.15.76:0/1939895494 >> v1:172.21.15.76:6789/0 conn(0x7f2b6409a2c0 legacy=0x7f2b640d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 --2- 172.21.15.76:0/1939895494 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b640d07a0 0x7f2b50003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 --2- 172.21.15.76:0/1939895494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b50004120 0x7f2b50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 --2- 172.21.15.76:0/1939895494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b50004d40 0x7f2b500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 -- 172.21.15.76:0/1939895494 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b640cd3d0 con 0x7f2b50004d40 2024-03-08T06:00:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 -- 172.21.15.76:0/1939895494 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5000bb00 con 0x7f2b50004120 2024-03-08T06:00:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b68d7c640 1 -- 172.21.15.76:0/1939895494 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5000bd00 con 0x7f2b640d07a0 2024-03-08T06:00:34.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b6a57f640 1 --2- 172.21.15.76:0/1939895494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b50004d40 0x7f2b500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:34.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b69d7e640 1 --2- 172.21.15.76:0/1939895494 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b640d07a0 0x7f2b50003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:34.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.271+0000 7f2b6957d640 1 --2- 172.21.15.76:0/1939895494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b50004120 0x7f2b50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:34.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.272+0000 7f2b6a57f640 1 -- 172.21.15.76:0/1939895494 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b640d07a0 msgr2=0x7f2b50003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:34.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.272+0000 7f2b6a57f640 1 --2- 172.21.15.76:0/1939895494 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b640d07a0 0x7f2b50003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:34.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.272+0000 7f2b6a57f640 1 -- 172.21.15.76:0/1939895494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b50004120 msgr2=0x7f2b50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:34.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.272+0000 7f2b6a57f640 1 --2- 172.21.15.76:0/1939895494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b50004120 0x7f2b50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:34.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.272+0000 7f2b6a57f640 1 -- 172.21.15.76:0/1939895494 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b50003520 con 0x7f2b50004d40 2024-03-08T06:00:34.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.272+0000 7f2b6a57f640 1 --2- 172.21.15.76:0/1939895494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b50004d40 0x7f2b500051c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009f90 tx=0x7f2b50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5b9e5212806d94 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:34.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.272+0000 7f2b68d7c640 1 -- 172.21.15.76:0/1939895494 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c039070 con 0x7f2b50004d40 2024-03-08T06:00:34.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.272+0000 7f2b68d7c640 1 -- 172.21.15.76:0/1939895494 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5c02cbd0 con 0x7f2b50004d40 2024-03-08T06:00:34.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.272+0000 7f2b68d7c640 1 -- 172.21.15.76:0/1939895494 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c009c10 con 0x7f2b50004d40 2024-03-08T06:00:34.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.273+0000 7f2b6bfe2640 1 -- 172.21.15.76:0/1939895494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b50004d40 msgr2=0x7f2b500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:34.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.273+0000 7f2b6bfe2640 1 --2- 172.21.15.76:0/1939895494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b50004d40 0x7f2b500051c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009f90 tx=0x7f2b50005830 comp rx=0 tx=0).stop 2024-03-08T06:00:34.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.273+0000 7f2b6a57f640 1 -- 172.21.15.76:0/1939895494 reap_dead start 2024-03-08T06:00:34.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.273+0000 7f2b6bfe2640 1 -- 172.21.15.76:0/1939895494 shutdown_connections 2024-03-08T06:00:34.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.273+0000 7f2b6bfe2640 1 -- 172.21.15.76:0/1939895494 >> 172.21.15.76:0/1939895494 conn(0x7f2b640b35a0 msgr2=0x7f2b640b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:34.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.273+0000 7f2b6bfe2640 1 -- 172.21.15.76:0/1939895494 shutdown_connections 2024-03-08T06:00:34.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.273+0000 7f2b6bfe2640 1 -- 172.21.15.76:0/1939895494 wait complete. 2024-03-08T06:00:34.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.274+0000 7f2b6bfe2640 1 Processor -- start 2024-03-08T06:00:34.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.274+0000 7f2b6bfe2640 1 -- start start 2024-03-08T06:00:34.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b6bfe2640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b50004d40 0x7f2b641f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:34.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b6bfe2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b641f7330 0x7f2b640c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:34.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b6bfe2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b640c9ad0 0x7f2b641e11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:34.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b6bfe2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b640cc840 con 0x7f2b640c9ad0 2024-03-08T06:00:34.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b6bfe2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b640cc6c0 con 0x7f2b641f7330 2024-03-08T06:00:34.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b6bfe2640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b640cd860 con 0x7f2b50004d40 2024-03-08T06:00:34.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b6a57f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b640c9ad0 0x7f2b641e11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:34.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b6a57f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b640c9ad0 0x7f2b641e11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49476/0 (socket says 172.21.15.76:49476) 2024-03-08T06:00:34.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b6a57f640 1 -- 172.21.15.76:0/2700143939 learned_addr learned my addr 172.21.15.76:0/2700143939 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:34.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b6957d640 1 --2- 172.21.15.76:0/2700143939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b641f7330 0x7f2b640c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:34.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b69d7e640 1 --2- 172.21.15.76:0/2700143939 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b50004d40 0x7f2b641f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:34.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b6a57f640 1 -- 172.21.15.76:0/2700143939 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b50004d40 msgr2=0x7f2b641f6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:34.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b6a57f640 1 --2- 172.21.15.76:0/2700143939 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b50004d40 0x7f2b641f6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:34.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b6a57f640 1 -- 172.21.15.76:0/2700143939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b641f7330 msgr2=0x7f2b640c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:34.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b6a57f640 1 --2- 172.21.15.76:0/2700143939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b641f7330 0x7f2b640c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:34.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b6a57f640 1 -- 172.21.15.76:0/2700143939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b640ca170 con 0x7f2b640c9ad0 2024-03-08T06:00:34.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.275+0000 7f2b6a57f640 1 --2- 172.21.15.76:0/2700143939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b640c9ad0 0x7f2b641e11f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c037040 tx=0x7f2b5c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:34.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.276+0000 7f2b5a7fc640 1 -- 172.21.15.76:0/2700143939 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c039030 con 0x7f2b640c9ad0 2024-03-08T06:00:34.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.276+0000 7f2b5a7fc640 1 -- 172.21.15.76:0/2700143939 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5c043070 con 0x7f2b640c9ad0 2024-03-08T06:00:34.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.276+0000 7f2b6bfe2640 1 -- 172.21.15.76:0/2700143939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b640ca450 con 0x7f2b640c9ad0 2024-03-08T06:00:34.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.276+0000 7f2b5a7fc640 1 -- 172.21.15.76:0/2700143939 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c03d7d0 con 0x7f2b640c9ad0 2024-03-08T06:00:34.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.276+0000 7f2b6bfe2640 1 -- 172.21.15.76:0/2700143939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b640caa10 con 0x7f2b640c9ad0 2024-03-08T06:00:34.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.277+0000 7f2b6bfe2640 1 -- 172.21.15.76:0/2700143939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b640cdbf0 con 0x7f2b640c9ad0 2024-03-08T06:00:34.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.277+0000 7f2b5a7fc640 1 -- 172.21.15.76:0/2700143939 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b5c005d10 con 0x7f2b640c9ad0 2024-03-08T06:00:34.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.277+0000 7f2b5a7fc640 1 --2- 172.21.15.76:0/2700143939 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2b4806b500 0x7f2b4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:34.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.278+0000 7f2b5a7fc640 1 -- 172.21.15.76:0/2700143939 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2b5c0ac300 con 0x7f2b640c9ad0 2024-03-08T06:00:34.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.278+0000 7f2b69d7e640 1 --2- 172.21.15.76:0/2700143939 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2b4806b500 0x7f2b4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:34.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.278+0000 7f2b69d7e640 1 --2- 172.21.15.76:0/2700143939 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2b4806b500 0x7f2b4806d9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b600036b0 tx=0x7f2b60009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:34.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.282+0000 7f2b5a7fc640 1 -- 172.21.15.76:0/2700143939 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b640cdbf0 con 0x7f2b640c9ad0 2024-03-08T06:00:34.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:34.455+0000 7f2b6bfe2640 1 -- 172.21.15.76:0/2700143939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f2b6406a1d0 con 0x7f2b640c9ad0 2024-03-08T06:00:35.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.148+0000 7f2b5a7fc640 1 -- 172.21.15.76:0/2700143939 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7f2b5c07b350 con 0x7f2b640c9ad0 2024-03-08T06:00:35.149 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2024-03-08T06:00:35.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.150+0000 7f2b6bfe2640 1 -- 172.21.15.76:0/2700143939 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2b4806b500 msgr2=0x7f2b4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:35.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.150+0000 7f2b6bfe2640 1 --2- 172.21.15.76:0/2700143939 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2b4806b500 0x7f2b4806d9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b600036b0 tx=0x7f2b60009040 comp rx=0 tx=0).stop 2024-03-08T06:00:35.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.150+0000 7f2b6bfe2640 1 -- 172.21.15.76:0/2700143939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b640c9ad0 msgr2=0x7f2b641e11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:35.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.150+0000 7f2b6bfe2640 1 --2- 172.21.15.76:0/2700143939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b640c9ad0 0x7f2b641e11f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c037040 tx=0x7f2b5c009750 comp rx=0 tx=0).stop 2024-03-08T06:00:35.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.150+0000 7f2b6bfe2640 1 -- 172.21.15.76:0/2700143939 shutdown_connections 2024-03-08T06:00:35.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.150+0000 7f2b6bfe2640 1 --2- 172.21.15.76:0/2700143939 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2b4806b500 0x7f2b4806d9c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:35.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.150+0000 7f2b6bfe2640 1 --2- 172.21.15.76:0/2700143939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b640c9ad0 0x7f2b641e11f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:35.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.150+0000 7f2b6bfe2640 1 --2- 172.21.15.76:0/2700143939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b641f7330 0x7f2b640c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:35.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.150+0000 7f2b6bfe2640 1 --2- 172.21.15.76:0/2700143939 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b50004d40 0x7f2b641f6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:35.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.150+0000 7f2b6bfe2640 1 -- 172.21.15.76:0/2700143939 >> 172.21.15.76:0/2700143939 conn(0x7f2b640b35a0 msgr2=0x7f2b64070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:35.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.151+0000 7f2b6bfe2640 1 -- 172.21.15.76:0/2700143939 shutdown_connections 2024-03-08T06:00:35.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.151+0000 7f2b6bfe2640 1 -- 172.21.15.76:0/2700143939 wait complete. 2024-03-08T06:00:35.172 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-08T06:00:35.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.280+0000 7fe135cf4640 1 Processor -- start 2024-03-08T06:00:35.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.280+0000 7fe135cf4640 1 -- start start 2024-03-08T06:00:35.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.280+0000 7fe135cf4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe13015dbc0 0x7fe1300d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:35.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.280+0000 7fe135cf4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe1300b5cf0 0x7fe1300b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:35.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.280+0000 7fe135cf4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1300d27d0 con 0x7fe1300b9840 2024-03-08T06:00:35.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.281+0000 7fe135cf4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1300d2ab0 con 0x7fe1300b5cf0 2024-03-08T06:00:35.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.281+0000 7fe135cf4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1300d2d90 con 0x7fe13015dbc0 2024-03-08T06:00:35.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.281+0000 7fe12effd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe1300b9840 0x7fe1300b55e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47026/0 (socket says 172.21.15.76:47026) 2024-03-08T06:00:35.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.281+0000 7fe12effd640 1 -- 172.21.15.76:0/716358605 learned_addr learned my addr 172.21.15.76:0/716358605 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:35.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.281+0000 7fe12ffff640 1 --2- 172.21.15.76:0/716358605 >> v2:172.21.15.81:3300/0 conn(0x7fe1300b5cf0 0x7fe1300b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:35.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.281+0000 7fe12f7fe640 1 --2- 172.21.15.76:0/716358605 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe13015dbc0 0x7fe1300d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:35.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.281+0000 7fe12e7fc640 1 -- 172.21.15.76:0/716358605 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3835350924 0 0) 0x7fe1300d27d0 con 0x7fe1300b9840 2024-03-08T06:00:35.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.281+0000 7fe12e7fc640 1 -- 172.21.15.76:0/716358605 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe118003520 con 0x7fe1300b9840 2024-03-08T06:00:35.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.281+0000 7fe12f7fe640 1 -- 172.21.15.76:0/716358605 >> v2:172.21.15.81:3300/0 conn(0x7fe1300b5cf0 msgr2=0x7fe1300b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:35.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.281+0000 7fe12e7fc640 1 -- 172.21.15.76:0/716358605 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2887839621 0 0) 0x7fe118003520 con 0x7fe1300b9840 2024-03-08T06:00:35.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.281+0000 7fe12f7fe640 1 --2- 172.21.15.76:0/716358605 >> v2:172.21.15.81:3300/0 conn(0x7fe1300b5cf0 0x7fe1300b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:35.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.281+0000 7fe12f7fe640 1 -- 172.21.15.76:0/716358605 >> v1:172.21.15.76:6789/0 conn(0x7fe1300b9840 legacy=0x7fe1300b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:35.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.281+0000 7fe12f7fe640 1 -- 172.21.15.76:0/716358605 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1300d3070 con 0x7fe13015dbc0 2024-03-08T06:00:35.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.281+0000 7fe12ffff640 1 --2- 172.21.15.76:0/716358605 >> v2:172.21.15.81:3300/0 conn(0x7fe1300b5cf0 0x7fe1300b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:00:35.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.282+0000 7fe12f7fe640 1 --2- 172.21.15.76:0/716358605 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe13015dbc0 0x7fe1300d1ef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe1200097a0 tx=0x7fe12002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf48f20d1ef5b1ce server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:35.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.282+0000 7fe12e7fc640 1 -- 172.21.15.76:0/716358605 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe120037070 con 0x7fe13015dbc0 2024-03-08T06:00:35.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.282+0000 7fe12e7fc640 1 -- 172.21.15.76:0/716358605 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe120030510 con 0x7fe13015dbc0 2024-03-08T06:00:35.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.282+0000 7fe12e7fc640 1 -- 172.21.15.76:0/716358605 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe124001670 con 0x7fe13015dbc0 2024-03-08T06:00:35.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.282+0000 7fe135cf4640 1 -- 172.21.15.76:0/716358605 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe13015dbc0 msgr2=0x7fe1300d1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:35.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.282+0000 7fe135cf4640 1 --2- 172.21.15.76:0/716358605 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe13015dbc0 0x7fe1300d1ef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe1200097a0 tx=0x7fe12002b790 comp rx=0 tx=0).stop 2024-03-08T06:00:35.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.283+0000 7fe135cf4640 1 -- 172.21.15.76:0/716358605 shutdown_connections 2024-03-08T06:00:35.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.283+0000 7fe135cf4640 1 --2- 172.21.15.76:0/716358605 >> v2:172.21.15.81:3300/0 conn(0x7fe1300b5cf0 0x7fe1300b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:35.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.283+0000 7fe135cf4640 1 --2- 172.21.15.76:0/716358605 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe13015dbc0 0x7fe1300d1ef0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:35.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.283+0000 7fe135cf4640 1 -- 172.21.15.76:0/716358605 >> 172.21.15.76:0/716358605 conn(0x7fe1300670d0 msgr2=0x7fe1300d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:35.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.283+0000 7fe135cf4640 1 -- 172.21.15.76:0/716358605 shutdown_connections 2024-03-08T06:00:35.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.283+0000 7fe135cf4640 1 -- 172.21.15.76:0/716358605 wait complete. 2024-03-08T06:00:35.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.284+0000 7fe135cf4640 1 Processor -- start 2024-03-08T06:00:35.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.284+0000 7fe135cf4640 1 -- start start 2024-03-08T06:00:35.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.284+0000 7fe135cf4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1300b5cf0 0x7fe13015ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:35.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe135cf4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1300b9840 0x7fe13015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:35.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe135cf4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe13015dbc0 0x7fe1300c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:35.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe135cf4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1300d25b0 con 0x7fe1300b5cf0 2024-03-08T06:00:35.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe135cf4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1300d2430 con 0x7fe13015dbc0 2024-03-08T06:00:35.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe135cf4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13015dfa0 con 0x7fe1300b9840 2024-03-08T06:00:35.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe12f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1300b5cf0 0x7fe13015ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:35.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe12f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1300b5cf0 0x7fe13015ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49482/0 (socket says 172.21.15.76:49482) 2024-03-08T06:00:35.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe12f7fe640 1 -- 172.21.15.76:0/824598596 learned_addr learned my addr 172.21.15.76:0/824598596 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe12effd640 1 --2- 172.21.15.76:0/824598596 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1300b9840 0x7fe13015a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe12ffff640 1 --2- 172.21.15.76:0/824598596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe13015dbc0 0x7fe1300c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe12f7fe640 1 -- 172.21.15.76:0/824598596 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1300b9840 msgr2=0x7fe13015a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe12f7fe640 1 --2- 172.21.15.76:0/824598596 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1300b9840 0x7fe13015a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe12f7fe640 1 -- 172.21.15.76:0/824598596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe13015dbc0 msgr2=0x7fe1300c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe12f7fe640 1 --2- 172.21.15.76:0/824598596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe13015dbc0 0x7fe1300c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe12f7fe640 1 -- 172.21.15.76:0/824598596 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1301e5480 con 0x7fe1300b5cf0 2024-03-08T06:00:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe12f7fe640 1 --2- 172.21.15.76:0/824598596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1300b5cf0 0x7fe13015ccd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe12002b7d0 tx=0x7fe1200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe117fff640 1 -- 172.21.15.76:0/824598596 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe120041020 con 0x7fe1300b5cf0 2024-03-08T06:00:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe117fff640 1 -- 172.21.15.76:0/824598596 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe120032700 con 0x7fe1300b5cf0 2024-03-08T06:00:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe135cf4640 1 -- 172.21.15.76:0/824598596 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1301e5760 con 0x7fe1300b5cf0 2024-03-08T06:00:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe117fff640 1 -- 172.21.15.76:0/824598596 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe120037040 con 0x7fe1300b5cf0 2024-03-08T06:00:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.285+0000 7fe135cf4640 1 -- 172.21.15.76:0/824598596 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1301e5d20 con 0x7fe1300b5cf0 2024-03-08T06:00:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.286+0000 7fe135cf4640 1 -- 172.21.15.76:0/824598596 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13004f030 con 0x7fe1300b5cf0 2024-03-08T06:00:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.287+0000 7fe117fff640 1 -- 172.21.15.76:0/824598596 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe120044020 con 0x7fe1300b5cf0 2024-03-08T06:00:35.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.287+0000 7fe117fff640 1 --2- 172.21.15.76:0/824598596 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe11c06b4d0 0x7fe11c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:35.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.287+0000 7fe117fff640 1 -- 172.21.15.76:0/824598596 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe1200ab570 con 0x7fe1300b5cf0 2024-03-08T06:00:35.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.287+0000 7fe12effd640 1 --2- 172.21.15.76:0/824598596 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe11c06b4d0 0x7fe11c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:35.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.288+0000 7fe12effd640 1 --2- 172.21.15.76:0/824598596 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe11c06b4d0 0x7fe11c06d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe1240013c0 tx=0x7fe124001350 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:35.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.292+0000 7fe117fff640 1 -- 172.21.15.76:0/824598596 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe12004b070 con 0x7fe1300b5cf0 2024-03-08T06:00:35.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:35.502+0000 7fe135cf4640 1 -- 172.21.15.76:0/824598596 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe1300b73b0 con 0x7fe1300b5cf0 2024-03-08T06:00:35.531 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:00:35.533 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:00:36.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.182+0000 7fe117fff640 1 -- 172.21.15.76:0/824598596 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe12007a450 con 0x7fe1300b5cf0 2024-03-08T06:00:36.183 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 2 and data pool 3 2024-03-08T06:00:36.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.186+0000 7fe135cf4640 1 -- 172.21.15.76:0/824598596 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe11c06b4d0 msgr2=0x7fe11c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:36.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.186+0000 7fe135cf4640 1 --2- 172.21.15.76:0/824598596 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe11c06b4d0 0x7fe11c06d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe1240013c0 tx=0x7fe124001350 comp rx=0 tx=0).stop 2024-03-08T06:00:36.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.186+0000 7fe135cf4640 1 -- 172.21.15.76:0/824598596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1300b5cf0 msgr2=0x7fe13015ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:36.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.186+0000 7fe135cf4640 1 --2- 172.21.15.76:0/824598596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1300b5cf0 0x7fe13015ccd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe12002b7d0 tx=0x7fe1200046c0 comp rx=0 tx=0).stop 2024-03-08T06:00:36.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.186+0000 7fe135cf4640 1 -- 172.21.15.76:0/824598596 shutdown_connections 2024-03-08T06:00:36.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.186+0000 7fe135cf4640 1 --2- 172.21.15.76:0/824598596 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe11c06b4d0 0x7fe11c06d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:36.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.186+0000 7fe135cf4640 1 --2- 172.21.15.76:0/824598596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe13015dbc0 0x7fe1300c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:36.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.186+0000 7fe135cf4640 1 --2- 172.21.15.76:0/824598596 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1300b9840 0x7fe13015a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:36.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.186+0000 7fe135cf4640 1 --2- 172.21.15.76:0/824598596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1300b5cf0 0x7fe13015ccd0 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe12002b7d0 tx=0x7fe1200046c0 comp rx=0 tx=0).stop 2024-03-08T06:00:36.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.186+0000 7fe135cf4640 1 -- 172.21.15.76:0/824598596 >> 172.21.15.76:0/824598596 conn(0x7fe1300670d0 msgr2=0x7fe1300c7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:36.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.188+0000 7fe135cf4640 1 -- 172.21.15.76:0/824598596 shutdown_connections 2024-03-08T06:00:36.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.188+0000 7fe135cf4640 1 -- 172.21.15.76:0/824598596 wait complete. 2024-03-08T06:00:36.207 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-08T06:00:36.207 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-08T06:00:36.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.314+0000 7f0161383640 1 Processor -- start 2024-03-08T06:00:36.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.314+0000 7f0161383640 1 -- start start 2024-03-08T06:00:36.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.314+0000 7f0161383640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f015c15ddd0 0x7f015c0ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:36.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.314+0000 7f0161383640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015c15e1b0 con 0x7f015c0cf080 2024-03-08T06:00:36.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.314+0000 7f0161383640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015c0d2cd0 con 0x7f015c0ce540 2024-03-08T06:00:36.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.314+0000 7f0161383640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015c0d2a80 con 0x7f015c15ddd0 2024-03-08T06:00:36.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.315+0000 7f015b7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f015c0cf080 0x7f015c0b6df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47030/0 (socket says 172.21.15.76:47030) 2024-03-08T06:00:36.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.315+0000 7f015b7fe640 1 -- 172.21.15.76:0/1383943061 learned_addr learned my addr 172.21.15.76:0/1383943061 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:36.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.315+0000 7f015affd640 1 --2- 172.21.15.76:0/1383943061 >> v2:172.21.15.81:3300/0 conn(0x7f015c15ddd0 0x7f015c0ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:36.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.315+0000 7f015a7fc640 1 -- 172.21.15.76:0/1383943061 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670460460 0 0) 0x7f015c15e1b0 con 0x7f015c0cf080 2024-03-08T06:00:36.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.315+0000 7f015a7fc640 1 -- 172.21.15.76:0/1383943061 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0148003520 con 0x7f015c0cf080 2024-03-08T06:00:36.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.315+0000 7f015affd640 1 -- 172.21.15.76:0/1383943061 >> v1:172.21.15.81:6789/0 conn(0x7f015c0ce540 legacy=0x7f015c0d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:36.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.315+0000 7f015affd640 1 -- 172.21.15.76:0/1383943061 >> v1:172.21.15.76:6789/0 conn(0x7f015c0cf080 legacy=0x7f015c0b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:36.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.315+0000 7f015affd640 1 -- 172.21.15.76:0/1383943061 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c0d2fb0 con 0x7f015c15ddd0 2024-03-08T06:00:36.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.316+0000 7f015affd640 1 --2- 172.21.15.76:0/1383943061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f015c15ddd0 0x7f015c0ceb40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f014c0097a0 tx=0x7f014c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=553e9f14af83191c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:36.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.316+0000 7f015a7fc640 1 -- 172.21.15.76:0/1383943061 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c035020 con 0x7f015c15ddd0 2024-03-08T06:00:36.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.316+0000 7f015a7fc640 1 -- 172.21.15.76:0/1383943061 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014c031830 con 0x7f015c15ddd0 2024-03-08T06:00:36.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.316+0000 7f015a7fc640 1 -- 172.21.15.76:0/1383943061 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c031c00 con 0x7f015c15ddd0 2024-03-08T06:00:36.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.316+0000 7f0161383640 1 -- 172.21.15.76:0/1383943061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f015c15ddd0 msgr2=0x7f015c0ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:36.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.316+0000 7f0161383640 1 --2- 172.21.15.76:0/1383943061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f015c15ddd0 0x7f015c0ceb40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f014c0097a0 tx=0x7f014c02b8d0 comp rx=0 tx=0).stop 2024-03-08T06:00:36.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.317+0000 7f0161383640 1 -- 172.21.15.76:0/1383943061 shutdown_connections 2024-03-08T06:00:36.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.317+0000 7f0161383640 1 --2- 172.21.15.76:0/1383943061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f015c15ddd0 0x7f015c0ceb40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:36.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.317+0000 7f0161383640 1 -- 172.21.15.76:0/1383943061 >> 172.21.15.76:0/1383943061 conn(0x7f015c066fc0 msgr2=0x7f015c0b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:36.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.317+0000 7f0161383640 1 -- 172.21.15.76:0/1383943061 shutdown_connections 2024-03-08T06:00:36.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.317+0000 7f0161383640 1 -- 172.21.15.76:0/1383943061 wait complete. 2024-03-08T06:00:36.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.318+0000 7f0161383640 1 Processor -- start 2024-03-08T06:00:36.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.318+0000 7f0161383640 1 -- start start 2024-03-08T06:00:36.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.318+0000 7f0161383640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f015c0ce540 0x7f015c0c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:36.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.318+0000 7f0161383640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f015c0cf080 0x7f015c148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:36.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.318+0000 7f0161383640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15ddd0 0x7f015c156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:36.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.318+0000 7f0161383640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c0d0c10 con 0x7f015c15ddd0 2024-03-08T06:00:36.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.318+0000 7f0161383640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c0d3570 con 0x7f015c0cf080 2024-03-08T06:00:36.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.318+0000 7f0161383640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c0d0d90 con 0x7f015c0ce540 2024-03-08T06:00:36.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.318+0000 7f015b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15ddd0 0x7f015c156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:36.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.318+0000 7f015b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15ddd0 0x7f015c156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49486/0 (socket says 172.21.15.76:49486) 2024-03-08T06:00:36.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.319+0000 7f015b7fe640 1 -- 172.21.15.76:0/4284820186 learned_addr learned my addr 172.21.15.76:0/4284820186 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:36.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.319+0000 7f015affd640 1 --2- 172.21.15.76:0/4284820186 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f015c0ce540 0x7f015c0c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:36.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.319+0000 7f0153fff640 1 --2- 172.21.15.76:0/4284820186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f015c0cf080 0x7f015c148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:36.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.319+0000 7f015b7fe640 1 -- 172.21.15.76:0/4284820186 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f015c0ce540 msgr2=0x7f015c0c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:36.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.319+0000 7f015b7fe640 1 --2- 172.21.15.76:0/4284820186 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f015c0ce540 0x7f015c0c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:36.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.319+0000 7f015b7fe640 1 -- 172.21.15.76:0/4284820186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f015c0cf080 msgr2=0x7f015c148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:36.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.319+0000 7f015b7fe640 1 --2- 172.21.15.76:0/4284820186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f015c0cf080 0x7f015c148d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:36.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.319+0000 7f015b7fe640 1 -- 172.21.15.76:0/4284820186 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c1492a0 con 0x7f015c15ddd0 2024-03-08T06:00:36.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.319+0000 7f015affd640 1 --2- 172.21.15.76:0/4284820186 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f015c0ce540 0x7f015c0c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:00:36.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.319+0000 7f015b7fe640 1 --2- 172.21.15.76:0/4284820186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15ddd0 0x7f015c156e80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f015400cb40 tx=0x7f015400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:36.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.319+0000 7f01537fe640 1 -- 172.21.15.76:0/4284820186 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154017070 con 0x7f015c15ddd0 2024-03-08T06:00:36.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.319+0000 7f01537fe640 1 -- 172.21.15.76:0/4284820186 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015400aca0 con 0x7f015c15ddd0 2024-03-08T06:00:36.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.319+0000 7f0161383640 1 -- 172.21.15.76:0/4284820186 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015c1578b0 con 0x7f015c15ddd0 2024-03-08T06:00:36.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.319+0000 7f01537fe640 1 -- 172.21.15.76:0/4284820186 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154012620 con 0x7f015c15ddd0 2024-03-08T06:00:36.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.319+0000 7f0161383640 1 -- 172.21.15.76:0/4284820186 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c157e70 con 0x7f015c15ddd0 2024-03-08T06:00:36.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.320+0000 7f0161383640 1 -- 172.21.15.76:0/4284820186 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015c04f030 con 0x7f015c15ddd0 2024-03-08T06:00:36.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.321+0000 7f01537fe640 1 -- 172.21.15.76:0/4284820186 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f015401a070 con 0x7f015c15ddd0 2024-03-08T06:00:36.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.321+0000 7f01537fe640 1 --2- 172.21.15.76:0/4284820186 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f013c06b4d0 0x7f013c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:36.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.321+0000 7f01537fe640 1 -- 172.21.15.76:0/4284820186 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0154089fb0 con 0x7f015c15ddd0 2024-03-08T06:00:36.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.321+0000 7f015affd640 1 --2- 172.21.15.76:0/4284820186 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f013c06b4d0 0x7f013c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:36.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.321+0000 7f015affd640 1 --2- 172.21.15.76:0/4284820186 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f013c06b4d0 0x7f013c06d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f014c036000 tx=0x7f014c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:36.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.325+0000 7f01537fe640 1 -- 172.21.15.76:0/4284820186 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f01540587e0 con 0x7f015c15ddd0 2024-03-08T06:00:36.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:36.503+0000 7f0161383640 1 -- 172.21.15.76:0/4284820186 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f015c15d1a0 con 0x7f015c15ddd0 2024-03-08T06:00:37.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.180+0000 7f01537fe640 1 -- 172.21.15.76:0/4284820186 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7f015405c880 con 0x7f015c15ddd0 2024-03-08T06:00:37.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.184+0000 7f0161383640 1 -- 172.21.15.76:0/4284820186 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f013c06b4d0 msgr2=0x7f013c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:37.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.184+0000 7f0161383640 1 --2- 172.21.15.76:0/4284820186 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f013c06b4d0 0x7f013c06d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f014c036000 tx=0x7f014c008030 comp rx=0 tx=0).stop 2024-03-08T06:00:37.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.184+0000 7f0161383640 1 -- 172.21.15.76:0/4284820186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15ddd0 msgr2=0x7f015c156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:37.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.184+0000 7f0161383640 1 --2- 172.21.15.76:0/4284820186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15ddd0 0x7f015c156e80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f015400cb40 tx=0x7f015400a060 comp rx=0 tx=0).stop 2024-03-08T06:00:37.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.185+0000 7f0161383640 1 -- 172.21.15.76:0/4284820186 shutdown_connections 2024-03-08T06:00:37.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.185+0000 7f0161383640 1 --2- 172.21.15.76:0/4284820186 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f013c06b4d0 0x7f013c06d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:37.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.185+0000 7f0161383640 1 --2- 172.21.15.76:0/4284820186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15ddd0 0x7f015c156e80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:37.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.185+0000 7f0161383640 1 --2- 172.21.15.76:0/4284820186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f015c0cf080 0x7f015c148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:37.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.185+0000 7f0161383640 1 --2- 172.21.15.76:0/4284820186 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f015c0ce540 0x7f015c0c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:37.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.185+0000 7f0161383640 1 -- 172.21.15.76:0/4284820186 >> 172.21.15.76:0/4284820186 conn(0x7f015c066fc0 msgr2=0x7f015c15c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:37.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.185+0000 7f0161383640 1 -- 172.21.15.76:0/4284820186 shutdown_connections 2024-03-08T06:00:37.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.185+0000 7f0161383640 1 -- 172.21.15.76:0/4284820186 wait complete. 2024-03-08T06:00:37.198 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-08T06:00:37.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.310+0000 7f6a5c66b640 1 Processor -- start 2024-03-08T06:00:37.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.310+0000 7f6a5c66b640 1 -- start start 2024-03-08T06:00:37.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.311+0000 7f6a5c66b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6a540ce5f0 0x7f6a5406c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:37.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.311+0000 7f6a5c66b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a54070180 0x7f6a5406cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:37.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.311+0000 7f6a5c66b640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6a540ce9d0 0x7f6a5406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:37.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.311+0000 7f6a5c66b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a540d0510 con 0x7f6a54070180 2024-03-08T06:00:37.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.311+0000 7f6a5c66b640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a540709e0 con 0x7f6a540ce5f0 2024-03-08T06:00:37.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.311+0000 7f6a5c66b640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540d0230 con 0x7f6a540ce9d0 2024-03-08T06:00:37.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.311+0000 7f6a59c06640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a54070180 0x7f6a5406cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:37.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.311+0000 7f6a59c06640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a54070180 0x7f6a5406cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49488/0 (socket says 172.21.15.76:49488) 2024-03-08T06:00:37.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.311+0000 7f6a59c06640 1 -- 172.21.15.76:0/3505279084 learned_addr learned my addr 172.21.15.76:0/3505279084 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:37.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.312+0000 7f6a5a407640 1 --2- 172.21.15.76:0/3505279084 >> v2:172.21.15.81:3300/0 conn(0x7f6a540ce5f0 0x7f6a5406c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:37.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.312+0000 7f6a5ac08640 1 --2- 172.21.15.76:0/3505279084 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6a540ce9d0 0x7f6a5406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:37.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.312+0000 7f6a59c06640 1 -- 172.21.15.76:0/3505279084 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6a540ce9d0 msgr2=0x7f6a5406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:37.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.312+0000 7f6a59c06640 1 --2- 172.21.15.76:0/3505279084 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6a540ce9d0 0x7f6a5406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:37.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.312+0000 7f6a59c06640 1 -- 172.21.15.76:0/3505279084 >> v2:172.21.15.81:3300/0 conn(0x7f6a540ce5f0 msgr2=0x7f6a5406c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:37.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.312+0000 7f6a59c06640 1 --2- 172.21.15.76:0/3505279084 >> v2:172.21.15.81:3300/0 conn(0x7f6a540ce5f0 0x7f6a5406c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:37.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.312+0000 7f6a59c06640 1 -- 172.21.15.76:0/3505279084 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5406bec0 con 0x7f6a54070180 2024-03-08T06:00:37.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.312+0000 7f6a59c06640 1 --2- 172.21.15.76:0/3505279084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a54070180 0x7f6a5406cee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6a5404ffe0 tx=0x7f6a4402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267cc2a2f7aa56e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:37.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.313+0000 7f6a59405640 1 -- 172.21.15.76:0/3505279084 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a44035020 con 0x7f6a54070180 2024-03-08T06:00:37.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.313+0000 7f6a59405640 1 -- 172.21.15.76:0/3505279084 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a44031870 con 0x7f6a54070180 2024-03-08T06:00:37.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.313+0000 7f6a59405640 1 -- 172.21.15.76:0/3505279084 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a44031c40 con 0x7f6a54070180 2024-03-08T06:00:37.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.313+0000 7f6a5c66b640 1 -- 172.21.15.76:0/3505279084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a54070180 msgr2=0x7f6a5406cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:37.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.313+0000 7f6a5c66b640 1 --2- 172.21.15.76:0/3505279084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a54070180 0x7f6a5406cee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6a5404ffe0 tx=0x7f6a4402b910 comp rx=0 tx=0).stop 2024-03-08T06:00:37.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.313+0000 7f6a5c66b640 1 -- 172.21.15.76:0/3505279084 shutdown_connections 2024-03-08T06:00:37.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.313+0000 7f6a5c66b640 1 --2- 172.21.15.76:0/3505279084 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6a540ce9d0 0x7f6a5406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:37.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.313+0000 7f6a5c66b640 1 --2- 172.21.15.76:0/3505279084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a54070180 0x7f6a5406cee0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:37.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.313+0000 7f6a5c66b640 1 --2- 172.21.15.76:0/3505279084 >> v2:172.21.15.81:3300/0 conn(0x7f6a540ce5f0 0x7f6a5406c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:37.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.313+0000 7f6a5c66b640 1 -- 172.21.15.76:0/3505279084 >> 172.21.15.76:0/3505279084 conn(0x7f6a5406f600 msgr2=0x7f6a5406f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:37.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.313+0000 7f6a5c66b640 1 -- 172.21.15.76:0/3505279084 shutdown_connections 2024-03-08T06:00:37.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.313+0000 7f6a5c66b640 1 -- 172.21.15.76:0/3505279084 wait complete. 2024-03-08T06:00:37.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.314+0000 7f6a5c66b640 1 Processor -- start 2024-03-08T06:00:37.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.314+0000 7f6a5c66b640 1 -- start start 2024-03-08T06:00:37.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.315+0000 7f6a5c66b640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6a54070180 0x7f6a540c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:37.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.315+0000 7f6a5c66b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6a540ce5f0 0x7f6a541561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:37.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.315+0000 7f6a5c66b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a540ce9d0 0x7f6a54156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:37.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.315+0000 7f6a5c66b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540cff00 con 0x7f6a540ce9d0 2024-03-08T06:00:37.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.315+0000 7f6a5c66b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5406c480 con 0x7f6a540ce5f0 2024-03-08T06:00:37.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.315+0000 7f6a5c66b640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540cc1e0 con 0x7f6a54070180 2024-03-08T06:00:37.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.315+0000 7f6a5ac08640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a540ce9d0 0x7f6a54156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:37.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.315+0000 7f6a5ac08640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a540ce9d0 0x7f6a54156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49502/0 (socket says 172.21.15.76:49502) 2024-03-08T06:00:37.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.315+0000 7f6a5ac08640 1 -- 172.21.15.76:0/2299646673 learned_addr learned my addr 172.21.15.76:0/2299646673 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:37.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.315+0000 7f6a59c06640 1 --2- 172.21.15.76:0/2299646673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6a540ce5f0 0x7f6a541561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:37.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.315+0000 7f6a5a407640 1 --2- 172.21.15.76:0/2299646673 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6a54070180 0x7f6a540c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:37.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.315+0000 7f6a5ac08640 1 -- 172.21.15.76:0/2299646673 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6a54070180 msgr2=0x7f6a540c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:37.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.315+0000 7f6a5ac08640 1 --2- 172.21.15.76:0/2299646673 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6a54070180 0x7f6a540c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:37.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.315+0000 7f6a5ac08640 1 -- 172.21.15.76:0/2299646673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6a540ce5f0 msgr2=0x7f6a541561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:37.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.315+0000 7f6a5ac08640 1 --2- 172.21.15.76:0/2299646673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6a540ce5f0 0x7f6a541561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:37.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.315+0000 7f6a5ac08640 1 -- 172.21.15.76:0/2299646673 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5414c140 con 0x7f6a540ce9d0 2024-03-08T06:00:37.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.316+0000 7f6a5ac08640 1 --2- 172.21.15.76:0/2299646673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a540ce9d0 0x7f6a54156710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00e7f0 tx=0x7f6a4c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:37.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.316+0000 7f6a4affd640 1 -- 172.21.15.76:0/2299646673 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c017070 con 0x7f6a540ce9d0 2024-03-08T06:00:37.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.316+0000 7f6a4affd640 1 -- 172.21.15.76:0/2299646673 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c00ee40 con 0x7f6a540ce9d0 2024-03-08T06:00:37.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.316+0000 7f6a5c66b640 1 -- 172.21.15.76:0/2299646673 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a540c6310 con 0x7f6a540ce9d0 2024-03-08T06:00:37.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.316+0000 7f6a4affd640 1 -- 172.21.15.76:0/2299646673 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c00cb20 con 0x7f6a540ce9d0 2024-03-08T06:00:37.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.316+0000 7f6a5c66b640 1 -- 172.21.15.76:0/2299646673 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a540c68d0 con 0x7f6a540ce9d0 2024-03-08T06:00:37.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.317+0000 7f6a5c66b640 1 -- 172.21.15.76:0/2299646673 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5404f030 con 0x7f6a540ce9d0 2024-03-08T06:00:37.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.317+0000 7f6a4affd640 1 -- 172.21.15.76:0/2299646673 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a4c01a070 con 0x7f6a540ce9d0 2024-03-08T06:00:37.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.318+0000 7f6a4affd640 1 --2- 172.21.15.76:0/2299646673 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6a3806b4d0 0x7f6a3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:37.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.318+0000 7f6a4affd640 1 -- 172.21.15.76:0/2299646673 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6007+0+0 (secure 0 0 0) 0x7f6a4c08a0c0 con 0x7f6a540ce9d0 2024-03-08T06:00:37.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.318+0000 7f6a5a407640 1 --2- 172.21.15.76:0/2299646673 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6a3806b4d0 0x7f6a3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:37.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.318+0000 7f6a5a407640 1 --2- 172.21.15.76:0/2299646673 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6a3806b4d0 0x7f6a3806d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a50011360 tx=0x7f6a5000d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:37.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.322+0000 7f6a4affd640 1 -- 172.21.15.76:0/2299646673 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a4c0587c0 con 0x7f6a540ce9d0 2024-03-08T06:00:37.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:37.494+0000 7f6a5c66b640 1 -- 172.21.15.76:0/2299646673 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6a540710a0 con 0x7f6a540ce9d0 2024-03-08T06:00:39.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.191+0000 7f6a4affd640 1 -- 172.21.15.76:0/2299646673 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7f6a4c05c860 con 0x7f6a540ce9d0 2024-03-08T06:00:39.192 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data_ec' created 2024-03-08T06:00:39.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.195+0000 7f6a5c66b640 1 -- 172.21.15.76:0/2299646673 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6a3806b4d0 msgr2=0x7f6a3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:39.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.195+0000 7f6a5c66b640 1 --2- 172.21.15.76:0/2299646673 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6a3806b4d0 0x7f6a3806d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a50011360 tx=0x7f6a5000d040 comp rx=0 tx=0).stop 2024-03-08T06:00:39.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.195+0000 7f6a5c66b640 1 -- 172.21.15.76:0/2299646673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a540ce9d0 msgr2=0x7f6a54156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:39.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.195+0000 7f6a5c66b640 1 --2- 172.21.15.76:0/2299646673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a540ce9d0 0x7f6a54156710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00e7f0 tx=0x7f6a4c00ebf0 comp rx=0 tx=0).stop 2024-03-08T06:00:39.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.196+0000 7f6a5c66b640 1 -- 172.21.15.76:0/2299646673 shutdown_connections 2024-03-08T06:00:39.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.196+0000 7f6a5c66b640 1 --2- 172.21.15.76:0/2299646673 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6a3806b4d0 0x7f6a3806d990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:39.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.196+0000 7f6a5c66b640 1 --2- 172.21.15.76:0/2299646673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a540ce9d0 0x7f6a54156710 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:39.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.196+0000 7f6a5c66b640 1 --2- 172.21.15.76:0/2299646673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6a540ce5f0 0x7f6a541561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:39.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.196+0000 7f6a5c66b640 1 --2- 172.21.15.76:0/2299646673 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6a54070180 0x7f6a540c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:39.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.196+0000 7f6a5c66b640 1 -- 172.21.15.76:0/2299646673 >> 172.21.15.76:0/2299646673 conn(0x7f6a5406f600 msgr2=0x7f6a540ccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:39.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.196+0000 7f6a5c66b640 1 -- 172.21.15.76:0/2299646673 shutdown_connections 2024-03-08T06:00:39.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.198+0000 7f6a5c66b640 1 -- 172.21.15.76:0/2299646673 wait complete. 2024-03-08T06:00:39.217 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-08T06:00:39.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.325+0000 7fdb97464640 1 Processor -- start 2024-03-08T06:00:39.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.325+0000 7fdb97464640 1 -- start start 2024-03-08T06:00:39.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.326+0000 7fdb97464640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdb9015ddd0 0x7fdb900ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:39.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.326+0000 7fdb97464640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdb900ce540 0x7fdb900d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:39.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.326+0000 7fdb97464640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9015e1b0 con 0x7fdb900cf080 2024-03-08T06:00:39.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.326+0000 7fdb97464640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb900d2a80 con 0x7fdb900ce540 2024-03-08T06:00:39.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.326+0000 7fdb97464640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb900d2cd0 con 0x7fdb9015ddd0 2024-03-08T06:00:39.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.326+0000 7fdb949ff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdb900ce540 0x7fdb900d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:39.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.326+0000 7fdb949ff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdb900ce540 0x7fdb900d0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34880/0 (socket says 172.21.15.76:34880) 2024-03-08T06:00:39.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.327+0000 7fdb949ff640 1 -- 172.21.15.76:0/381611199 learned_addr learned my addr 172.21.15.76:0/381611199 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:39.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.327+0000 7fdb95200640 1 --2- 172.21.15.76:0/381611199 >> v2:172.21.15.81:3300/0 conn(0x7fdb9015ddd0 0x7fdb900ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:39.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.327+0000 7fdb949ff640 1 -- 172.21.15.76:0/381611199 >> v2:172.21.15.81:3300/0 conn(0x7fdb9015ddd0 msgr2=0x7fdb900ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:39.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.327+0000 7fdb949ff640 1 --2- 172.21.15.76:0/381611199 >> v2:172.21.15.81:3300/0 conn(0x7fdb9015ddd0 0x7fdb900ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:39.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.327+0000 7fdb949ff640 1 -- 172.21.15.76:0/381611199 >> v1:172.21.15.76:6789/0 conn(0x7fdb900cf080 legacy=0x7fdb9006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:39.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.327+0000 7fdb949ff640 1 -- 172.21.15.76:0/381611199 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb900d2fb0 con 0x7fdb900ce540 2024-03-08T06:00:39.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.327+0000 7fdb83fff640 1 -- 172.21.15.76:0/381611199 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 252858126 0 0) 0x7fdb9015e1b0 con 0x7fdb900cf080 2024-03-08T06:00:39.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.328+0000 7fdb949ff640 1 --2- 172.21.15.76:0/381611199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb900ce540 0x7fdb900d0500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdb900b6530 tx=0x7fdb8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5634b9e3665e8549 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:39.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.328+0000 7fdb83fff640 1 -- 172.21.15.76:0/381611199 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84035020 con 0x7fdb900ce540 2024-03-08T06:00:39.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.328+0000 7fdb83fff640 1 -- 172.21.15.76:0/381611199 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb84031870 con 0x7fdb900ce540 2024-03-08T06:00:39.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.328+0000 7fdb83fff640 1 -- 172.21.15.76:0/381611199 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84031c40 con 0x7fdb900ce540 2024-03-08T06:00:39.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.328+0000 7fdb97464640 1 -- 172.21.15.76:0/381611199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb900ce540 msgr2=0x7fdb900d0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:39.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.328+0000 7fdb97464640 1 --2- 172.21.15.76:0/381611199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb900ce540 0x7fdb900d0500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdb900b6530 tx=0x7fdb8402b910 comp rx=0 tx=0).stop 2024-03-08T06:00:39.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.328+0000 7fdb97464640 1 -- 172.21.15.76:0/381611199 shutdown_connections 2024-03-08T06:00:39.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.328+0000 7fdb97464640 1 --2- 172.21.15.76:0/381611199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb900ce540 0x7fdb900d0500 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:39.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.328+0000 7fdb97464640 1 --2- 172.21.15.76:0/381611199 >> v2:172.21.15.81:3300/0 conn(0x7fdb9015ddd0 0x7fdb900ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:39.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.328+0000 7fdb97464640 1 -- 172.21.15.76:0/381611199 >> 172.21.15.76:0/381611199 conn(0x7fdb90066fc0 msgr2=0x7fdb900b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:39.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.329+0000 7fdb97464640 1 -- 172.21.15.76:0/381611199 shutdown_connections 2024-03-08T06:00:39.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.329+0000 7fdb97464640 1 -- 172.21.15.76:0/381611199 wait complete. 2024-03-08T06:00:39.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.330+0000 7fdb97464640 1 Processor -- start 2024-03-08T06:00:39.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.330+0000 7fdb97464640 1 -- start start 2024-03-08T06:00:39.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.330+0000 7fdb97464640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb900ce540 0x7fdb90147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:39.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.330+0000 7fdb97464640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdb900cf080 0x7fdb90154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:39.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.330+0000 7fdb97464640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb9015ddd0 0x7fdb90145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:39.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.330+0000 7fdb97464640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb900b7e00 con 0x7fdb9015ddd0 2024-03-08T06:00:39.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.330+0000 7fdb97464640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb900d3570 con 0x7fdb900ce540 2024-03-08T06:00:39.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.330+0000 7fdb97464640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb900b7f80 con 0x7fdb900cf080 2024-03-08T06:00:39.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.330+0000 7fdb95a01640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb9015ddd0 0x7fdb90145fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:39.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.330+0000 7fdb95a01640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb9015ddd0 0x7fdb90145fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34884/0 (socket says 172.21.15.76:34884) 2024-03-08T06:00:39.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.330+0000 7fdb95a01640 1 -- 172.21.15.76:0/371901318 learned_addr learned my addr 172.21.15.76:0/371901318 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:39.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.331+0000 7fdb95200640 1 --2- 172.21.15.76:0/371901318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb900ce540 0x7fdb90147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:39.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.331+0000 7fdb949ff640 1 --2- 172.21.15.76:0/371901318 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdb900cf080 0x7fdb90154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:39.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.331+0000 7fdb95a01640 1 -- 172.21.15.76:0/371901318 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdb900cf080 msgr2=0x7fdb90154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:39.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.331+0000 7fdb95a01640 1 --2- 172.21.15.76:0/371901318 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdb900cf080 0x7fdb90154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:39.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.331+0000 7fdb95a01640 1 -- 172.21.15.76:0/371901318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb900ce540 msgr2=0x7fdb90147960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:39.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.331+0000 7fdb95a01640 1 --2- 172.21.15.76:0/371901318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb900ce540 0x7fdb90147960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:39.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.331+0000 7fdb95a01640 1 -- 172.21.15.76:0/371901318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb901553b0 con 0x7fdb9015ddd0 2024-03-08T06:00:39.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.331+0000 7fdb95200640 1 --2- 172.21.15.76:0/371901318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb900ce540 0x7fdb90147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:39.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.331+0000 7fdb95a01640 1 --2- 172.21.15.76:0/371901318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb9015ddd0 0x7fdb90145fc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800c430 tx=0x7fdb8800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:39.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.331+0000 7fdb81ffb640 1 -- 172.21.15.76:0/371901318 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88017070 con 0x7fdb9015ddd0 2024-03-08T06:00:39.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.331+0000 7fdb81ffb640 1 -- 172.21.15.76:0/371901318 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb880035f0 con 0x7fdb9015ddd0 2024-03-08T06:00:39.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.331+0000 7fdb97464640 1 -- 172.21.15.76:0/371901318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb901468c0 con 0x7fdb9015ddd0 2024-03-08T06:00:39.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.331+0000 7fdb81ffb640 1 -- 172.21.15.76:0/371901318 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb880039e0 con 0x7fdb9015ddd0 2024-03-08T06:00:39.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.331+0000 7fdb97464640 1 -- 172.21.15.76:0/371901318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb90146e80 con 0x7fdb9015ddd0 2024-03-08T06:00:39.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.332+0000 7fdb97464640 1 -- 172.21.15.76:0/371901318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9004f030 con 0x7fdb9015ddd0 2024-03-08T06:00:39.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.333+0000 7fdb81ffb640 1 -- 172.21.15.76:0/371901318 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb8801a070 con 0x7fdb9015ddd0 2024-03-08T06:00:39.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.333+0000 7fdb81ffb640 1 --2- 172.21.15.76:0/371901318 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdb7406b4d0 0x7fdb7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:39.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.333+0000 7fdb95200640 1 --2- 172.21.15.76:0/371901318 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdb7406b4d0 0x7fdb7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:39.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.333+0000 7fdb81ffb640 1 -- 172.21.15.76:0/371901318 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdb88089910 con 0x7fdb9015ddd0 2024-03-08T06:00:39.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.334+0000 7fdb95200640 1 --2- 172.21.15.76:0/371901318 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdb7406b4d0 0x7fdb7406d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdb9015b8f0 tx=0x7fdb8c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:39.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.338+0000 7fdb81ffb640 1 -- 172.21.15.76:0/371901318 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb88058500 con 0x7fdb9015ddd0 2024-03-08T06:00:39.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:39.509+0000 7fdb97464640 1 -- 172.21.15.76:0/371901318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fdb90157ba0 con 0x7fdb9015ddd0 2024-03-08T06:00:40.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.199+0000 7fdb81ffb640 1 -- 172.21.15.76:0/371901318 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdb88057e00 con 0x7fdb9015ddd0 2024-03-08T06:00:40.200 INFO:teuthology.orchestra.run.smithi076.stderr:set pool 4 allow_ec_overwrites to true 2024-03-08T06:00:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.202+0000 7fdb97464640 1 -- 172.21.15.76:0/371901318 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdb7406b4d0 msgr2=0x7fdb7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.202+0000 7fdb97464640 1 --2- 172.21.15.76:0/371901318 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdb7406b4d0 0x7fdb7406d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdb9015b8f0 tx=0x7fdb8c00b040 comp rx=0 tx=0).stop 2024-03-08T06:00:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.202+0000 7fdb97464640 1 -- 172.21.15.76:0/371901318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb9015ddd0 msgr2=0x7fdb90145fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.202+0000 7fdb97464640 1 --2- 172.21.15.76:0/371901318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb9015ddd0 0x7fdb90145fc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800c430 tx=0x7fdb8800c8f0 comp rx=0 tx=0).stop 2024-03-08T06:00:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.202+0000 7fdb97464640 1 -- 172.21.15.76:0/371901318 shutdown_connections 2024-03-08T06:00:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.202+0000 7fdb97464640 1 --2- 172.21.15.76:0/371901318 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdb7406b4d0 0x7fdb7406d990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.202+0000 7fdb97464640 1 --2- 172.21.15.76:0/371901318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb9015ddd0 0x7fdb90145fc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.202+0000 7fdb97464640 1 --2- 172.21.15.76:0/371901318 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdb900cf080 0x7fdb90154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.202+0000 7fdb97464640 1 --2- 172.21.15.76:0/371901318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb900ce540 0x7fdb90147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.202+0000 7fdb97464640 1 -- 172.21.15.76:0/371901318 >> 172.21.15.76:0/371901318 conn(0x7fdb90066fc0 msgr2=0x7fdb9015cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.204+0000 7fdb97464640 1 -- 172.21.15.76:0/371901318 shutdown_connections 2024-03-08T06:00:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.204+0000 7fdb97464640 1 -- 172.21.15.76:0/371901318 wait complete. 2024-03-08T06:00:40.230 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-08T06:00:40.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.332+0000 7fc486823640 1 Processor -- start 2024-03-08T06:00:40.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.332+0000 7fc486823640 1 -- start start 2024-03-08T06:00:40.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.333+0000 7fc486823640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc48015dbc0 0x7fc4800d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:40.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.333+0000 7fc486823640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc4800b9840 0x7fc4800b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:40.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.333+0000 7fc486823640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4800b5f40 con 0x7fc4800b5b20 2024-03-08T06:00:40.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.333+0000 7fc486823640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4800d27d0 con 0x7fc48015dbc0 2024-03-08T06:00:40.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.333+0000 7fc486823640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4800d2ab0 con 0x7fc4800b9840 2024-03-08T06:00:40.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.333+0000 7fc47ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc48015dbc0 0x7fc4800d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:40.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.333+0000 7fc47ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc48015dbc0 0x7fc4800d1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34892/0 (socket says 172.21.15.76:34892) 2024-03-08T06:00:40.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.333+0000 7fc47ffff640 1 -- 172.21.15.76:0/22894264 learned_addr learned my addr 172.21.15.76:0/22894264 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:40.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.334+0000 7fc47f7fe640 1 --2- 172.21.15.76:0/22894264 >> v2:172.21.15.81:3300/0 conn(0x7fc4800b9840 0x7fc4800b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:40.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.334+0000 7fc47ffff640 1 -- 172.21.15.76:0/22894264 >> v2:172.21.15.81:3300/0 conn(0x7fc4800b9840 msgr2=0x7fc4800b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:40.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.334+0000 7fc47ffff640 1 --2- 172.21.15.76:0/22894264 >> v2:172.21.15.81:3300/0 conn(0x7fc4800b9840 0x7fc4800b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:40.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.334+0000 7fc47ffff640 1 -- 172.21.15.76:0/22894264 >> v1:172.21.15.81:6789/0 conn(0x7fc4800b5b20 legacy=0x7fc48006c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:40.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.334+0000 7fc47ffff640 1 -- 172.21.15.76:0/22894264 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4800d2d90 con 0x7fc48015dbc0 2024-03-08T06:00:40.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.334+0000 7fc47ffff640 1 --2- 172.21.15.76:0/22894264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc48015dbc0 0x7fc4800d1ef0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4800d19d0 tx=0x7fc47002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7a75deb9b7f2d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:40.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.334+0000 7fc47effd640 1 -- 172.21.15.76:0/22894264 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470035020 con 0x7fc48015dbc0 2024-03-08T06:00:40.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.334+0000 7fc47effd640 1 -- 172.21.15.76:0/22894264 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc470031870 con 0x7fc48015dbc0 2024-03-08T06:00:40.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.334+0000 7fc47effd640 1 -- 172.21.15.76:0/22894264 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470031c40 con 0x7fc48015dbc0 2024-03-08T06:00:40.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.335+0000 7fc486823640 1 -- 172.21.15.76:0/22894264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc48015dbc0 msgr2=0x7fc4800d1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:40.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.335+0000 7fc486823640 1 --2- 172.21.15.76:0/22894264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc48015dbc0 0x7fc4800d1ef0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4800d19d0 tx=0x7fc47002b910 comp rx=0 tx=0).stop 2024-03-08T06:00:40.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.335+0000 7fc486823640 1 -- 172.21.15.76:0/22894264 shutdown_connections 2024-03-08T06:00:40.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.335+0000 7fc486823640 1 --2- 172.21.15.76:0/22894264 >> v2:172.21.15.81:3300/0 conn(0x7fc4800b9840 0x7fc4800b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:40.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.335+0000 7fc486823640 1 --2- 172.21.15.76:0/22894264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc48015dbc0 0x7fc4800d1ef0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:40.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.335+0000 7fc486823640 1 -- 172.21.15.76:0/22894264 >> 172.21.15.76:0/22894264 conn(0x7fc4800670d0 msgr2=0x7fc4800d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:40.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.335+0000 7fc486823640 1 -- 172.21.15.76:0/22894264 shutdown_connections 2024-03-08T06:00:40.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.335+0000 7fc486823640 1 -- 172.21.15.76:0/22894264 wait complete. 2024-03-08T06:00:40.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.336+0000 7fc486823640 1 Processor -- start 2024-03-08T06:00:40.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.336+0000 7fc486823640 1 -- start start 2024-03-08T06:00:40.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.337+0000 7fc486823640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4800b5b20 0x7fc4801f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:40.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.337+0000 7fc486823640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4800b9840 0x7fc4800cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:40.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.337+0000 7fc486823640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc48015dbc0 0x7fc4800cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:40.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.337+0000 7fc486823640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4800d2430 con 0x7fc4800b5b20 2024-03-08T06:00:40.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.337+0000 7fc486823640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4800d3870 con 0x7fc4800b9840 2024-03-08T06:00:40.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.337+0000 7fc486823640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4800d25b0 con 0x7fc48015dbc0 2024-03-08T06:00:40.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.337+0000 7fc47ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4800b5b20 0x7fc4801f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:40.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.337+0000 7fc47ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4800b5b20 0x7fc4801f2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34908/0 (socket says 172.21.15.76:34908) 2024-03-08T06:00:40.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.337+0000 7fc47ffff640 1 -- 172.21.15.76:0/4231687036 learned_addr learned my addr 172.21.15.76:0/4231687036 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:40.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.337+0000 7fc47ffff640 1 -- 172.21.15.76:0/4231687036 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc48015dbc0 msgr2=0x7fc4800cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:40.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.337+0000 7fc47ffff640 1 --2- 172.21.15.76:0/4231687036 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc48015dbc0 0x7fc4800cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:40.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.337+0000 7fc47ffff640 1 -- 172.21.15.76:0/4231687036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4800b9840 msgr2=0x7fc4800cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:40.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.337+0000 7fc47ffff640 1 --2- 172.21.15.76:0/4231687036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4800b9840 0x7fc4800cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:40.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.337+0000 7fc47ffff640 1 -- 172.21.15.76:0/4231687036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4800cb890 con 0x7fc4800b5b20 2024-03-08T06:00:40.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.337+0000 7fc47ffff640 1 --2- 172.21.15.76:0/4231687036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4800b5b20 0x7fc4801f2fa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc470000c00 tx=0x7fc4700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:40.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.338+0000 7fc47cff9640 1 -- 172.21.15.76:0/4231687036 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470040070 con 0x7fc4800b5b20 2024-03-08T06:00:40.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.338+0000 7fc47cff9640 1 -- 172.21.15.76:0/4231687036 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47003a420 con 0x7fc4800b5b20 2024-03-08T06:00:40.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.338+0000 7fc486823640 1 -- 172.21.15.76:0/4231687036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4800cbb70 con 0x7fc4800b5b20 2024-03-08T06:00:40.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.338+0000 7fc47cff9640 1 -- 172.21.15.76:0/4231687036 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470035040 con 0x7fc4800b5b20 2024-03-08T06:00:40.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.338+0000 7fc486823640 1 -- 172.21.15.76:0/4231687036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4800cc130 con 0x7fc4800b5b20 2024-03-08T06:00:40.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.339+0000 7fc486823640 1 -- 172.21.15.76:0/4231687036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4800cc510 con 0x7fc4800b5b20 2024-03-08T06:00:40.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.339+0000 7fc47cff9640 1 -- 172.21.15.76:0/4231687036 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc47004b060 con 0x7fc4800b5b20 2024-03-08T06:00:40.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.339+0000 7fc47cff9640 1 --2- 172.21.15.76:0/4231687036 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc46c06b4d0 0x7fc46c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:40.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.339+0000 7fc47cff9640 1 -- 172.21.15.76:0/4231687036 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc4700ac3e0 con 0x7fc4800b5b20 2024-03-08T06:00:40.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.339+0000 7fc47f7fe640 1 --2- 172.21.15.76:0/4231687036 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc46c06b4d0 0x7fc46c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:40.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.340+0000 7fc47f7fe640 1 --2- 172.21.15.76:0/4231687036 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc46c06b4d0 0x7fc46c06d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc474006fd0 tx=0x7fc474008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:40.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.344+0000 7fc47cff9640 1 -- 172.21.15.76:0/4231687036 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4800cc510 con 0x7fc4800b5b20 2024-03-08T06:00:40.536 INFO:tasks.ceph.mds.d.smithi081.stderr:2024-03-08T06:00:40.536+0000 7f935f27d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-08T06:00:40.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:40.551+0000 7fc486823640 1 -- 172.21.15.76:0/4231687036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fc48004f030 con 0x7fc4800b5b20 2024-03-08T06:00:41.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.209+0000 7fc47cff9640 1 -- 172.21.15.76:0/4231687036 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7fc47007afd0 con 0x7fc4800b5b20 2024-03-08T06:00:41.209 INFO:teuthology.orchestra.run.smithi076.stderr:added data pool 4 to fsmap 2024-03-08T06:00:41.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.212+0000 7fc486823640 1 -- 172.21.15.76:0/4231687036 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc46c06b4d0 msgr2=0x7fc46c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.212+0000 7fc486823640 1 --2- 172.21.15.76:0/4231687036 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc46c06b4d0 0x7fc46c06d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc474006fd0 tx=0x7fc474008040 comp rx=0 tx=0).stop 2024-03-08T06:00:41.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.212+0000 7fc486823640 1 -- 172.21.15.76:0/4231687036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4800b5b20 msgr2=0x7fc4801f2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.212+0000 7fc486823640 1 --2- 172.21.15.76:0/4231687036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4800b5b20 0x7fc4801f2fa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc470000c00 tx=0x7fc4700046c0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.212+0000 7fc486823640 1 -- 172.21.15.76:0/4231687036 shutdown_connections 2024-03-08T06:00:41.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.212+0000 7fc486823640 1 --2- 172.21.15.76:0/4231687036 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc46c06b4d0 0x7fc46c06d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.212+0000 7fc486823640 1 --2- 172.21.15.76:0/4231687036 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc48015dbc0 0x7fc4800cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.212+0000 7fc486823640 1 --2- 172.21.15.76:0/4231687036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4800b9840 0x7fc4800cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.212+0000 7fc486823640 1 --2- 172.21.15.76:0/4231687036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4800b5b20 0x7fc4801f2fa0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.212+0000 7fc486823640 1 -- 172.21.15.76:0/4231687036 >> 172.21.15.76:0/4231687036 conn(0x7fc4800670d0 msgr2=0x7fc48015c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:41.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.212+0000 7fc486823640 1 -- 172.21.15.76:0/4231687036 shutdown_connections 2024-03-08T06:00:41.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.212+0000 7fc486823640 1 -- 172.21.15.76:0/4231687036 wait complete. 2024-03-08T06:00:41.224 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:00:41.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.324+0000 7f0618b0c640 1 Processor -- start 2024-03-08T06:00:41.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.324+0000 7f0618b0c640 1 -- start start 2024-03-08T06:00:41.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.325+0000 7f0618b0c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f061415bb40 0x7f06140b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.325+0000 7f0618b0c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f06140cf5b0 0x7f06140ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.325+0000 7f0618b0c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06140b5000 con 0x7f061415bf20 2024-03-08T06:00:41.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.325+0000 7f0618b0c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06140cfb50 con 0x7f061415bb40 2024-03-08T06:00:41.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.325+0000 7f0618b0c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06140cfe30 con 0x7f06140cf5b0 2024-03-08T06:00:41.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.325+0000 7f061259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f061415bb40 0x7f06140b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.326+0000 7f061259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f061415bb40 0x7f06140b0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34910/0 (socket says 172.21.15.76:34910) 2024-03-08T06:00:41.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.326+0000 7f061259c640 1 -- 172.21.15.76:0/2992104775 learned_addr learned my addr 172.21.15.76:0/2992104775 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:41.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.326+0000 7f0612d9d640 1 --2- 172.21.15.76:0/2992104775 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f06140cf5b0 0x7f06140ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.326+0000 7f061259c640 1 -- 172.21.15.76:0/2992104775 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f06140cf5b0 msgr2=0x7f06140ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:41.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.326+0000 7f061259c640 1 --2- 172.21.15.76:0/2992104775 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f06140cf5b0 0x7f06140ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.326+0000 7f061259c640 1 -- 172.21.15.76:0/2992104775 >> v1:172.21.15.81:6789/0 conn(0x7f061415bf20 legacy=0x7f06140b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.326+0000 7f061259c640 1 -- 172.21.15.76:0/2992104775 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06140d0110 con 0x7f061415bb40 2024-03-08T06:00:41.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.326+0000 7f061259c640 1 --2- 172.21.15.76:0/2992104775 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061415bb40 0x7f06140b0c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f060c009e50 tx=0x7f060c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa4d5274cb9935f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:41.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.327+0000 7f061159a640 1 -- 172.21.15.76:0/2992104775 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c02cb00 con 0x7f061415bb40 2024-03-08T06:00:41.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.327+0000 7f061159a640 1 -- 172.21.15.76:0/2992104775 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060c02cca0 con 0x7f061415bb40 2024-03-08T06:00:41.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.327+0000 7f061159a640 1 -- 172.21.15.76:0/2992104775 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c034610 con 0x7f061415bb40 2024-03-08T06:00:41.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.327+0000 7f0618b0c640 1 -- 172.21.15.76:0/2992104775 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061415bb40 msgr2=0x7f06140b0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.327+0000 7f0618b0c640 1 --2- 172.21.15.76:0/2992104775 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061415bb40 0x7f06140b0c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f060c009e50 tx=0x7f060c02bf60 comp rx=0 tx=0).stop 2024-03-08T06:00:41.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.327+0000 7f0618b0c640 1 -- 172.21.15.76:0/2992104775 shutdown_connections 2024-03-08T06:00:41.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.327+0000 7f0618b0c640 1 --2- 172.21.15.76:0/2992104775 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f06140cf5b0 0x7f06140ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.327+0000 7f0618b0c640 1 --2- 172.21.15.76:0/2992104775 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061415bb40 0x7f06140b0c90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.327+0000 7f0618b0c640 1 -- 172.21.15.76:0/2992104775 >> 172.21.15.76:0/2992104775 conn(0x7f061406f5e0 msgr2=0x7f061406f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:41.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.327+0000 7f0618b0c640 1 -- 172.21.15.76:0/2992104775 shutdown_connections 2024-03-08T06:00:41.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.327+0000 7f0618b0c640 1 -- 172.21.15.76:0/2992104775 wait complete. 2024-03-08T06:00:41.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.329+0000 7f0618b0c640 1 Processor -- start 2024-03-08T06:00:41.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.329+0000 7f0618b0c640 1 -- start start 2024-03-08T06:00:41.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.329+0000 7f0618b0c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06140cf5b0 0x7f0614151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.329+0000 7f0618b0c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061415bb40 0x7f06140b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.329+0000 7f0618b0c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f061415bf20 0x7f0614157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.329+0000 7f0618b0c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06140b11d0 con 0x7f061415bb40 2024-03-08T06:00:41.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.329+0000 7f0618b0c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06140d06d0 con 0x7f06140cf5b0 2024-03-08T06:00:41.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.329+0000 7f0618b0c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06140ce8f0 con 0x7f061415bf20 2024-03-08T06:00:41.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.329+0000 7f0611d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061415bb40 0x7f06140b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.329+0000 7f0611d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061415bb40 0x7f06140b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34920/0 (socket says 172.21.15.76:34920) 2024-03-08T06:00:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.329+0000 7f0611d9b640 1 -- 172.21.15.76:0/2718575025 learned_addr learned my addr 172.21.15.76:0/2718575025 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.330+0000 7f061259c640 1 --2- 172.21.15.76:0/2718575025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06140cf5b0 0x7f0614151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.330+0000 7f0612d9d640 1 --2- 172.21.15.76:0/2718575025 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f061415bf20 0x7f0614157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.330+0000 7f0611d9b640 1 -- 172.21.15.76:0/2718575025 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f061415bf20 msgr2=0x7f0614157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.330+0000 7f0611d9b640 1 --2- 172.21.15.76:0/2718575025 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f061415bf20 0x7f0614157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.330+0000 7f0611d9b640 1 -- 172.21.15.76:0/2718575025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06140cf5b0 msgr2=0x7f0614151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.330+0000 7f0611d9b640 1 --2- 172.21.15.76:0/2718575025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06140cf5b0 0x7f0614151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.330+0000 7f0611d9b640 1 -- 172.21.15.76:0/2718575025 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06141f4000 con 0x7f061415bb40 2024-03-08T06:00:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.330+0000 7f0612d9d640 1 --2- 172.21.15.76:0/2718575025 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f061415bf20 0x7f0614157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.330+0000 7f0611d9b640 1 --2- 172.21.15.76:0/2718575025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061415bb40 0x7f06140b2100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f060400c3c0 tx=0x7f060400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.330+0000 7f060affd640 1 -- 172.21.15.76:0/2718575025 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06040052d0 con 0x7f061415bb40 2024-03-08T06:00:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.330+0000 7f060affd640 1 -- 172.21.15.76:0/2718575025 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0604005490 con 0x7f061415bb40 2024-03-08T06:00:41.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.330+0000 7f0618b0c640 1 -- 172.21.15.76:0/2718575025 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06140c6ee0 con 0x7f061415bb40 2024-03-08T06:00:41.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.330+0000 7f060affd640 1 -- 172.21.15.76:0/2718575025 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604007260 con 0x7f061415bb40 2024-03-08T06:00:41.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.330+0000 7f0618b0c640 1 -- 172.21.15.76:0/2718575025 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06140b2640 con 0x7f061415bb40 2024-03-08T06:00:41.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.331+0000 7f0618b0c640 1 -- 172.21.15.76:0/2718575025 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061404f030 con 0x7f061415bb40 2024-03-08T06:00:41.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.331+0000 7f060affd640 1 -- 172.21.15.76:0/2718575025 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0604007400 con 0x7f061415bb40 2024-03-08T06:00:41.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.332+0000 7f060affd640 1 --2- 172.21.15.76:0/2718575025 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f05f406b4d0 0x7f05f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.332+0000 7f060affd640 1 -- 172.21.15.76:0/2718575025 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0604089990 con 0x7f061415bb40 2024-03-08T06:00:41.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.332+0000 7f061259c640 1 --2- 172.21.15.76:0/2718575025 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f05f406b4d0 0x7f05f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.332+0000 7f061259c640 1 --2- 172.21.15.76:0/2718575025 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f05f406b4d0 0x7f05f406d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f060c035000 tx=0x7f060c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:41.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.336+0000 7f060affd640 1 -- 172.21.15.76:0/2718575025 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0604058560 con 0x7f061415bb40 2024-03-08T06:00:41.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.544+0000 7f0618b0c640 1 -- 172.21.15.76:0/2718575025 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06140008d0 con 0x7f061415bb40 2024-03-08T06:00:41.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.545+0000 7f060affd640 1 -- 172.21.15.76:0/2718575025 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f0604057e60 con 0x7f061415bb40 2024-03-08T06:00:41.545 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:41.546 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4249563566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4249563566},{"type":"v1","addr":"172.21.15.76:6835","nonce":4249563566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1514641617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1514641617},{"type":"v1","addr":"172.21.15.76:6837","nonce":1514641617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6834/1332442802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1332442802},{"type":"v1","addr":"172.21.15.81:6834","nonce":1332442802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:00:35.504231+0000","modified":"2024-03-08T06:00:40.552708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6835/2357927814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":2357927814},{"type":"v1","addr":"172.21.15.81:6835","nonce":2357927814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-08T06:00:41.546 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 6 2024-03-08T06:00:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.547+0000 7f0618b0c640 1 -- 172.21.15.76:0/2718575025 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f05f406b4d0 msgr2=0x7f05f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.547+0000 7f0618b0c640 1 --2- 172.21.15.76:0/2718575025 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f05f406b4d0 0x7f05f406d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f060c035000 tx=0x7f060c030000 comp rx=0 tx=0).stop 2024-03-08T06:00:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.547+0000 7f0618b0c640 1 -- 172.21.15.76:0/2718575025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061415bb40 msgr2=0x7f06140b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.547+0000 7f0618b0c640 1 --2- 172.21.15.76:0/2718575025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061415bb40 0x7f06140b2100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f060400c3c0 tx=0x7f060400c880 comp rx=0 tx=0).stop 2024-03-08T06:00:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.547+0000 7f0618b0c640 1 -- 172.21.15.76:0/2718575025 shutdown_connections 2024-03-08T06:00:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.547+0000 7f0618b0c640 1 --2- 172.21.15.76:0/2718575025 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f05f406b4d0 0x7f05f406d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.547+0000 7f0618b0c640 1 --2- 172.21.15.76:0/2718575025 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f061415bf20 0x7f0614157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.547+0000 7f0618b0c640 1 --2- 172.21.15.76:0/2718575025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061415bb40 0x7f06140b2100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.547+0000 7f0618b0c640 1 --2- 172.21.15.76:0/2718575025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06140cf5b0 0x7f0614151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.547+0000 7f0618b0c640 1 -- 172.21.15.76:0/2718575025 >> 172.21.15.76:0/2718575025 conn(0x7f061406f5e0 msgr2=0x7f06140b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.547+0000 7f0618b0c640 1 -- 172.21.15.76:0/2718575025 shutdown_connections 2024-03-08T06:00:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.547+0000 7f0618b0c640 1 -- 172.21.15.76:0/2718575025 wait complete. 2024-03-08T06:00:41.558 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:00:41.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.657+0000 7fa110ff4640 1 Processor -- start 2024-03-08T06:00:41.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.657+0000 7fa110ff4640 1 -- start start 2024-03-08T06:00:41.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.658+0000 7fa110ff4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa10c15dc60 0x7fa10c0d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.658+0000 7fa110ff4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa10c09a250 0x7fa10c0cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.658+0000 7fa110ff4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa10c0cfcc0 con 0x7fa10c15e040 2024-03-08T06:00:41.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.658+0000 7fa110ff4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10c0b4360 con 0x7fa10c15dc60 2024-03-08T06:00:41.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.658+0000 7fa110ff4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c0cff60 con 0x7fa10c09a250 2024-03-08T06:00:41.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.658+0000 7fa10a59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa10c15dc60 0x7fa10c0d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.658+0000 7fa10a59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa10c15dc60 0x7fa10c0d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34926/0 (socket says 172.21.15.76:34926) 2024-03-08T06:00:41.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.658+0000 7fa10a59c640 1 -- 172.21.15.76:0/3858156171 learned_addr learned my addr 172.21.15.76:0/3858156171 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:41.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.658+0000 7fa10a59c640 1 -- 172.21.15.76:0/3858156171 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa10c09a250 msgr2=0x7fa10c0cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:41.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.658+0000 7fa10ad9d640 1 --2- 172.21.15.76:0/3858156171 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa10c09a250 0x7fa10c0cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.658+0000 7fa10a59c640 1 --2- 172.21.15.76:0/3858156171 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa10c09a250 0x7fa10c0cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.658+0000 7fa10a59c640 1 -- 172.21.15.76:0/3858156171 >> v1:172.21.15.81:6789/0 conn(0x7fa10c15e040 legacy=0x7fa10c099b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:41.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.658+0000 7fa10a59c640 1 -- 172.21.15.76:0/3858156171 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c0d0240 con 0x7fa10c15dc60 2024-03-08T06:00:41.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.659+0000 7fa10a59c640 1 --2- 172.21.15.76:0/3858156171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10c15dc60 0x7fa10c0d1000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009e50 tx=0x7fa0fc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad48d2685abddcd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:41.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.659+0000 7fa10959a640 1 -- 172.21.15.76:0/3858156171 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc02cb00 con 0x7fa10c15dc60 2024-03-08T06:00:41.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.659+0000 7fa10959a640 1 -- 172.21.15.76:0/3858156171 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc02cca0 con 0x7fa10c15dc60 2024-03-08T06:00:41.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.659+0000 7fa10959a640 1 -- 172.21.15.76:0/3858156171 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc034610 con 0x7fa10c15dc60 2024-03-08T06:00:41.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.660+0000 7fa110ff4640 1 -- 172.21.15.76:0/3858156171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10c15dc60 msgr2=0x7fa10c0d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.660+0000 7fa110ff4640 1 --2- 172.21.15.76:0/3858156171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10c15dc60 0x7fa10c0d1000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009e50 tx=0x7fa0fc02bf60 comp rx=0 tx=0).stop 2024-03-08T06:00:41.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.660+0000 7fa110ff4640 1 -- 172.21.15.76:0/3858156171 shutdown_connections 2024-03-08T06:00:41.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.660+0000 7fa110ff4640 1 --2- 172.21.15.76:0/3858156171 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa10c09a250 0x7fa10c0cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.660+0000 7fa110ff4640 1 --2- 172.21.15.76:0/3858156171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10c15dc60 0x7fa10c0d1000 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.660+0000 7fa110ff4640 1 -- 172.21.15.76:0/3858156171 >> 172.21.15.76:0/3858156171 conn(0x7fa10c0b3440 msgr2=0x7fa10c0b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:41.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.660+0000 7fa110ff4640 1 -- 172.21.15.76:0/3858156171 shutdown_connections 2024-03-08T06:00:41.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.660+0000 7fa110ff4640 1 -- 172.21.15.76:0/3858156171 wait complete. 2024-03-08T06:00:41.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.661+0000 7fa110ff4640 1 Processor -- start 2024-03-08T06:00:41.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.661+0000 7fa110ff4640 1 -- start start 2024-03-08T06:00:41.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa110ff4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10c09a250 0x7fa10c1569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa110ff4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa10c15dc60 0x7fa10c0c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa110ff4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10c15e040 0x7fa10c0b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa110ff4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c0d0ae0 con 0x7fa10c09a250 2024-03-08T06:00:41.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa110ff4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c0d0800 con 0x7fa10c15e040 2024-03-08T06:00:41.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa110ff4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c0d1540 con 0x7fa10c15dc60 2024-03-08T06:00:41.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa10a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10c09a250 0x7fa10c1569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa10a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10c09a250 0x7fa10c1569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34930/0 (socket says 172.21.15.76:34930) 2024-03-08T06:00:41.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa10a59c640 1 -- 172.21.15.76:0/2867268414 learned_addr learned my addr 172.21.15.76:0/2867268414 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:41.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa10ad9d640 1 --2- 172.21.15.76:0/2867268414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10c15e040 0x7fa10c0b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa109d9b640 1 --2- 172.21.15.76:0/2867268414 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa10c15dc60 0x7fa10c0c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa10a59c640 1 -- 172.21.15.76:0/2867268414 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa10c15dc60 msgr2=0x7fa10c0c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa10a59c640 1 --2- 172.21.15.76:0/2867268414 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa10c15dc60 0x7fa10c0c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa10a59c640 1 -- 172.21.15.76:0/2867268414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10c15e040 msgr2=0x7fa10c0b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa10a59c640 1 --2- 172.21.15.76:0/2867268414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10c15e040 0x7fa10c0b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa10a59c640 1 -- 172.21.15.76:0/2867268414 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c0c71b0 con 0x7fa10c09a250 2024-03-08T06:00:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa109d9b640 1 --2- 172.21.15.76:0/2867268414 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa10c15dc60 0x7fa10c0c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.662+0000 7fa10a59c640 1 --2- 172.21.15.76:0/2867268414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10c09a250 0x7fa10c1569f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009e50 tx=0x7fa0fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.663+0000 7fa0faffd640 1 -- 172.21.15.76:0/2867268414 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc043020 con 0x7fa10c09a250 2024-03-08T06:00:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.663+0000 7fa0faffd640 1 -- 172.21.15.76:0/2867268414 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc02ce50 con 0x7fa10c09a250 2024-03-08T06:00:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.663+0000 7fa110ff4640 1 -- 172.21.15.76:0/2867268414 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10c0c7490 con 0x7fa10c09a250 2024-03-08T06:00:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.663+0000 7fa0faffd640 1 -- 172.21.15.76:0/2867268414 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc03e690 con 0x7fa10c09a250 2024-03-08T06:00:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.663+0000 7fa110ff4640 1 -- 172.21.15.76:0/2867268414 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10c0c7a50 con 0x7fa10c09a250 2024-03-08T06:00:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.664+0000 7fa110ff4640 1 -- 172.21.15.76:0/2867268414 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10c04f030 con 0x7fa10c09a250 2024-03-08T06:00:41.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.664+0000 7fa0faffd640 1 -- 172.21.15.76:0/2867268414 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0fc046020 con 0x7fa10c09a250 2024-03-08T06:00:41.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.664+0000 7fa0faffd640 1 --2- 172.21.15.76:0/2867268414 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa0ec06b4d0 0x7fa0ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.664+0000 7fa109d9b640 1 --2- 172.21.15.76:0/2867268414 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa0ec06b4d0 0x7fa0ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.664+0000 7fa0faffd640 1 -- 172.21.15.76:0/2867268414 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa0fc0ace00 con 0x7fa10c09a250 2024-03-08T06:00:41.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.665+0000 7fa109d9b640 1 --2- 172.21.15.76:0/2867268414 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa0ec06b4d0 0x7fa0ec06d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0713b0 tx=0x7fa100008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:41.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.669+0000 7fa0faffd640 1 -- 172.21.15.76:0/2867268414 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0fc04d030 con 0x7fa10c09a250 2024-03-08T06:00:41.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.841+0000 7fa110ff4640 1 -- 172.21.15.76:0/2867268414 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa10c1614d0 con 0x7fa10c09a250 2024-03-08T06:00:41.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.842+0000 7fa0faffd640 1 -- 172.21.15.76:0/2867268414 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18443 (secure 0 0 0) 0x7fa0fc07ba60 con 0x7fa10c09a250 2024-03-08T06:00:41.843 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:41.843 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:40.552495+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:00:41.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.844+0000 7fa110ff4640 1 -- 172.21.15.76:0/2867268414 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa0ec06b4d0 msgr2=0x7fa0ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.844+0000 7fa110ff4640 1 --2- 172.21.15.76:0/2867268414 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa0ec06b4d0 0x7fa0ec06d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0713b0 tx=0x7fa100008040 comp rx=0 tx=0).stop 2024-03-08T06:00:41.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.844+0000 7fa110ff4640 1 -- 172.21.15.76:0/2867268414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10c09a250 msgr2=0x7fa10c1569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.844+0000 7fa110ff4640 1 --2- 172.21.15.76:0/2867268414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10c09a250 0x7fa10c1569f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009e50 tx=0x7fa0fc004a40 comp rx=0 tx=0).stop 2024-03-08T06:00:41.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.844+0000 7fa110ff4640 1 -- 172.21.15.76:0/2867268414 shutdown_connections 2024-03-08T06:00:41.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.844+0000 7fa110ff4640 1 --2- 172.21.15.76:0/2867268414 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa0ec06b4d0 0x7fa0ec06d990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.844+0000 7fa110ff4640 1 --2- 172.21.15.76:0/2867268414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10c15e040 0x7fa10c0b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.844+0000 7fa110ff4640 1 --2- 172.21.15.76:0/2867268414 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa10c15dc60 0x7fa10c0c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.844+0000 7fa110ff4640 1 --2- 172.21.15.76:0/2867268414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10c09a250 0x7fa10c1569f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.844+0000 7fa110ff4640 1 -- 172.21.15.76:0/2867268414 >> 172.21.15.76:0/2867268414 conn(0x7fa10c0b3440 msgr2=0x7fa10c0704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:41.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.844+0000 7fa110ff4640 1 -- 172.21.15.76:0/2867268414 shutdown_connections 2024-03-08T06:00:41.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.844+0000 7fa110ff4640 1 -- 172.21.15.76:0/2867268414 wait complete. 2024-03-08T06:00:41.856 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:00:41.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.956+0000 7fee26181640 1 Processor -- start 2024-03-08T06:00:41.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.956+0000 7fee26181640 1 -- start start 2024-03-08T06:00:41.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.957+0000 7fee26181640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fee2015dc60 0x7fee200d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.957+0000 7fee26181640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fee2015e040 0x7fee20099b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.957+0000 7fee26181640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee200cfcc0 con 0x7fee2006bfa0 2024-03-08T06:00:41.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.957+0000 7fee26181640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee200b4360 con 0x7fee2015dc60 2024-03-08T06:00:41.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.957+0000 7fee26181640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee200cff60 con 0x7fee2015e040 2024-03-08T06:00:41.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.957+0000 7fee1effd640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fee2015e040 0x7fee20099b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.957+0000 7fee1f7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fee2015dc60 0x7fee200d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.957+0000 7fee1ffff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fee2006bfa0 0x7fee200cbfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:36508/0 (socket says 172.21.15.76:36508) 2024-03-08T06:00:41.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.957+0000 7fee1ffff640 1 -- 172.21.15.76:0/3179127094 learned_addr learned my addr 172.21.15.76:0/3179127094 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:41.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.958+0000 7fee1f7fe640 1 -- 172.21.15.76:0/3179127094 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fee2015e040 msgr2=0x7fee20099b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:41.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.958+0000 7fee1f7fe640 1 --2- 172.21.15.76:0/3179127094 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fee2015e040 0x7fee20099b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.958+0000 7fee1f7fe640 1 -- 172.21.15.76:0/3179127094 >> v1:172.21.15.81:6789/0 conn(0x7fee2006bfa0 legacy=0x7fee200cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.958+0000 7fee1f7fe640 1 -- 172.21.15.76:0/3179127094 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee200d0240 con 0x7fee2015dc60 2024-03-08T06:00:41.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.958+0000 7fee1e7fc640 1 -- 172.21.15.76:0/3179127094 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185052489 0 0) 0x7fee200cfcc0 con 0x7fee2006bfa0 2024-03-08T06:00:41.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.958+0000 7fee1effd640 1 --2- 172.21.15.76:0/3179127094 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fee2015e040 0x7fee20099b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:00:41.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.958+0000 7fee1f7fe640 1 --2- 172.21.15.76:0/3179127094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee2015dc60 0x7fee200d1000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fee100097a0 tx=0x7fee1002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18bb8a671e09538 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:41.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.958+0000 7fee1e7fc640 1 -- 172.21.15.76:0/3179127094 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee10034070 con 0x7fee2015dc60 2024-03-08T06:00:41.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.959+0000 7fee1e7fc640 1 -- 172.21.15.76:0/3179127094 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee10030610 con 0x7fee2015dc60 2024-03-08T06:00:41.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.959+0000 7fee1e7fc640 1 -- 172.21.15.76:0/3179127094 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee100309e0 con 0x7fee2015dc60 2024-03-08T06:00:41.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.959+0000 7fee26181640 1 -- 172.21.15.76:0/3179127094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee2015dc60 msgr2=0x7fee200d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.959+0000 7fee26181640 1 --2- 172.21.15.76:0/3179127094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee2015dc60 0x7fee200d1000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fee100097a0 tx=0x7fee1002b960 comp rx=0 tx=0).stop 2024-03-08T06:00:41.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.959+0000 7fee26181640 1 -- 172.21.15.76:0/3179127094 shutdown_connections 2024-03-08T06:00:41.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.959+0000 7fee26181640 1 --2- 172.21.15.76:0/3179127094 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fee2015e040 0x7fee20099b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.959+0000 7fee26181640 1 --2- 172.21.15.76:0/3179127094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee2015dc60 0x7fee200d1000 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.959+0000 7fee26181640 1 -- 172.21.15.76:0/3179127094 >> 172.21.15.76:0/3179127094 conn(0x7fee200b3440 msgr2=0x7fee200b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:41.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.959+0000 7fee26181640 1 -- 172.21.15.76:0/3179127094 shutdown_connections 2024-03-08T06:00:41.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.959+0000 7fee26181640 1 -- 172.21.15.76:0/3179127094 wait complete. 2024-03-08T06:00:41.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.960+0000 7fee26181640 1 Processor -- start 2024-03-08T06:00:41.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.960+0000 7fee26181640 1 -- start start 2024-03-08T06:00:41.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.961+0000 7fee26181640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee2006bfa0 0x7fee2014cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.961+0000 7fee26181640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fee2015dc60 0x7fee2015e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.961+0000 7fee26181640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee2015e040 0x7fee201496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.961+0000 7fee26181640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee200d0ae0 con 0x7fee2015e040 2024-03-08T06:00:41.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.961+0000 7fee26181640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee200d0800 con 0x7fee2006bfa0 2024-03-08T06:00:41.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.961+0000 7fee26181640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee200d1540 con 0x7fee2015dc60 2024-03-08T06:00:41.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.961+0000 7fee1ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee2015e040 0x7fee201496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.961+0000 7fee1ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee2015e040 0x7fee201496f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34946/0 (socket says 172.21.15.76:34946) 2024-03-08T06:00:41.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.961+0000 7fee1ffff640 1 -- 172.21.15.76:0/1919879854 learned_addr learned my addr 172.21.15.76:0/1919879854 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:41.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.961+0000 7fee1f7fe640 1 --2- 172.21.15.76:0/1919879854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee2006bfa0 0x7fee2014cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.961+0000 7fee1effd640 1 --2- 172.21.15.76:0/1919879854 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fee2015dc60 0x7fee2015e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.962+0000 7fee1ffff640 1 -- 172.21.15.76:0/1919879854 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fee2015dc60 msgr2=0x7fee2015e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.962+0000 7fee1ffff640 1 --2- 172.21.15.76:0/1919879854 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fee2015dc60 0x7fee2015e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.962+0000 7fee1ffff640 1 -- 172.21.15.76:0/1919879854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee2006bfa0 msgr2=0x7fee2014cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:41.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.962+0000 7fee1ffff640 1 --2- 172.21.15.76:0/1919879854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee2006bfa0 0x7fee2014cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:41.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.962+0000 7fee1ffff640 1 -- 172.21.15.76:0/1919879854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee20161310 con 0x7fee2015e040 2024-03-08T06:00:41.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.962+0000 7fee1ffff640 1 --2- 172.21.15.76:0/1919879854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee2015e040 0x7fee201496f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fee1800c7a0 tx=0x7fee1800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:41.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.962+0000 7fee07fff640 1 -- 172.21.15.76:0/1919879854 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18018070 con 0x7fee2015e040 2024-03-08T06:00:41.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.962+0000 7fee07fff640 1 -- 172.21.15.76:0/1919879854 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1800a5e0 con 0x7fee2015e040 2024-03-08T06:00:41.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.962+0000 7fee07fff640 1 -- 172.21.15.76:0/1919879854 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1800aa10 con 0x7fee2015e040 2024-03-08T06:00:41.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.962+0000 7fee26181640 1 -- 172.21.15.76:0/1919879854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee200cd100 con 0x7fee2015e040 2024-03-08T06:00:41.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.962+0000 7fee26181640 1 -- 172.21.15.76:0/1919879854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee20149c30 con 0x7fee2015e040 2024-03-08T06:00:41.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.963+0000 7fee26181640 1 -- 172.21.15.76:0/1919879854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2004f030 con 0x7fee2015e040 2024-03-08T06:00:41.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.963+0000 7fee07fff640 1 -- 172.21.15.76:0/1919879854 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee1801b070 con 0x7fee2015e040 2024-03-08T06:00:41.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.964+0000 7fee07fff640 1 --2- 172.21.15.76:0/1919879854 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fee0c06b4d0 0x7fee0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:41.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.964+0000 7fee1f7fe640 1 --2- 172.21.15.76:0/1919879854 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fee0c06b4d0 0x7fee0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:41.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.964+0000 7fee07fff640 1 -- 172.21.15.76:0/1919879854 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fee18089960 con 0x7fee2015e040 2024-03-08T06:00:41.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.964+0000 7fee1f7fe640 1 --2- 172.21.15.76:0/1919879854 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fee0c06b4d0 0x7fee0c06d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fee10036000 tx=0x7fee10008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:41.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:41.968+0000 7fee07fff640 1 -- 172.21.15.76:0/1919879854 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee18057ee0 con 0x7fee2015e040 2024-03-08T06:00:42.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.140+0000 7fee26181640 1 -- 172.21.15.76:0/1919879854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee200cd2d0 con 0x7fee2015e040 2024-03-08T06:00:42.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.141+0000 7fee07fff640 1 -- 172.21.15.76:0/1919879854 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18443 (secure 0 0 0) 0x7fee1805bf80 con 0x7fee2015e040 2024-03-08T06:00:42.142 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:42.142 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:40.552495+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:00:42.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.143+0000 7fee26181640 1 -- 172.21.15.76:0/1919879854 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fee0c06b4d0 msgr2=0x7fee0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:42.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.143+0000 7fee26181640 1 --2- 172.21.15.76:0/1919879854 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fee0c06b4d0 0x7fee0c06d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fee10036000 tx=0x7fee10008030 comp rx=0 tx=0).stop 2024-03-08T06:00:42.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.143+0000 7fee26181640 1 -- 172.21.15.76:0/1919879854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee2015e040 msgr2=0x7fee201496f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:42.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.143+0000 7fee26181640 1 --2- 172.21.15.76:0/1919879854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee2015e040 0x7fee201496f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fee1800c7a0 tx=0x7fee1800cc60 comp rx=0 tx=0).stop 2024-03-08T06:00:42.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.143+0000 7fee26181640 1 -- 172.21.15.76:0/1919879854 shutdown_connections 2024-03-08T06:00:42.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.143+0000 7fee26181640 1 --2- 172.21.15.76:0/1919879854 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fee0c06b4d0 0x7fee0c06d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.143+0000 7fee26181640 1 --2- 172.21.15.76:0/1919879854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee2015e040 0x7fee201496f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.143+0000 7fee26181640 1 --2- 172.21.15.76:0/1919879854 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fee2015dc60 0x7fee2015e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.143+0000 7fee26181640 1 --2- 172.21.15.76:0/1919879854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee2006bfa0 0x7fee2014cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.143+0000 7fee26181640 1 -- 172.21.15.76:0/1919879854 >> 172.21.15.76:0/1919879854 conn(0x7fee200b3440 msgr2=0x7fee200b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.143+0000 7fee26181640 1 -- 172.21.15.76:0/1919879854 shutdown_connections 2024-03-08T06:00:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.143+0000 7fee26181640 1 -- 172.21.15.76:0/1919879854 wait complete. 2024-03-08T06:00:42.167 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:00:42.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.269+0000 7f7276ed4640 1 Processor -- start 2024-03-08T06:00:42.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.269+0000 7f7276ed4640 1 -- start start 2024-03-08T06:00:42.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f7276ed4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f727015dfc0 0x7f727015e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:42.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f7276ed4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f727015e8e0 0x7f727006c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f7276ed4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f727006c750 0x7f72700b5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f7276ed4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72700b6c10 con 0x7f727015e8e0 2024-03-08T06:00:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f7276ed4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727015ef30 con 0x7f727015dfc0 2024-03-08T06:00:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f7276ed4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727015f210 con 0x7f727006c750 2024-03-08T06:00:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f726ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f727015e8e0 0x7f727006c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f726ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f727015e8e0 0x7f727006c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34958/0 (socket says 172.21.15.76:34958) 2024-03-08T06:00:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f726ffff640 1 -- 172.21.15.76:0/3475305302 learned_addr learned my addr 172.21.15.76:0/3475305302 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f7275471640 1 --2- 172.21.15.76:0/3475305302 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f727006c750 0x7f72700b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f7274c70640 1 --2- 172.21.15.76:0/3475305302 >> v2:172.21.15.81:3300/0 conn(0x7f727015dfc0 0x7f727015e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f726ffff640 1 -- 172.21.15.76:0/3475305302 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f727006c750 msgr2=0x7f72700b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f726ffff640 1 --2- 172.21.15.76:0/3475305302 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f727006c750 0x7f72700b5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f726ffff640 1 -- 172.21.15.76:0/3475305302 >> v2:172.21.15.81:3300/0 conn(0x7f727015dfc0 msgr2=0x7f727015e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f726ffff640 1 --2- 172.21.15.76:0/3475305302 >> v2:172.21.15.81:3300/0 conn(0x7f727015dfc0 0x7f727015e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f726ffff640 1 -- 172.21.15.76:0/3475305302 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72700d1f00 con 0x7f727015e8e0 2024-03-08T06:00:42.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.270+0000 7f7275471640 1 --2- 172.21.15.76:0/3475305302 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f727006c750 0x7f72700b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:00:42.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.271+0000 7f726ffff640 1 --2- 172.21.15.76:0/3475305302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727015e8e0 0x7f727006c210 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f727006ce30 tx=0x7f725c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc17b3847ff5c35 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:42.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.271+0000 7f726f7fe640 1 -- 172.21.15.76:0/3475305302 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c004230 con 0x7f727015e8e0 2024-03-08T06:00:42.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.271+0000 7f726f7fe640 1 -- 172.21.15.76:0/3475305302 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c0043d0 con 0x7f727015e8e0 2024-03-08T06:00:42.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.271+0000 7f726f7fe640 1 -- 172.21.15.76:0/3475305302 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c030840 con 0x7f727015e8e0 2024-03-08T06:00:42.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.271+0000 7f7276ed4640 1 -- 172.21.15.76:0/3475305302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727015e8e0 msgr2=0x7f727006c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:42.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.271+0000 7f7276ed4640 1 --2- 172.21.15.76:0/3475305302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727015e8e0 0x7f727006c210 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f727006ce30 tx=0x7f725c02b980 comp rx=0 tx=0).stop 2024-03-08T06:00:42.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.271+0000 7f7276ed4640 1 -- 172.21.15.76:0/3475305302 shutdown_connections 2024-03-08T06:00:42.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.272+0000 7f7276ed4640 1 --2- 172.21.15.76:0/3475305302 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f727006c750 0x7f72700b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.272+0000 7f7276ed4640 1 --2- 172.21.15.76:0/3475305302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727015e8e0 0x7f727006c210 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.272+0000 7f7276ed4640 1 --2- 172.21.15.76:0/3475305302 >> v2:172.21.15.81:3300/0 conn(0x7f727015dfc0 0x7f727015e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.272+0000 7f7276ed4640 1 -- 172.21.15.76:0/3475305302 >> 172.21.15.76:0/3475305302 conn(0x7f7270066de0 msgr2=0x7f72700671f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:42.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.272+0000 7f7276ed4640 1 -- 172.21.15.76:0/3475305302 shutdown_connections 2024-03-08T06:00:42.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.272+0000 7f7276ed4640 1 -- 172.21.15.76:0/3475305302 wait complete. 2024-03-08T06:00:42.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.273+0000 7f7276ed4640 1 Processor -- start 2024-03-08T06:00:42.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.273+0000 7f7276ed4640 1 -- start start 2024-03-08T06:00:42.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7276ed4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727006c750 0x7f72701e5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:42.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7276ed4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f727015dfc0 0x7f72701e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:42.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7276ed4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f727015e8e0 0x7f72701e4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:42.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7276ed4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72700b5b70 con 0x7f727006c750 2024-03-08T06:00:42.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7276ed4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72700b9720 con 0x7f727015e8e0 2024-03-08T06:00:42.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7276ed4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72700b5cf0 con 0x7f727015dfc0 2024-03-08T06:00:42.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7274c70640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727006c750 0x7f72701e5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:42.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7274c70640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727006c750 0x7f72701e5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34968/0 (socket says 172.21.15.76:34968) 2024-03-08T06:00:42.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7274c70640 1 -- 172.21.15.76:0/2479441677 learned_addr learned my addr 172.21.15.76:0/2479441677 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:42.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f726ffff640 1 --2- 172.21.15.76:0/2479441677 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f727015dfc0 0x7f72701e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:42.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7275471640 1 --2- 172.21.15.76:0/2479441677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f727015e8e0 0x7f72701e4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:42.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7274c70640 1 -- 172.21.15.76:0/2479441677 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f727015dfc0 msgr2=0x7f72701e65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:42.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7274c70640 1 --2- 172.21.15.76:0/2479441677 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f727015dfc0 0x7f72701e65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7274c70640 1 -- 172.21.15.76:0/2479441677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f727015e8e0 msgr2=0x7f72701e4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:42.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7274c70640 1 --2- 172.21.15.76:0/2479441677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f727015e8e0 0x7f72701e4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7274c70640 1 -- 172.21.15.76:0/2479441677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72701e4b70 con 0x7f727006c750 2024-03-08T06:00:42.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7274c70640 1 --2- 172.21.15.76:0/2479441677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727006c750 0x7f72701e5c10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f726400e9b0 tx=0x7f726400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:42.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f726d7fa640 1 -- 172.21.15.76:0/2479441677 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7264017070 con 0x7f727006c750 2024-03-08T06:00:42.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f726d7fa640 1 -- 172.21.15.76:0/2479441677 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726400c9a0 con 0x7f727006c750 2024-03-08T06:00:42.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7276ed4640 1 -- 172.21.15.76:0/2479441677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72701e4e50 con 0x7f727006c750 2024-03-08T06:00:42.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f726d7fa640 1 -- 172.21.15.76:0/2479441677 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726400cd90 con 0x7f727006c750 2024-03-08T06:00:42.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.274+0000 7f7276ed4640 1 -- 172.21.15.76:0/2479441677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72701e5410 con 0x7f727006c750 2024-03-08T06:00:42.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.275+0000 7f7276ed4640 1 -- 172.21.15.76:0/2479441677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727004f030 con 0x7f727006c750 2024-03-08T06:00:42.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.276+0000 7f726d7fa640 1 -- 172.21.15.76:0/2479441677 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f726401a070 con 0x7f727006c750 2024-03-08T06:00:42.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.276+0000 7f726d7fa640 1 --2- 172.21.15.76:0/2479441677 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f725406b4d0 0x7f725406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:42.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.276+0000 7f726d7fa640 1 -- 172.21.15.76:0/2479441677 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f726408a920 con 0x7f727006c750 2024-03-08T06:00:42.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.276+0000 7f726ffff640 1 --2- 172.21.15.76:0/2479441677 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f725406b4d0 0x7f725406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:42.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.276+0000 7f726ffff640 1 --2- 172.21.15.76:0/2479441677 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f725406b4d0 0x7f725406d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f725c035000 tx=0x7f725c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:42.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.280+0000 7f726d7fa640 1 -- 172.21.15.76:0/2479441677 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7264059460 con 0x7f727006c750 2024-03-08T06:00:42.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.485+0000 7f7276ed4640 1 -- 172.21.15.76:0/2479441677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72701e5830 con 0x7f727006c750 2024-03-08T06:00:42.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.485+0000 7f726d7fa640 1 -- 172.21.15.76:0/2479441677 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f7264058d60 con 0x7f727006c750 2024-03-08T06:00:42.486 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:42.486 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4249563566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4249563566},{"type":"v1","addr":"172.21.15.76:6835","nonce":4249563566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1514641617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1514641617},{"type":"v1","addr":"172.21.15.76:6837","nonce":1514641617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6834/1332442802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1332442802},{"type":"v1","addr":"172.21.15.81:6834","nonce":1332442802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:00:35.504231+0000","modified":"2024-03-08T06:00:40.552708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6835/2357927814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":2357927814},{"type":"v1","addr":"172.21.15.81:6835","nonce":2357927814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-08T06:00:42.486 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 6 2024-03-08T06:00:42.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.487+0000 7f7276ed4640 1 -- 172.21.15.76:0/2479441677 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f725406b4d0 msgr2=0x7f725406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:42.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.487+0000 7f7276ed4640 1 --2- 172.21.15.76:0/2479441677 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f725406b4d0 0x7f725406d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f725c035000 tx=0x7f725c008030 comp rx=0 tx=0).stop 2024-03-08T06:00:42.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.487+0000 7f7276ed4640 1 -- 172.21.15.76:0/2479441677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727006c750 msgr2=0x7f72701e5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:42.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.487+0000 7f7276ed4640 1 --2- 172.21.15.76:0/2479441677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727006c750 0x7f72701e5c10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f726400e9b0 tx=0x7f726400ee70 comp rx=0 tx=0).stop 2024-03-08T06:00:42.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.487+0000 7f7276ed4640 1 -- 172.21.15.76:0/2479441677 shutdown_connections 2024-03-08T06:00:42.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.487+0000 7f7276ed4640 1 --2- 172.21.15.76:0/2479441677 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f725406b4d0 0x7f725406d990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.487+0000 7f7276ed4640 1 --2- 172.21.15.76:0/2479441677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f727015e8e0 0x7f72701e4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.487+0000 7f7276ed4640 1 --2- 172.21.15.76:0/2479441677 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f727015dfc0 0x7f72701e65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.487+0000 7f7276ed4640 1 --2- 172.21.15.76:0/2479441677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727006c750 0x7f72701e5c10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.487+0000 7f7276ed4640 1 -- 172.21.15.76:0/2479441677 >> 172.21.15.76:0/2479441677 conn(0x7f7270066de0 msgr2=0x7f72700c9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:42.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.487+0000 7f7276ed4640 1 -- 172.21.15.76:0/2479441677 shutdown_connections 2024-03-08T06:00:42.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.487+0000 7f7276ed4640 1 -- 172.21.15.76:0/2479441677 wait complete. 2024-03-08T06:00:42.499 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:00:42.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.598+0000 7f08e75d0640 1 Processor -- start 2024-03-08T06:00:42.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.598+0000 7f08e75d0640 1 -- start start 2024-03-08T06:00:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.599+0000 7f08e75d0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f08e00c98e0 0x7f08e00c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.599+0000 7f08e75d0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08e00ca2d0 0x7f08e00cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.599+0000 7f08e75d0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e00ae8d0 con 0x7f08e015a9f0 2024-03-08T06:00:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.599+0000 7f08e75d0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e00ce790 con 0x7f08e00c98e0 2024-03-08T06:00:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.599+0000 7f08e75d0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e00aebb0 con 0x7f08e00ca2d0 2024-03-08T06:00:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.599+0000 7f08e4b6b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f08e00c98e0 0x7f08e00c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.599+0000 7f08e536c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f08e015a9f0 0x7f08e0071da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33794/0 (socket says 172.21.15.76:33794) 2024-03-08T06:00:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.599+0000 7f08e536c640 1 -- 172.21.15.76:0/1108395957 learned_addr learned my addr 172.21.15.76:0/1108395957 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.599+0000 7f08e4b6b640 1 -- 172.21.15.76:0/1108395957 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08e00ca2d0 msgr2=0x7f08e00cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.599+0000 7f08e5b6d640 1 --2- 172.21.15.76:0/1108395957 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08e00ca2d0 0x7f08e00cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.599+0000 7f08d7fff640 1 -- 172.21.15.76:0/1108395957 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640384499 0 0) 0x7f08e00ae8d0 con 0x7f08e015a9f0 2024-03-08T06:00:42.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.599+0000 7f08e4b6b640 1 --2- 172.21.15.76:0/1108395957 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08e00ca2d0 0x7f08e00cec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.599+0000 7f08e4b6b640 1 -- 172.21.15.76:0/1108395957 >> v1:172.21.15.76:6789/0 conn(0x7f08e015a9f0 legacy=0x7f08e0071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:42.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.599+0000 7f08e4b6b640 1 -- 172.21.15.76:0/1108395957 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e00aee90 con 0x7f08e00c98e0 2024-03-08T06:00:42.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.600+0000 7f08e4b6b640 1 --2- 172.21.15.76:0/1108395957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e00c98e0 0x7f08e00c9d90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f08d00097a0 tx=0x7f08d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f014addbfe1f33 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:42.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.600+0000 7f08d7fff640 1 -- 172.21.15.76:0/1108395957 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0035020 con 0x7f08e00c98e0 2024-03-08T06:00:42.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.600+0000 7f08d7fff640 1 -- 172.21.15.76:0/1108395957 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d0031830 con 0x7f08e00c98e0 2024-03-08T06:00:42.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.600+0000 7f08d7fff640 1 -- 172.21.15.76:0/1108395957 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0031c00 con 0x7f08e00c98e0 2024-03-08T06:00:42.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.600+0000 7f08e75d0640 1 -- 172.21.15.76:0/1108395957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e00c98e0 msgr2=0x7f08e00c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:42.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.600+0000 7f08e75d0640 1 --2- 172.21.15.76:0/1108395957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e00c98e0 0x7f08e00c9d90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f08d00097a0 tx=0x7f08d002b8d0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.601+0000 7f08e75d0640 1 -- 172.21.15.76:0/1108395957 shutdown_connections 2024-03-08T06:00:42.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.601+0000 7f08e75d0640 1 --2- 172.21.15.76:0/1108395957 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08e00ca2d0 0x7f08e00cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.601+0000 7f08e75d0640 1 --2- 172.21.15.76:0/1108395957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e00c98e0 0x7f08e00c9d90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.601+0000 7f08e75d0640 1 -- 172.21.15.76:0/1108395957 >> 172.21.15.76:0/1108395957 conn(0x7f08e00cf740 msgr2=0x7f08e00cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:42.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.601+0000 7f08e75d0640 1 -- 172.21.15.76:0/1108395957 shutdown_connections 2024-03-08T06:00:42.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.601+0000 7f08e75d0640 1 -- 172.21.15.76:0/1108395957 wait complete. 2024-03-08T06:00:42.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.602+0000 7f08e75d0640 1 Processor -- start 2024-03-08T06:00:42.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.602+0000 7f08e75d0640 1 -- start start 2024-03-08T06:00:42.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.602+0000 7f08e75d0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08e00c98e0 0x7f08e0142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:42.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.602+0000 7f08e75d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e00ca2d0 0x7f08e0153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:42.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.602+0000 7f08e75d0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08e015a9f0 0x7f08e0153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:42.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e75d0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e015af50 con 0x7f08e00ca2d0 2024-03-08T06:00:42.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e75d0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e015add0 con 0x7f08e015a9f0 2024-03-08T06:00:42.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e75d0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e00cf160 con 0x7f08e00c98e0 2024-03-08T06:00:42.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e4b6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e00ca2d0 0x7f08e0153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:42.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e4b6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e00ca2d0 0x7f08e0153a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35000/0 (socket says 172.21.15.76:35000) 2024-03-08T06:00:42.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e4b6b640 1 -- 172.21.15.76:0/3548940017 learned_addr learned my addr 172.21.15.76:0/3548940017 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:42.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e4b6b640 1 -- 172.21.15.76:0/3548940017 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08e00c98e0 msgr2=0x7f08e0142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:42.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e5b6d640 1 --2- 172.21.15.76:0/3548940017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08e015a9f0 0x7f08e0153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:42.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e536c640 1 --2- 172.21.15.76:0/3548940017 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08e00c98e0 0x7f08e0142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:42.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e4b6b640 1 --2- 172.21.15.76:0/3548940017 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08e00c98e0 0x7f08e0142060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e4b6b640 1 -- 172.21.15.76:0/3548940017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08e015a9f0 msgr2=0x7f08e0153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:42.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e4b6b640 1 --2- 172.21.15.76:0/3548940017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08e015a9f0 0x7f08e0153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e4b6b640 1 -- 172.21.15.76:0/3548940017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e014fda0 con 0x7f08e00ca2d0 2024-03-08T06:00:42.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e5b6d640 1 --2- 172.21.15.76:0/3548940017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08e015a9f0 0x7f08e0153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:42.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e4b6b640 1 --2- 172.21.15.76:0/3548940017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e00ca2d0 0x7f08e0153a00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f08d002bee0 tx=0x7f08d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:42.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08d5ffb640 1 -- 172.21.15.76:0/3548940017 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0041070 con 0x7f08e00ca2d0 2024-03-08T06:00:42.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08d5ffb640 1 -- 172.21.15.76:0/3548940017 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d003a950 con 0x7f08e00ca2d0 2024-03-08T06:00:42.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e75d0640 1 -- 172.21.15.76:0/3548940017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e0150080 con 0x7f08e00ca2d0 2024-03-08T06:00:42.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08d5ffb640 1 -- 172.21.15.76:0/3548940017 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0035040 con 0x7f08e00ca2d0 2024-03-08T06:00:42.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.603+0000 7f08e75d0640 1 -- 172.21.15.76:0/3548940017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e0150640 con 0x7f08e00ca2d0 2024-03-08T06:00:42.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.604+0000 7f08e75d0640 1 -- 172.21.15.76:0/3548940017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e004f030 con 0x7f08e00ca2d0 2024-03-08T06:00:42.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.605+0000 7f08d5ffb640 1 -- 172.21.15.76:0/3548940017 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08d004c060 con 0x7f08e00ca2d0 2024-03-08T06:00:42.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.605+0000 7f08d5ffb640 1 --2- 172.21.15.76:0/3548940017 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f08c406b4d0 0x7f08c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:42.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.605+0000 7f08d5ffb640 1 -- 172.21.15.76:0/3548940017 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f08d00abfd0 con 0x7f08e00ca2d0 2024-03-08T06:00:42.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.605+0000 7f08e536c640 1 --2- 172.21.15.76:0/3548940017 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f08c406b4d0 0x7f08c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:42.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.606+0000 7f08e536c640 1 --2- 172.21.15.76:0/3548940017 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f08c406b4d0 0x7f08c406d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f08dc000f60 tx=0x7f08dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.609+0000 7f08d5ffb640 1 -- 172.21.15.76:0/3548940017 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f08d007aa70 con 0x7f08e00ca2d0 2024-03-08T06:00:42.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.781+0000 7f08e75d0640 1 -- 172.21.15.76:0/3548940017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08e00725a0 con 0x7f08e00ca2d0 2024-03-08T06:00:42.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.782+0000 7f08d5ffb640 1 -- 172.21.15.76:0/3548940017 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f08d007a370 con 0x7f08e00ca2d0 2024-03-08T06:00:42.783 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:42.783 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":26,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:42.202011+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:00:42.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.784+0000 7f08e75d0640 1 -- 172.21.15.76:0/3548940017 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f08c406b4d0 msgr2=0x7f08c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:42.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.784+0000 7f08e75d0640 1 --2- 172.21.15.76:0/3548940017 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f08c406b4d0 0x7f08c406d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f08dc000f60 tx=0x7f08dc009040 comp rx=0 tx=0).stop 2024-03-08T06:00:42.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.784+0000 7f08e75d0640 1 -- 172.21.15.76:0/3548940017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e00ca2d0 msgr2=0x7f08e0153a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:42.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.784+0000 7f08e75d0640 1 --2- 172.21.15.76:0/3548940017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e00ca2d0 0x7f08e0153a00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f08d002bee0 tx=0x7f08d00046c0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.784+0000 7f08e75d0640 1 -- 172.21.15.76:0/3548940017 shutdown_connections 2024-03-08T06:00:42.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.784+0000 7f08e75d0640 1 --2- 172.21.15.76:0/3548940017 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f08c406b4d0 0x7f08c406d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.784+0000 7f08e75d0640 1 --2- 172.21.15.76:0/3548940017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08e015a9f0 0x7f08e0153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.784+0000 7f08e75d0640 1 --2- 172.21.15.76:0/3548940017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e00ca2d0 0x7f08e0153a00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.784+0000 7f08e75d0640 1 --2- 172.21.15.76:0/3548940017 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08e00c98e0 0x7f08e0142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:42.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.784+0000 7f08e75d0640 1 -- 172.21.15.76:0/3548940017 >> 172.21.15.76:0/3548940017 conn(0x7f08e00cf740 msgr2=0x7f08e0154f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:42.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.784+0000 7f08e75d0640 1 -- 172.21.15.76:0/3548940017 shutdown_connections 2024-03-08T06:00:42.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:42.784+0000 7f08e75d0640 1 -- 172.21.15.76:0/3548940017 wait complete. 2024-03-08T06:00:43.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-08T06:00:35.504231+0000', 'modified': '2024-03-08T06:00:40.552708+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.81:6835/2357927814', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6833', 'nonce': 2357927814}, {'type': 'v1', 'addr': '172.21.15.81:6835', 'nonce': 2357927814}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-03-08T06:00:43.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-08T06:00:43.198 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-08T06:00:43.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.302+0000 7fabe3888640 1 Processor -- start 2024-03-08T06:00:43.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.302+0000 7fabe3888640 1 -- start start 2024-03-08T06:00:43.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.303+0000 7fabe3888640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabdc0b5cf0 0x7fabdc168840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.303+0000 7fabe3888640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabdc0d27d0 con 0x7fabdc0b9840 2024-03-08T06:00:43.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.303+0000 7fabe3888640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabdc0d2ab0 con 0x7fabdc15dbc0 2024-03-08T06:00:43.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.303+0000 7fabe3888640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0d2d90 con 0x7fabdc0b5cf0 2024-03-08T06:00:43.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.303+0000 7fabe0e23640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fabdc0b9840 0x7fabdc0b55e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33810/0 (socket says 172.21.15.76:33810) 2024-03-08T06:00:43.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.303+0000 7fabe0e23640 1 -- 172.21.15.76:0/322809792 learned_addr learned my addr 172.21.15.76:0/322809792 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:43.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.303+0000 7fabd3fff640 1 -- 172.21.15.76:0/322809792 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 526350376 0 0) 0x7fabdc0d27d0 con 0x7fabdc0b9840 2024-03-08T06:00:43.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.303+0000 7fabd3fff640 1 -- 172.21.15.76:0/322809792 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabc8003520 con 0x7fabdc0b9840 2024-03-08T06:00:43.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.303+0000 7fabe1e25640 1 --2- 172.21.15.76:0/322809792 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabdc0b5cf0 0x7fabdc168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabd3fff640 1 -- 172.21.15.76:0/322809792 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 524743113 0 0) 0x7fabc8003520 con 0x7fabdc0b9840 2024-03-08T06:00:43.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabd3fff640 1 -- 172.21.15.76:0/322809792 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabdc0b5cf0 msgr2=0x7fabdc168840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:43.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabd3fff640 1 --2- 172.21.15.76:0/322809792 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabdc0b5cf0 0x7fabdc168840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabd3fff640 1 -- 172.21.15.76:0/322809792 >> v1:172.21.15.81:6789/0 conn(0x7fabdc15dbc0 legacy=0x7fabdc0d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:43.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabd3fff640 1 -- 172.21.15.76:0/322809792 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc0d3070 con 0x7fabdc0b9840 2024-03-08T06:00:43.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabd3fff640 1 -- 172.21.15.76:0/322809792 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fabcc001670 con 0x7fabdc0b9840 2024-03-08T06:00:43.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabd3fff640 1 -- 172.21.15.76:0/322809792 >> v1:172.21.15.76:6789/0 conn(0x7fabdc0b9840 legacy=0x7fabdc0b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:43.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabd3fff640 1 --2- 172.21.15.76:0/322809792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc8003a50 0x7fabc8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabd3fff640 1 --2- 172.21.15.76:0/322809792 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabdc0b5cf0 0x7fabc8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabd3fff640 1 --2- 172.21.15.76:0/322809792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabc8004d40 0x7fabc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabd3fff640 1 -- 172.21.15.76:0/322809792 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0d2d90 con 0x7fabc8004d40 2024-03-08T06:00:43.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabd3fff640 1 -- 172.21.15.76:0/322809792 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc800bb00 con 0x7fabc8003a50 2024-03-08T06:00:43.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabd3fff640 1 -- 172.21.15.76:0/322809792 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc800bd00 con 0x7fabdc0b5cf0 2024-03-08T06:00:43.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabe1e25640 1 --2- 172.21.15.76:0/322809792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabc8004d40 0x7fabc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabe0e23640 1 --2- 172.21.15.76:0/322809792 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabdc0b5cf0 0x7fabc8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabe1624640 1 --2- 172.21.15.76:0/322809792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc8003a50 0x7fabc8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabe1e25640 1 -- 172.21.15.76:0/322809792 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabdc0b5cf0 msgr2=0x7fabc8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabe1e25640 1 --2- 172.21.15.76:0/322809792 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabdc0b5cf0 0x7fabc8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabe1e25640 1 -- 172.21.15.76:0/322809792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc8003a50 msgr2=0x7fabc8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabe1e25640 1 --2- 172.21.15.76:0/322809792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc8003a50 0x7fabc8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.304+0000 7fabe1e25640 1 -- 172.21.15.76:0/322809792 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc8003520 con 0x7fabc8004d40 2024-03-08T06:00:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.305+0000 7fabe1e25640 1 --2- 172.21.15.76:0/322809792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabc8004d40 0x7fabc80051c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0b9cd0 tx=0x7fabc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5d0f25dda781bd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.305+0000 7fabd3fff640 1 -- 172.21.15.76:0/322809792 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8039070 con 0x7fabc8004d40 2024-03-08T06:00:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.305+0000 7fabd3fff640 1 -- 172.21.15.76:0/322809792 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd802cb90 con 0x7fabc8004d40 2024-03-08T06:00:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.305+0000 7fabd3fff640 1 -- 172.21.15.76:0/322809792 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8009c10 con 0x7fabc8004d40 2024-03-08T06:00:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.305+0000 7fabe3888640 1 -- 172.21.15.76:0/322809792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabc8004d40 msgr2=0x7fabc80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.305+0000 7fabe3888640 1 --2- 172.21.15.76:0/322809792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabc8004d40 0x7fabc80051c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0b9cd0 tx=0x7fabc8005830 comp rx=0 tx=0).stop 2024-03-08T06:00:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.305+0000 7fabe1e25640 1 -- 172.21.15.76:0/322809792 reap_dead start 2024-03-08T06:00:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.306+0000 7fabe3888640 1 -- 172.21.15.76:0/322809792 shutdown_connections 2024-03-08T06:00:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.306+0000 7fabe3888640 1 -- 172.21.15.76:0/322809792 >> 172.21.15.76:0/322809792 conn(0x7fabdc0670d0 msgr2=0x7fabdc0d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:43.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.306+0000 7fabe3888640 1 -- 172.21.15.76:0/322809792 shutdown_connections 2024-03-08T06:00:43.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.306+0000 7fabe3888640 1 -- 172.21.15.76:0/322809792 wait complete. 2024-03-08T06:00:43.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.307+0000 7fabe3888640 1 Processor -- start 2024-03-08T06:00:43.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.307+0000 7fabe3888640 1 -- start start 2024-03-08T06:00:43.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.307+0000 7fabe3888640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabc8004d40 0x7fabdc1e3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.307+0000 7fabe3888640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc159c10 0x7fabdc0cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.307+0000 7fabe3888640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabdc1f2eb0 0x7fabdc1f3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.307+0000 7fabe3888640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc15e120 con 0x7fabdc159c10 2024-03-08T06:00:43.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.307+0000 7fabe3888640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc15dfa0 con 0x7fabdc1f2eb0 2024-03-08T06:00:43.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.307+0000 7fabe3888640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0b7ca0 con 0x7fabc8004d40 2024-03-08T06:00:43.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabe0e23640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc159c10 0x7fabdc0cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabe0e23640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc159c10 0x7fabdc0cd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35018/0 (socket says 172.21.15.76:35018) 2024-03-08T06:00:43.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabe0e23640 1 -- 172.21.15.76:0/2440113388 learned_addr learned my addr 172.21.15.76:0/2440113388 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:43.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabe1624640 1 --2- 172.21.15.76:0/2440113388 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabc8004d40 0x7fabdc1e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabe1e25640 1 --2- 172.21.15.76:0/2440113388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabdc1f2eb0 0x7fabdc1f3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabe0e23640 1 -- 172.21.15.76:0/2440113388 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabc8004d40 msgr2=0x7fabdc1e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:43.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabe0e23640 1 --2- 172.21.15.76:0/2440113388 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabc8004d40 0x7fabdc1e3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabe0e23640 1 -- 172.21.15.76:0/2440113388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabdc1f2eb0 msgr2=0x7fabdc1f3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:43.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabe0e23640 1 --2- 172.21.15.76:0/2440113388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabdc1f2eb0 0x7fabdc1f3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabe0e23640 1 -- 172.21.15.76:0/2440113388 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc14ab70 con 0x7fabdc159c10 2024-03-08T06:00:43.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabe1e25640 1 --2- 172.21.15.76:0/2440113388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabdc1f2eb0 0x7fabdc1f3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:43.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabe1624640 1 --2- 172.21.15.76:0/2440113388 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabc8004d40 0x7fabdc1e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:43.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabe0e23640 1 --2- 172.21.15.76:0/2440113388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc159c10 0x7fabdc0cd070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fabcc007340 tx=0x7fabcc007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:43.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabd1ffb640 1 -- 172.21.15.76:0/2440113388 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc00f8c0 con 0x7fabdc159c10 2024-03-08T06:00:43.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabd1ffb640 1 -- 172.21.15.76:0/2440113388 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc018070 con 0x7fabdc159c10 2024-03-08T06:00:43.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabe3888640 1 -- 172.21.15.76:0/2440113388 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabdc14ae10 con 0x7fabdc159c10 2024-03-08T06:00:43.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabd1ffb640 1 -- 172.21.15.76:0/2440113388 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc00fca0 con 0x7fabdc159c10 2024-03-08T06:00:43.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.308+0000 7fabe3888640 1 -- 172.21.15.76:0/2440113388 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabdc14b3d0 con 0x7fabdc159c10 2024-03-08T06:00:43.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.310+0000 7fabd1ffb640 1 -- 172.21.15.76:0/2440113388 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fabcc01d460 con 0x7fabdc159c10 2024-03-08T06:00:43.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.310+0000 7fabd1ffb640 1 --2- 172.21.15.76:0/2440113388 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fabc006b500 0x7fabc006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.310+0000 7fabd1ffb640 1 -- 172.21.15.76:0/2440113388 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fabcc08a4f0 con 0x7fabdc159c10 2024-03-08T06:00:43.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.310+0000 7fabe1624640 1 --2- 172.21.15.76:0/2440113388 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fabc006b500 0x7fabc006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.310+0000 7fabe1624640 1 --2- 172.21.15.76:0/2440113388 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fabc006b500 0x7fabc006d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fabd40035f0 tx=0x7fabd4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:43.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.311+0000 7fabe646f740 1 Processor -- start 2024-03-08T06:00:43.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.311+0000 7fabe646f740 1 -- start start 2024-03-08T06:00:43.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe646f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b7898870 0x55e8b7898c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe646f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b789c8c0 0x55e8b789e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe646f740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c8600 0x55e8b78c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe646f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b78026b0 con 0x55e8b789c8c0 2024-03-08T06:00:43.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe646f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b77aba60 con 0x55e8b7898870 2024-03-08T06:00:43.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe646f740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8b77fe750 con 0x55e8b78c8600 2024-03-08T06:00:43.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe0e23640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b789c8c0 0x55e8b789e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe0e23640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b789c8c0 0x55e8b789e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35024/0 (socket says 172.21.15.76:35024) 2024-03-08T06:00:43.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe0e23640 1 -- 172.21.15.76:0/3958641667 learned_addr learned my addr 172.21.15.76:0/3958641667 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:43.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe1e25640 1 --2- 172.21.15.76:0/3958641667 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c8600 0x55e8b78c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe1624640 1 --2- 172.21.15.76:0/3958641667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b7898870 0x55e8b7898c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe0e23640 1 -- 172.21.15.76:0/3958641667 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c8600 msgr2=0x55e8b78c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:43.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe0e23640 1 --2- 172.21.15.76:0/3958641667 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c8600 0x55e8b78c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe0e23640 1 -- 172.21.15.76:0/3958641667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b7898870 msgr2=0x55e8b7898c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:43.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe0e23640 1 --2- 172.21.15.76:0/3958641667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b7898870 0x55e8b7898c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe0e23640 1 -- 172.21.15.76:0/3958641667 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8b77c9f30 con 0x55e8b789c8c0 2024-03-08T06:00:43.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabe0e23640 1 --2- 172.21.15.76:0/3958641667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b789c8c0 0x55e8b789e360 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0886e0 tx=0x7fabcc058a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fd61e5b059c44d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:43.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabc6ffd640 1 -- 172.21.15.76:0/3958641667 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc056b20 con 0x55e8b789c8c0 2024-03-08T06:00:43.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabc6ffd640 1 -- 172.21.15.76:0/3958641667 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc085070 con 0x55e8b789c8c0 2024-03-08T06:00:43.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.312+0000 7fabc6ffd640 1 -- 172.21.15.76:0/3958641667 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc0550f0 con 0x55e8b789c8c0 2024-03-08T06:00:43.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.313+0000 7fabe646f740 1 -- 172.21.15.76:0/3958641667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b789c8c0 msgr2=0x55e8b789e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:43.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.313+0000 7fabe646f740 1 --2- 172.21.15.76:0/3958641667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b789c8c0 0x55e8b789e360 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0886e0 tx=0x7fabcc058a80 comp rx=0 tx=0).stop 2024-03-08T06:00:43.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.313+0000 7fabe646f740 1 -- 172.21.15.76:0/3958641667 shutdown_connections 2024-03-08T06:00:43.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.313+0000 7fabe646f740 1 --2- 172.21.15.76:0/3958641667 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c8600 0x55e8b78c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.313+0000 7fabe646f740 1 --2- 172.21.15.76:0/3958641667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b789c8c0 0x55e8b789e360 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.313+0000 7fabe646f740 1 --2- 172.21.15.76:0/3958641667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b7898870 0x55e8b7898c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.313+0000 7fabe646f740 1 -- 172.21.15.76:0/3958641667 >> 172.21.15.76:0/3958641667 conn(0x55e8b789a8f0 msgr2=0x55e8b789c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:43.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.313+0000 7fabe646f740 1 -- 172.21.15.76:0/3958641667 shutdown_connections 2024-03-08T06:00:43.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.313+0000 7fabe646f740 1 -- 172.21.15.76:0/3958641667 wait complete. 2024-03-08T06:00:43.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.314+0000 7fabe646f740 1 -- start start 2024-03-08T06:00:43.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.314+0000 7fabe646f740 1 Processor -- start 2024-03-08T06:00:43.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.314+0000 7fabe646f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c8600 0x55e8b78e1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe646f740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c33d0 0x55e8b78e16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe646f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c37d0 0x55e8b78de8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe646f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b789cd50 con 0x55e8b78c37d0 2024-03-08T06:00:43.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe646f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b77beea0 con 0x55e8b78c8600 2024-03-08T06:00:43.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe646f740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8b789d030 con 0x55e8b78c33d0 2024-03-08T06:00:43.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe646f740 10 client.0 _ll_register_callbacks cb 0x55e8b77be870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:00:43.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe1e25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c37d0 0x55e8b78de8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe1e25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c37d0 0x55e8b78de8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35036/0 (socket says 172.21.15.76:35036) 2024-03-08T06:00:43.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe1e25640 1 -- 172.21.15.76:0/388596976 learned_addr learned my addr 172.21.15.76:0/388596976 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:43.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe0e23640 1 --2- 172.21.15.76:0/388596976 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c33d0 0x55e8b78e16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe1624640 1 --2- 172.21.15.76:0/388596976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c8600 0x55e8b78e1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe1e25640 1 -- 172.21.15.76:0/388596976 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c33d0 msgr2=0x55e8b78e16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:43.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe1e25640 1 --2- 172.21.15.76:0/388596976 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c33d0 0x55e8b78e16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe1e25640 1 -- 172.21.15.76:0/388596976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c8600 msgr2=0x55e8b78e1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:43.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe1e25640 1 --2- 172.21.15.76:0/388596976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c8600 0x55e8b78e1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe1e25640 1 -- 172.21.15.76:0/388596976 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8b77bf270 con 0x55e8b78c37d0 2024-03-08T06:00:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe1624640 1 --2- 172.21.15.76:0/388596976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c8600 0x55e8b78e1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:00:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe1e25640 1 --2- 172.21.15.76:0/388596976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c37d0 0x55e8b78de8d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fabd800e200 tx=0x7fabd8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfcbac882a2c8ae2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-08T06:00:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabbbfff640 1 -- 172.21.15.76:0/388596976 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8045060 con 0x55e8b78c37d0 2024-03-08T06:00:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabbbfff640 1 -- 172.21.15.76:0/388596976 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fabd802cb90 con 0x55e8b78c37d0 2024-03-08T06:00:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe646f740 1 -- 172.21.15.76:0/388596976 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fabcc085070 con 0x55e8b78c37d0 2024-03-08T06:00:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabbbfff640 1 -- 172.21.15.76:0/388596976 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabd804b050 con 0x55e8b78c37d0 2024-03-08T06:00:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe646f740 10 client.4431 fetch_fsmap learned FSMap version 6 2024-03-08T06:00:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabe646f740 1 -- 172.21.15.76:0/388596976 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55e8b75172d0 con 0x55e8b78c37d0 2024-03-08T06:00:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.315+0000 7fabbbfff640 1 -- 172.21.15.76:0/388596976 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc085070 con 0x55e8b78c37d0 2024-03-08T06:00:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.316+0000 7fabbbfff640 1 -- 172.21.15.76:0/388596976 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 6) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fabd804e020 con 0x55e8b78c37d0 2024-03-08T06:00:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.316+0000 7fabe646f740 10 client.4431 fetch_fsmap finished waiting for FSMap version 6 2024-03-08T06:00:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.316+0000 7fabe646f740 10 client.4431 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-08T06:00:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.316+0000 7fabe646f740 20 client.4431 populate_metadata read hostname 'smithi076' 2024-03-08T06:00:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.316+0000 7fabe646f740 1 --2- 172.21.15.76:0/388596976 >> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] conn(0x55e8b78e4040 0x55e8b79029d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.316+0000 7fabe646f740 4 client.4431 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T06:00:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.316+0000 7fabe646f740 1 -- 172.21.15.76:0/388596976 --> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8b780fbf0 con 0x55e8b78e4040 2024-03-08T06:00:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.316+0000 7fabe1624640 1 --2- 172.21.15.76:0/388596976 >> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] conn(0x55e8b78e4040 0x55e8b79029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.316+0000 7fabbbfff640 10 client.4431 ms_handle_connect on v2:172.21.15.81:6833/2357927814 2024-03-08T06:00:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.316+0000 7fabe1624640 1 --2- 172.21.15.76:0/388596976 >> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] conn(0x55e8b78e4040 0x55e8b79029d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2923c4b8b201f454 server_cookie=288324df51c77250 in_seq=0 out_seq=0 2024-03-08T06:00:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.317+0000 7fabbbfff640 1 -- 172.21.15.76:0/388596976 <== mds.0 v2:172.21.15.81:6833/2357927814 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e8b780fbf0 con 0x55e8b78e4040 2024-03-08T06:00:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.317+0000 7fabbbfff640 10 client.4431 handle_command_reply: tid=0 2024-03-08T06:00:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.317+0000 7fabe646f740 1 client.4431 shutdown 2024-03-08T06:00:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.317+0000 7fabe646f740 1 -- 172.21.15.76:0/388596976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c37d0 msgr2=0x55e8b78de8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.317+0000 7fabe646f740 1 --2- 172.21.15.76:0/388596976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c37d0 0x55e8b78de8d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fabd800e200 tx=0x7fabd8004ab0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.317+0000 7fabe646f740 1 -- 172.21.15.76:0/388596976 shutdown_connections 2024-03-08T06:00:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.317+0000 7fabe646f740 1 --2- 172.21.15.76:0/388596976 >> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] conn(0x55e8b78e4040 0x55e8b79029d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.317+0000 7fabe646f740 1 --2- 172.21.15.76:0/388596976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c37d0 0x55e8b78de8d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.317+0000 7fabe646f740 1 --2- 172.21.15.76:0/388596976 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c33d0 0x55e8b78e16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.317+0000 7fabbbfff640 0 client.4431 ms_handle_reset on v2:172.21.15.81:6833/2357927814 2024-03-08T06:00:43.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.317+0000 7fabe646f740 1 --2- 172.21.15.76:0/388596976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c8600 0x55e8b78e1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.317+0000 7fabe646f740 1 -- 172.21.15.76:0/388596976 >> 172.21.15.76:0/388596976 conn(0x55e8b7898870 msgr2=0x55e8b7897510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:43.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.317+0000 7fabe646f740 1 -- 172.21.15.76:0/388596976 shutdown_connections 2024-03-08T06:00:43.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.317+0000 7fabe646f740 1 -- 172.21.15.76:0/388596976 wait complete. 2024-03-08T06:00:43.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.317+0000 7fabe646f740 20 client.4431 trim_cache size 0 max 16384 2024-03-08T06:00:43.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.333+0000 7fabe646f740 1 Processor -- start 2024-03-08T06:00:43.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.333+0000 7fabe646f740 1 -- start start 2024-03-08T06:00:43.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe646f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c33d0 0x55e8b7904e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe646f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c37d0 0x55e8b78c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe646f740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c8600 0x55e8b78c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe646f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc201d90 con 0x55e8b78c33d0 2024-03-08T06:00:43.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe646f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc200fa0 con 0x55e8b78c37d0 2024-03-08T06:00:43.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe646f740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc14ce30 con 0x55e8b78c8600 2024-03-08T06:00:43.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe1624640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c33d0 0x55e8b7904e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe1624640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c33d0 0x55e8b7904e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35042/0 (socket says 172.21.15.76:35042) 2024-03-08T06:00:43.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe1624640 1 -- 172.21.15.76:0/3743675933 learned_addr learned my addr 172.21.15.76:0/3743675933 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:43.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe1e25640 1 --2- 172.21.15.76:0/3743675933 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c8600 0x55e8b78c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe0e23640 1 --2- 172.21.15.76:0/3743675933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c37d0 0x55e8b78c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe1624640 1 -- 172.21.15.76:0/3743675933 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c8600 msgr2=0x55e8b78c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:43.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe1624640 1 --2- 172.21.15.76:0/3743675933 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c8600 0x55e8b78c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe1624640 1 -- 172.21.15.76:0/3743675933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c37d0 msgr2=0x55e8b78c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:43.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe1624640 1 --2- 172.21.15.76:0/3743675933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c37d0 0x55e8b78c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe1624640 1 -- 172.21.15.76:0/3743675933 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8b75e16b0 con 0x55e8b78c33d0 2024-03-08T06:00:43.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe1e25640 1 --2- 172.21.15.76:0/3743675933 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c8600 0x55e8b78c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:43.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe0e23640 1 --2- 172.21.15.76:0/3743675933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c37d0 0x55e8b78c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:43.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabe1624640 1 --2- 172.21.15.76:0/3743675933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c33d0 0x55e8b7904e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55e8b78e39a0 tx=0x7fabd4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd342b0c631ea661 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:43.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabc67fc640 1 -- 172.21.15.76:0/3743675933 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd401a070 con 0x55e8b78c33d0 2024-03-08T06:00:43.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabc67fc640 1 -- 172.21.15.76:0/3743675933 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd402b020 con 0x55e8b78c33d0 2024-03-08T06:00:43.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.334+0000 7fabc67fc640 1 -- 172.21.15.76:0/3743675933 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4025b50 con 0x55e8b78c33d0 2024-03-08T06:00:43.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.335+0000 7fabe646f740 1 -- 172.21.15.76:0/3743675933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c33d0 msgr2=0x55e8b7904e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:43.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.335+0000 7fabe646f740 1 --2- 172.21.15.76:0/3743675933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c33d0 0x55e8b7904e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55e8b78e39a0 tx=0x7fabd4013000 comp rx=0 tx=0).stop 2024-03-08T06:00:43.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.335+0000 7fabe646f740 1 -- 172.21.15.76:0/3743675933 shutdown_connections 2024-03-08T06:00:43.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.335+0000 7fabe646f740 1 --2- 172.21.15.76:0/3743675933 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c8600 0x55e8b78c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.335+0000 7fabe646f740 1 --2- 172.21.15.76:0/3743675933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c37d0 0x55e8b78c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.335+0000 7fabe646f740 1 --2- 172.21.15.76:0/3743675933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c33d0 0x55e8b7904e10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.335+0000 7fabe646f740 1 -- 172.21.15.76:0/3743675933 >> 172.21.15.76:0/3743675933 conn(0x55e8b7898870 msgr2=0x55e8b78ca3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:43.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.335+0000 7fabe646f740 1 -- 172.21.15.76:0/3743675933 shutdown_connections 2024-03-08T06:00:43.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.335+0000 7fabe646f740 1 -- 172.21.15.76:0/3743675933 wait complete. 2024-03-08T06:00:43.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.335+0000 7fabe646f740 1 -- start start 2024-03-08T06:00:43.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.335+0000 7fabe646f740 1 Processor -- start 2024-03-08T06:00:43.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.336+0000 7fabe646f740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c33d0 0x55e8b78cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.336+0000 7fabe646f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c37d0 0x55e8b78cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.336+0000 7fabe646f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c8600 0x55e8b78e6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.336+0000 7fabe646f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b78cb440 con 0x55e8b78c37d0 2024-03-08T06:00:43.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.336+0000 7fabe646f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc2009a0 con 0x55e8b78c8600 2024-03-08T06:00:43.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.336+0000 7fabe646f740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8b78df950 con 0x55e8b78c33d0 2024-03-08T06:00:43.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.336+0000 7fabe646f740 10 client.0 _ll_register_callbacks cb 0x7fabd8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:00:43.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.336+0000 7fabe0e23640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c37d0 0x55e8b78cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.336+0000 7fabe0e23640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c37d0 0x55e8b78cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35050/0 (socket says 172.21.15.76:35050) 2024-03-08T06:00:43.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.336+0000 7fabe0e23640 1 -- 172.21.15.76:0/2655574171 learned_addr learned my addr 172.21.15.76:0/2655574171 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:43.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe1624640 1 --2- 172.21.15.76:0/2655574171 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c33d0 0x55e8b78cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe1e25640 1 --2- 172.21.15.76:0/2655574171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c8600 0x55e8b78e6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe0e23640 1 -- 172.21.15.76:0/2655574171 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c33d0 msgr2=0x55e8b78cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:43.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe0e23640 1 --2- 172.21.15.76:0/2655574171 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c33d0 0x55e8b78cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe0e23640 1 -- 172.21.15.76:0/2655574171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c8600 msgr2=0x55e8b78e6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:43.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe0e23640 1 --2- 172.21.15.76:0/2655574171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c8600 0x55e8b78e6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe0e23640 1 -- 172.21.15.76:0/2655574171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8b78e1300 con 0x55e8b78c37d0 2024-03-08T06:00:43.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe1624640 1 --2- 172.21.15.76:0/2655574171 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c33d0 0x55e8b78cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:43.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe1e25640 1 --2- 172.21.15.76:0/2655574171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c8600 0x55e8b78e6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:43.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe0e23640 1 --2- 172.21.15.76:0/2655574171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c37d0 0x55e8b78cc020 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fabcc058780 tx=0x7fabcc015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7acc97de3117833 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:43.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-08T06:00:43.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabbbfff640 1 -- 172.21.15.76:0/2655574171 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc061190 con 0x55e8b78c37d0 2024-03-08T06:00:43.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabbbfff640 1 -- 172.21.15.76:0/2655574171 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fabcc088380 con 0x55e8b78c37d0 2024-03-08T06:00:43.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe646f740 1 -- 172.21.15.76:0/2655574171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fabd402b020 con 0x55e8b78c37d0 2024-03-08T06:00:43.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabbbfff640 1 -- 172.21.15.76:0/2655574171 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabcc07e050 con 0x55e8b78c37d0 2024-03-08T06:00:43.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe646f740 10 client.4437 fetch_fsmap learned FSMap version 6 2024-03-08T06:00:43.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe646f740 1 -- 172.21.15.76:0/2655574171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55e8b78e1920 con 0x55e8b78c37d0 2024-03-08T06:00:43.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabbbfff640 1 -- 172.21.15.76:0/2655574171 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd402b020 con 0x55e8b78c37d0 2024-03-08T06:00:43.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabbbfff640 1 -- 172.21.15.76:0/2655574171 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 6) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fabcc081020 con 0x55e8b78c37d0 2024-03-08T06:00:43.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe646f740 10 client.4437 fetch_fsmap finished waiting for FSMap version 6 2024-03-08T06:00:43.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe646f740 10 client.4437 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-08T06:00:43.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe646f740 20 client.4437 populate_metadata read hostname 'smithi076' 2024-03-08T06:00:43.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe646f740 1 --2- 172.21.15.76:0/2655574171 >> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] conn(0x55e8b78e7250 0x55e8b78bffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe646f740 4 client.4437 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T06:00:43.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe646f740 1 -- 172.21.15.76:0/2655574171 --> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e8b789b050 con 0x55e8b78e7250 2024-03-08T06:00:43.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.337+0000 7fabe1624640 1 --2- 172.21.15.76:0/2655574171 >> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] conn(0x55e8b78e7250 0x55e8b78bffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabbbfff640 10 client.4437 ms_handle_connect on v2:172.21.15.81:6833/2357927814 2024-03-08T06:00:43.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabe1624640 1 --2- 172.21.15.76:0/2655574171 >> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] conn(0x55e8b78e7250 0x55e8b78bffc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e9b75aae645280 server_cookie=eead3fb4a5b18ea2 in_seq=0 out_seq=0 2024-03-08T06:00:43.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabbbfff640 1 -- 172.21.15.76:0/2655574171 <== mds.0 v2:172.21.15.81:6833/2357927814 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x55e8b789b050 con 0x55e8b78e7250 2024-03-08T06:00:43.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabbbfff640 10 client.4437 handle_command_reply: tid=0 2024-03-08T06:00:43.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabe646f740 1 client.4437 shutdown 2024-03-08T06:00:43.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabe646f740 1 -- 172.21.15.76:0/2655574171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c37d0 msgr2=0x55e8b78cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:43.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabe646f740 1 --2- 172.21.15.76:0/2655574171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c37d0 0x55e8b78cc020 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fabcc058780 tx=0x7fabcc015040 comp rx=0 tx=0).stop 2024-03-08T06:00:43.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabe646f740 1 -- 172.21.15.76:0/2655574171 shutdown_connections 2024-03-08T06:00:43.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabe646f740 1 --2- 172.21.15.76:0/2655574171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8b78c8600 0x55e8b78e6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabe646f740 1 --2- 172.21.15.76:0/2655574171 >> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] conn(0x55e8b78e7250 0x55e8b78bffc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabe646f740 1 --2- 172.21.15.76:0/2655574171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8b78c37d0 0x55e8b78cc020 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabe646f740 1 --2- 172.21.15.76:0/2655574171 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55e8b78c33d0 0x55e8b78cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabbbfff640 0 client.4437 ms_handle_reset on v2:172.21.15.81:6833/2357927814 2024-03-08T06:00:43.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabe646f740 1 -- 172.21.15.76:0/2655574171 >> 172.21.15.76:0/2655574171 conn(0x55e8b7898870 msgr2=0x55e8b78c1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:43.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabe646f740 1 -- 172.21.15.76:0/2655574171 shutdown_connections 2024-03-08T06:00:43.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabe646f740 1 -- 172.21.15.76:0/2655574171 wait complete. 2024-03-08T06:00:43.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.338+0000 7fabe646f740 20 client.4437 trim_cache size 0 max 16384 2024-03-08T06:00:43.346 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-08T06:00:43.346 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "8a42235b-b7c3-45b2-aa98-336e6b1846d1", 2024-03-08T06:00:43.346 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-03-08T06:00:43.347 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4285, 2024-03-08T06:00:43.347 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-03-08T06:00:43.347 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-03-08T06:00:43.347 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-03-08T06:00:43.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 7.1608180639999999, 2024-03-08T06:00:43.347 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 6, 2024-03-08T06:00:43.347 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 26, 2024-03-08T06:00:43.347 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 25, 2024-03-08T06:00:43.347 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 13.80944418 2024-03-08T06:00:43.347 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-08T06:00:43.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.339+0000 7fabe3888640 1 -- 172.21.15.76:0/2440113388 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fabc006b500 msgr2=0x7fabc006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:43.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.339+0000 7fabe3888640 1 --2- 172.21.15.76:0/2440113388 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fabc006b500 0x7fabc006d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fabd40035f0 tx=0x7fabd4009040 comp rx=0 tx=0).stop 2024-03-08T06:00:43.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.339+0000 7fabe3888640 1 -- 172.21.15.76:0/2440113388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc159c10 msgr2=0x7fabdc0cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:43.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.339+0000 7fabe3888640 1 --2- 172.21.15.76:0/2440113388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc159c10 0x7fabdc0cd070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fabcc007340 tx=0x7fabcc007800 comp rx=0 tx=0).stop 2024-03-08T06:00:43.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.339+0000 7fabe3888640 1 -- 172.21.15.76:0/2440113388 shutdown_connections 2024-03-08T06:00:43.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.339+0000 7fabe3888640 1 --2- 172.21.15.76:0/2440113388 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fabc006b500 0x7fabc006d9c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.339+0000 7fabe3888640 1 --2- 172.21.15.76:0/2440113388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabdc1f2eb0 0x7fabdc1f3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.339+0000 7fabe3888640 1 --2- 172.21.15.76:0/2440113388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc159c10 0x7fabdc0cd070 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.339+0000 7fabe3888640 1 --2- 172.21.15.76:0/2440113388 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabc8004d40 0x7fabdc1e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.339+0000 7fabe3888640 1 -- 172.21.15.76:0/2440113388 >> 172.21.15.76:0/2440113388 conn(0x7fabdc0670d0 msgr2=0x7fabdc0c7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:43.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.339+0000 7fabe3888640 1 -- 172.21.15.76:0/2440113388 shutdown_connections 2024-03-08T06:00:43.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.339+0000 7fabe3888640 1 -- 172.21.15.76:0/2440113388 wait complete. 2024-03-08T06:00:43.350 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-08T06:00:43.350 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-08T06:00:43.350 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T06:00:43.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.449+0000 7fe3ac0f4640 1 Processor -- start 2024-03-08T06:00:43.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.449+0000 7fe3ac0f4640 1 -- start start 2024-03-08T06:00:43.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.450+0000 7fe3ac0f4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe3a41584b0 0x7fe3a415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.450+0000 7fe3ac0f4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe3a40b2ea0 0x7fe3a406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.450+0000 7fe3ac0f4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3a406c1d0 0x7fe3a4161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.450+0000 7fe3ac0f4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a406fb40 con 0x7fe3a41584b0 2024-03-08T06:00:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.450+0000 7fe3ac0f4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a406c5e0 con 0x7fe3a40b2ea0 2024-03-08T06:00:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.450+0000 7fe3ac0f4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a415f200 con 0x7fe3a406c1d0 2024-03-08T06:00:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.450+0000 7fe3a9e90640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe3a41584b0 0x7fe3a415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.450+0000 7fe3a9e90640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe3a41584b0 0x7fe3a415ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35054/0 (socket says 172.21.15.76:35054) 2024-03-08T06:00:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.450+0000 7fe3a9e90640 1 -- 172.21.15.76:0/1754869857 learned_addr learned my addr 172.21.15.76:0/1754869857 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.450+0000 7fe3aa691640 1 --2- 172.21.15.76:0/1754869857 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3a406c1d0 0x7fe3a4161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.450+0000 7fe3a968f640 1 --2- 172.21.15.76:0/1754869857 >> v2:172.21.15.81:3300/0 conn(0x7fe3a40b2ea0 0x7fe3a406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.451+0000 7fe3a9e90640 1 -- 172.21.15.76:0/1754869857 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3a406c1d0 msgr2=0x7fe3a4161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.451+0000 7fe3a9e90640 1 --2- 172.21.15.76:0/1754869857 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3a406c1d0 0x7fe3a4161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.451+0000 7fe3a9e90640 1 -- 172.21.15.76:0/1754869857 >> v2:172.21.15.81:3300/0 conn(0x7fe3a40b2ea0 msgr2=0x7fe3a406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.451+0000 7fe3a9e90640 1 --2- 172.21.15.76:0/1754869857 >> v2:172.21.15.81:3300/0 conn(0x7fe3a40b2ea0 0x7fe3a406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.451+0000 7fe3a9e90640 1 -- 172.21.15.76:0/1754869857 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a40b39c0 con 0x7fe3a41584b0 2024-03-08T06:00:43.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.451+0000 7fe3aa691640 1 --2- 172.21.15.76:0/1754869857 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3a406c1d0 0x7fe3a4161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:00:43.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.451+0000 7fe3a968f640 1 --2- 172.21.15.76:0/1754869857 >> v2:172.21.15.81:3300/0 conn(0x7fe3a40b2ea0 0x7fe3a406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:00:43.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.451+0000 7fe3a9e90640 1 --2- 172.21.15.76:0/1754869857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a41584b0 0x7fe3a415ecc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe3a40cbfc0 tx=0x7fe3a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8a4f1d64785fa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:43.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.452+0000 7fe3a8e8e640 1 -- 172.21.15.76:0/1754869857 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0035020 con 0x7fe3a41584b0 2024-03-08T06:00:43.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.452+0000 7fe3a8e8e640 1 -- 172.21.15.76:0/1754869857 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a0031870 con 0x7fe3a41584b0 2024-03-08T06:00:43.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.452+0000 7fe3a8e8e640 1 -- 172.21.15.76:0/1754869857 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0031c40 con 0x7fe3a41584b0 2024-03-08T06:00:43.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.452+0000 7fe3ac0f4640 1 -- 172.21.15.76:0/1754869857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a41584b0 msgr2=0x7fe3a415ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:43.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.452+0000 7fe3ac0f4640 1 --2- 172.21.15.76:0/1754869857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a41584b0 0x7fe3a415ecc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe3a40cbfc0 tx=0x7fe3a002b910 comp rx=0 tx=0).stop 2024-03-08T06:00:43.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.452+0000 7fe3ac0f4640 1 -- 172.21.15.76:0/1754869857 shutdown_connections 2024-03-08T06:00:43.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.452+0000 7fe3ac0f4640 1 --2- 172.21.15.76:0/1754869857 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3a406c1d0 0x7fe3a4161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.452+0000 7fe3ac0f4640 1 --2- 172.21.15.76:0/1754869857 >> v2:172.21.15.81:3300/0 conn(0x7fe3a40b2ea0 0x7fe3a406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.452+0000 7fe3ac0f4640 1 --2- 172.21.15.76:0/1754869857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a41584b0 0x7fe3a415ecc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.452+0000 7fe3ac0f4640 1 -- 172.21.15.76:0/1754869857 >> 172.21.15.76:0/1754869857 conn(0x7fe3a4071c30 msgr2=0x7fe3a4072010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.452+0000 7fe3ac0f4640 1 -- 172.21.15.76:0/1754869857 shutdown_connections 2024-03-08T06:00:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.452+0000 7fe3ac0f4640 1 -- 172.21.15.76:0/1754869857 wait complete. 2024-03-08T06:00:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.453+0000 7fe3ac0f4640 1 Processor -- start 2024-03-08T06:00:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.453+0000 7fe3ac0f4640 1 -- start start 2024-03-08T06:00:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3ac0f4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a406c1d0 0x7fe3a4151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3ac0f4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3a40b2ea0 0x7fe3a41569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3ac0f4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a41584b0 0x7fe3a4156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3ac0f4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a406dad0 con 0x7fe3a406c1d0 2024-03-08T06:00:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3ac0f4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a40b3f80 con 0x7fe3a41584b0 2024-03-08T06:00:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3ac0f4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a406ddb0 con 0x7fe3a40b2ea0 2024-03-08T06:00:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3a9e90640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a406c1d0 0x7fe3a4151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3a9e90640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a406c1d0 0x7fe3a4151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35068/0 (socket says 172.21.15.76:35068) 2024-03-08T06:00:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3a9e90640 1 -- 172.21.15.76:0/1902601147 learned_addr learned my addr 172.21.15.76:0/1902601147 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3a968f640 1 --2- 172.21.15.76:0/1902601147 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3a40b2ea0 0x7fe3a41569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3aa691640 1 --2- 172.21.15.76:0/1902601147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a41584b0 0x7fe3a4156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3a9e90640 1 -- 172.21.15.76:0/1902601147 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3a40b2ea0 msgr2=0x7fe3a41569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3a9e90640 1 --2- 172.21.15.76:0/1902601147 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3a40b2ea0 0x7fe3a41569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3a9e90640 1 -- 172.21.15.76:0/1902601147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a41584b0 msgr2=0x7fe3a4156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3a9e90640 1 --2- 172.21.15.76:0/1902601147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a41584b0 0x7fe3a4156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3a9e90640 1 -- 172.21.15.76:0/1902601147 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a415e2f0 con 0x7fe3a406c1d0 2024-03-08T06:00:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.454+0000 7fe3a9e90640 1 --2- 172.21.15.76:0/1902601147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a406c1d0 0x7fe3a4151190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe3a002b950 tx=0x7fe3a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.455+0000 7fe39a7fc640 1 -- 172.21.15.76:0/1902601147 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0040070 con 0x7fe3a406c1d0 2024-03-08T06:00:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.455+0000 7fe39a7fc640 1 -- 172.21.15.76:0/1902601147 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a0030d00 con 0x7fe3a406c1d0 2024-03-08T06:00:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.455+0000 7fe3ac0f4640 1 -- 172.21.15.76:0/1902601147 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a415c0c0 con 0x7fe3a406c1d0 2024-03-08T06:00:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.455+0000 7fe39a7fc640 1 -- 172.21.15.76:0/1902601147 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0035040 con 0x7fe3a406c1d0 2024-03-08T06:00:43.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.455+0000 7fe3ac0f4640 1 -- 172.21.15.76:0/1902601147 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a415cba0 con 0x7fe3a406c1d0 2024-03-08T06:00:43.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.456+0000 7fe3ac0f4640 1 -- 172.21.15.76:0/1902601147 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a4152130 con 0x7fe3a406c1d0 2024-03-08T06:00:43.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.456+0000 7fe39a7fc640 1 -- 172.21.15.76:0/1902601147 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3a004b060 con 0x7fe3a406c1d0 2024-03-08T06:00:43.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.456+0000 7fe39a7fc640 1 --2- 172.21.15.76:0/1902601147 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe38806b4d0 0x7fe38806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:43.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.457+0000 7fe39a7fc640 1 -- 172.21.15.76:0/1902601147 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe3a00ac2c0 con 0x7fe3a406c1d0 2024-03-08T06:00:43.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.457+0000 7fe3a968f640 1 --2- 172.21.15.76:0/1902601147 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe38806b4d0 0x7fe38806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:43.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.457+0000 7fe3a968f640 1 --2- 172.21.15.76:0/1902601147 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe38806b4d0 0x7fe38806d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4157ea0 tx=0x7fe394008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:43.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.461+0000 7fe39a7fc640 1 -- 172.21.15.76:0/1902601147 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe3a007ae90 con 0x7fe3a406c1d0 2024-03-08T06:00:43.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.666+0000 7fe3ac0f4640 1 -- 172.21.15.76:0/1902601147 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe3a415cdd0 con 0x7fe3a406c1d0 2024-03-08T06:00:43.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.666+0000 7fe39a7fc640 1 -- 172.21.15.76:0/1902601147 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7fe3a007a790 con 0x7fe3a406c1d0 2024-03-08T06:00:43.667 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-08T06:00:43.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.668+0000 7fe3ac0f4640 1 -- 172.21.15.76:0/1902601147 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe38806b4d0 msgr2=0x7fe38806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:43.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.668+0000 7fe3ac0f4640 1 --2- 172.21.15.76:0/1902601147 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe38806b4d0 0x7fe38806d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4157ea0 tx=0x7fe394008040 comp rx=0 tx=0).stop 2024-03-08T06:00:43.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.668+0000 7fe3ac0f4640 1 -- 172.21.15.76:0/1902601147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a406c1d0 msgr2=0x7fe3a4151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:43.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.668+0000 7fe3ac0f4640 1 --2- 172.21.15.76:0/1902601147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a406c1d0 0x7fe3a4151190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe3a002b950 tx=0x7fe3a00046c0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.668+0000 7fe3ac0f4640 1 -- 172.21.15.76:0/1902601147 shutdown_connections 2024-03-08T06:00:43.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.668+0000 7fe3ac0f4640 1 --2- 172.21.15.76:0/1902601147 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe38806b4d0 0x7fe38806d990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.668+0000 7fe3ac0f4640 1 --2- 172.21.15.76:0/1902601147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a41584b0 0x7fe3a4156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.668+0000 7fe3ac0f4640 1 --2- 172.21.15.76:0/1902601147 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3a40b2ea0 0x7fe3a41569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.668+0000 7fe3ac0f4640 1 --2- 172.21.15.76:0/1902601147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a406c1d0 0x7fe3a4151190 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:43.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.668+0000 7fe3ac0f4640 1 -- 172.21.15.76:0/1902601147 >> 172.21.15.76:0/1902601147 conn(0x7fe3a4071c30 msgr2=0x7fe3a415fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:43.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.668+0000 7fe3ac0f4640 1 -- 172.21.15.76:0/1902601147 shutdown_connections 2024-03-08T06:00:43.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:43.668+0000 7fe3ac0f4640 1 -- 172.21.15.76:0/1902601147 wait complete. 2024-03-08T06:00:43.679 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-08T06:00:43.680 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T06:00:43.680 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-03-08T06:00:43.680 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-08T06:00:43.680 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-08T06:00:43.680 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-08T06:00:43.680 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-08T06:00:43.680 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T06:00:43.680 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-08T06:00:43.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:43.681 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-03-08T06:00:43.696 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:00:43.696 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:00:43.696 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:00:43.696 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:00:43.696 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-03-08T06:00:43.696 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:00:43.696 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:00:43.696 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:00:43.696 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-03-08T06:00:43.696 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:00:43.696 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:00:43.697 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-03-08T06:00:43.697 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:00:43.697 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:00:43.697 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T06:00:43.697 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:00:43.697 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-08T06:00:43.697 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42763sec preferred_lft 42763sec 2024-03-08T06:00:43.697 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-03-08T06:00:43.697 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:00:43.698 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-08T06:00:43.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:43.698 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2024-03-08T06:00:43.698 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-08T06:00:43.698 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add name ceph-brx type bridge 2024-03-08T06:00:43.698 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr flush dev ceph-brx 2024-03-08T06:00:43.698 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set ceph-brx up 2024-03-08T06:00:43.698 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-08T06:00:43.699 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-08T06:00:43.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:43.863 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T06:00:43.889 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-03-08T06:00:43.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:43.891 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-03-08T06:00:43.961 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-03-08T06:00:43.961 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T06:00:43.961 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-08T06:00:43.961 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-08T06:00:43.961 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T06:00:43.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:43.962 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2024-03-08T06:00:43.962 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-08T06:00:43.962 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-08T06:00:43.962 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-08T06:00:43.962 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-08T06:00:43.962 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-08T06:00:44.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:44.228 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-03-08T06:00:44.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:44.243 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list-id 2024-03-08T06:00:44.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:44.299 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2024-03-08T06:00:44.299 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-08T06:00:44.299 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-08T06:00:44.300 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-08T06:00:44.300 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-08T06:00:44.384 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-08T06:00:44.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:44.384 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2024-03-08T06:00:44.384 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-08T06:00:44.384 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-08T06:00:44.384 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-08T06:00:44.384 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-08T06:00:44.385 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-08T06:00:44.385 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-08T06:00:44.385 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-08T06:00:44.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:44.524 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2024-03-08T06:00:44.524 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-08T06:00:44.524 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2024-03-08T06:00:44.524 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set dev brx.0 master ceph-brx 2024-03-08T06:00:44.524 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-08T06:00:44.565 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-08T06:00:44.565 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:00:44.565 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-08T06:00:44.620 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-08T06:00:44.620 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:00:44.620 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-08T06:00:44.676 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-03-08T06:00:44.743 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-03-08T06:00:44.799 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-03-08T06:00:44.799 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-03-08T06:00:44.799 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-03-08T06:00:44.799 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-03-08T06:00:44.799 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-03-08T06:00:44.799 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-03-08T06:00:44.799 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-03-08T06:00:44.799 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-03-08T06:00:44.799 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-03-08T06:00:44.799 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-03-08T06:00:44.799 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-03-08T06:00:44.799 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2024-03-08T06:00:44.799 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-03-08T06:00:44.800 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-03-08T06:00:44.800 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-03-08T06:00:44.800 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-03-08T06:00:44.800 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-03-08T06:00:44.800 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-03-08T06:00:44.800 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2024-03-08T06:00:44.800 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-08T06:00:44.800 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-03-08T06:00:44.800 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-08T06:00:44.800 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2024-03-08T06:00:44.800 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-03-08T06:00:44.800 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-03-08T06:00:44.800 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-03-08T06:00:44.800 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-03-08T06:00:44.801 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-03-08T06:00:44.801 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-03-08T06:00:44.801 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-03-08T06:00:44.801 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-08T06:00:44.801 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-08T06:00:44.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:44.801 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-03-08T06:00:44.853 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-08T06:00:44.854 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-08T06:00:44.896 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-03-08T06:00:44.922 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-03-08T06:00:44.977 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-03-08T06:00:44.977 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-03-08T06:00:44.978 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-03-08T06:00:44.978 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-03-08T06:00:44.978 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-03-08T06:00:44.978 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-03-08T06:00:44.978 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-03-08T06:00:44.978 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-03-08T06:00:44.978 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-03-08T06:00:44.978 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-03-08T06:00:44.978 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-03-08T06:00:44.978 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2024-03-08T06:00:44.978 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-03-08T06:00:44.978 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-03-08T06:00:44.978 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-03-08T06:00:44.978 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-03-08T06:00:44.978 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-03-08T06:00:44.979 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-03-08T06:00:44.979 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2024-03-08T06:00:44.979 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-08T06:00:44.979 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-03-08T06:00:44.979 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-08T06:00:44.979 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2024-03-08T06:00:44.979 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-03-08T06:00:44.979 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-03-08T06:00:44.979 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-03-08T06:00:44.979 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-03-08T06:00:44.979 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-03-08T06:00:44.979 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-03-08T06:00:44.979 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-03-08T06:00:44.979 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-08T06:00:44.979 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-08T06:00:44.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:44.980 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-03-08T06:00:44.984 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi076.stderr:2024-03-08T06:00:44.983+0000 7f9c7924b480 -1 init, newargv = 0x55b9923eee30 newargc=15 2024-03-08T06:00:44.984 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi076.stderr:ceph-fuse[44901]: starting ceph client 2024-03-08T06:00:44.990 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi076.stderr:ceph-fuse[44901]: starting fuse 2024-03-08T06:00:44.999 INFO:teuthology.orchestra.run.smithi076.stdout:41 2024-03-08T06:00:44.999 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-03-08T06:00:44.999 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2024-03-08T06:00:44.999 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2024-03-08T06:00:44.999 DEBUG:teuthology.orchestra.run.smithi076:> import re 2024-03-08T06:00:44.999 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-03-08T06:00:44.999 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2024-03-08T06:00:44.999 DEBUG:teuthology.orchestra.run.smithi076:> 2024-03-08T06:00:44.999 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2024-03-08T06:00:44.999 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-08T06:00:44.999 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2024-03-08T06:00:44.999 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-08T06:00:45.000 DEBUG:teuthology.orchestra.run.smithi076:> 2024-03-08T06:00:45.000 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2024-03-08T06:00:45.000 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2024-03-08T06:00:45.000 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2024-03-08T06:00:45.000 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2024-03-08T06:00:45.000 DEBUG:teuthology.orchestra.run.smithi076:> 2024-03-08T06:00:45.000 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2024-03-08T06:00:45.000 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-08T06:00:45.000 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-08T06:00:45.000 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-08T06:00:45.000 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2024-03-08T06:00:45.000 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2024-03-08T06:00:45.000 DEBUG:teuthology.orchestra.run.smithi076:> return f 2024-03-08T06:00:45.000 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-08T06:00:45.000 DEBUG:teuthology.orchestra.run.smithi076:> 2024-03-08T06:00:45.000 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.admin")) 2024-03-08T06:00:45.001 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-03-08T06:00:45.120 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.admin.44901.asok 2024-03-08T06:00:45.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T06:00:45.131 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.44901.asok 2024-03-08T06:00:45.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:45.131 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.44901.asok status 2024-03-08T06:00:45.268 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-08T06:00:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2024-03-08T06:00:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "1fb5d10608dccee73edd6ef8a2c7b804a097c324", 2024-03-08T06:00:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)", 2024-03-08T06:00:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "admin", 2024-03-08T06:00:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2024-03-08T06:00:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-08T06:00:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "44901", 2024-03-08T06:00:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2024-03-08T06:00:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:00:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2024-03-08T06:00:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2024-03-08T06:00:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4446, 2024-03-08T06:00:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2024-03-08T06:00:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2024-03-08T06:00:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2024-03-08T06:00:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4446 2024-03-08T06:00:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:00:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-03-08T06:00:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-03-08T06:00:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-03-08T06:00:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2423827331 2024-03-08T06:00:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:00:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:00:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-03-08T06:00:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-03-08T06:00:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-03-08T06:00:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2423827331 2024-03-08T06:00:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:00:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.4446 192.168.0.1:0/2423827331", 2024-03-08T06:00:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/2423827331", 2024-03-08T06:00:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2024-03-08T06:00:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 6, 2024-03-08T06:00:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 26, 2024-03-08T06:00:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2024-03-08T06:00:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2024-03-08T06:00:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2024-03-08T06:00:45.270 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-08T06:00:45.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:45.276 DEBUG:teuthology.orchestra.run.smithi076:> stat --file-system '--printf=%T 2024-03-08T06:00:45.276 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-08T06:00:45.291 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2024-03-08T06:00:45.291 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-08T06:00:45.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:45.291 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-08T06:00:45.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:45.359 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-08T06:00:45.430 INFO:teuthology.orchestra.run.smithi076.stdout:# file: . 2024-03-08T06:00:45.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-08T06:00:45.431 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:45.431 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:00:45.431 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:00:45.487 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:00:45.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.627+0000 7f391413a640 1 Processor -- start 2024-03-08T06:00:45.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.628+0000 7f391413a640 1 -- start start 2024-03-08T06:00:45.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.628+0000 7f391413a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f390c15dc10 0x7f390c1623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:45.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.628+0000 7f391413a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f390c15dff0 0x7f390c0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:45.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.628+0000 7f391413a640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f390c0ce3f0 0x7f390c0ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:45.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.628+0000 7f391413a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390c0ced80 con 0x7f390c15dff0 2024-03-08T06:00:45.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.628+0000 7f391413a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390c06c780 con 0x7f390c15dc10 2024-03-08T06:00:45.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.628+0000 7f391413a640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390c0cffc0 con 0x7f390c0ce3f0 2024-03-08T06:00:45.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.629+0000 7f39116d5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f390c15dff0 0x7f390c0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:45.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.629+0000 7f39116d5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f390c15dff0 0x7f390c0cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35076/0 (socket says 172.21.15.76:35076) 2024-03-08T06:00:45.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.629+0000 7f39116d5640 1 -- 172.21.15.76:0/1328291294 learned_addr learned my addr 172.21.15.76:0/1328291294 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:45.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.629+0000 7f39126d7640 1 --2- 172.21.15.76:0/1328291294 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f390c0ce3f0 0x7f390c0ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:45.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.629+0000 7f3911ed6640 1 --2- 172.21.15.76:0/1328291294 >> v2:172.21.15.81:3300/0 conn(0x7f390c15dc10 0x7f390c1623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:45.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.629+0000 7f39116d5640 1 -- 172.21.15.76:0/1328291294 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f390c0ce3f0 msgr2=0x7f390c0ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:45.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.629+0000 7f39116d5640 1 --2- 172.21.15.76:0/1328291294 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f390c0ce3f0 0x7f390c0ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:45.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.629+0000 7f39116d5640 1 -- 172.21.15.76:0/1328291294 >> v2:172.21.15.81:3300/0 conn(0x7f390c15dc10 msgr2=0x7f390c1623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:45.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.629+0000 7f39116d5640 1 --2- 172.21.15.76:0/1328291294 >> v2:172.21.15.81:3300/0 conn(0x7f390c15dc10 0x7f390c1623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:45.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.629+0000 7f39116d5640 1 -- 172.21.15.76:0/1328291294 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c0d02a0 con 0x7f390c15dff0 2024-03-08T06:00:45.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.629+0000 7f3911ed6640 1 --2- 172.21.15.76:0/1328291294 >> v2:172.21.15.81:3300/0 conn(0x7f390c15dc10 0x7f390c1623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:45.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.630+0000 7f39116d5640 1 --2- 172.21.15.76:0/1328291294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f390c15dff0 0x7f390c0cf7c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3900009870 tx=0x7f390002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4ae49cfea32031 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:45.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.630+0000 7f3910ed4640 1 -- 172.21.15.76:0/1328291294 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39000317c0 con 0x7f390c15dff0 2024-03-08T06:00:45.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.630+0000 7f3910ed4640 1 -- 172.21.15.76:0/1328291294 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3900037070 con 0x7f390c15dff0 2024-03-08T06:00:45.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.630+0000 7f3910ed4640 1 -- 172.21.15.76:0/1328291294 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900031b80 con 0x7f390c15dff0 2024-03-08T06:00:45.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.630+0000 7f391413a640 1 -- 172.21.15.76:0/1328291294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f390c15dff0 msgr2=0x7f390c0cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:45.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.630+0000 7f391413a640 1 --2- 172.21.15.76:0/1328291294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f390c15dff0 0x7f390c0cf7c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3900009870 tx=0x7f390002b860 comp rx=0 tx=0).stop 2024-03-08T06:00:45.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.630+0000 7f391413a640 1 -- 172.21.15.76:0/1328291294 shutdown_connections 2024-03-08T06:00:45.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.630+0000 7f391413a640 1 --2- 172.21.15.76:0/1328291294 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f390c0ce3f0 0x7f390c0ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:45.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.630+0000 7f391413a640 1 --2- 172.21.15.76:0/1328291294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f390c15dff0 0x7f390c0cf7c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:45.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.630+0000 7f391413a640 1 --2- 172.21.15.76:0/1328291294 >> v2:172.21.15.81:3300/0 conn(0x7f390c15dc10 0x7f390c1623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:45.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.630+0000 7f391413a640 1 -- 172.21.15.76:0/1328291294 >> 172.21.15.76:0/1328291294 conn(0x7f390c06f5e0 msgr2=0x7f390c06f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:45.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.630+0000 7f391413a640 1 -- 172.21.15.76:0/1328291294 shutdown_connections 2024-03-08T06:00:45.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.630+0000 7f391413a640 1 -- 172.21.15.76:0/1328291294 wait complete. 2024-03-08T06:00:45.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.632+0000 7f391413a640 1 Processor -- start 2024-03-08T06:00:45.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.632+0000 7f391413a640 1 -- start start 2024-03-08T06:00:45.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.632+0000 7f391413a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f390c0ce3f0 0x7f390c1f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:45.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.632+0000 7f391413a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f390c15dc10 0x7f390c14e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:45.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.632+0000 7f391413a640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f390c15dff0 0x7f390c1f0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:45.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.632+0000 7f391413a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c0d0580 con 0x7f390c0ce3f0 2024-03-08T06:00:45.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.632+0000 7f391413a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c0cc170 con 0x7f390c15dc10 2024-03-08T06:00:45.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.632+0000 7f391413a640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390c0b3460 con 0x7f390c15dff0 2024-03-08T06:00:45.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.632+0000 7f3911ed6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f390c0ce3f0 0x7f390c1f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:45.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.632+0000 7f3911ed6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f390c0ce3f0 0x7f390c1f7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35090/0 (socket says 172.21.15.76:35090) 2024-03-08T06:00:45.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.632+0000 7f3911ed6640 1 -- 172.21.15.76:0/1846984260 learned_addr learned my addr 172.21.15.76:0/1846984260 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:45.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.633+0000 7f39116d5640 1 --2- 172.21.15.76:0/1846984260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f390c15dc10 0x7f390c14e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:45.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.633+0000 7f39126d7640 1 --2- 172.21.15.76:0/1846984260 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f390c15dff0 0x7f390c1f0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:45.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.633+0000 7f3911ed6640 1 -- 172.21.15.76:0/1846984260 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f390c15dff0 msgr2=0x7f390c1f0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:45.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.633+0000 7f3911ed6640 1 --2- 172.21.15.76:0/1846984260 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f390c15dff0 0x7f390c1f0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:45.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.633+0000 7f3911ed6640 1 -- 172.21.15.76:0/1846984260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f390c15dc10 msgr2=0x7f390c14e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:45.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.633+0000 7f3911ed6640 1 --2- 172.21.15.76:0/1846984260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f390c15dc10 0x7f390c14e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:45.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.633+0000 7f3911ed6640 1 -- 172.21.15.76:0/1846984260 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c14ecf0 con 0x7f390c0ce3f0 2024-03-08T06:00:45.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.633+0000 7f3911ed6640 1 --2- 172.21.15.76:0/1846984260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f390c0ce3f0 0x7f390c1f7870 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f390400cf90 tx=0x7f390400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:45.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.633+0000 7f390a7fc640 1 -- 172.21.15.76:0/1846984260 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390401a070 con 0x7f390c0ce3f0 2024-03-08T06:00:45.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.633+0000 7f390a7fc640 1 -- 172.21.15.76:0/1846984260 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390400d840 con 0x7f390c0ce3f0 2024-03-08T06:00:45.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.633+0000 7f391413a640 1 -- 172.21.15.76:0/1846984260 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390c14efd0 con 0x7f390c0ce3f0 2024-03-08T06:00:45.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.633+0000 7f390a7fc640 1 -- 172.21.15.76:0/1846984260 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390400dc10 con 0x7f390c0ce3f0 2024-03-08T06:00:45.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.633+0000 7f391413a640 1 -- 172.21.15.76:0/1846984260 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390c1f1190 con 0x7f390c0ce3f0 2024-03-08T06:00:45.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.634+0000 7f391413a640 1 -- 172.21.15.76:0/1846984260 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390c04f030 con 0x7f390c0ce3f0 2024-03-08T06:00:45.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.634+0000 7f390a7fc640 1 -- 172.21.15.76:0/1846984260 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f390401d070 con 0x7f390c0ce3f0 2024-03-08T06:00:45.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.635+0000 7f390a7fc640 1 --2- 172.21.15.76:0/1846984260 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f38f806b4d0 0x7f38f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:45.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.635+0000 7f39116d5640 1 --2- 172.21.15.76:0/1846984260 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f38f806b4d0 0x7f38f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:45.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.635+0000 7f390a7fc640 1 -- 172.21.15.76:0/1846984260 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f390408acf0 con 0x7f390c0ce3f0 2024-03-08T06:00:45.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.635+0000 7f39116d5640 1 --2- 172.21.15.76:0/1846984260 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f38f806b4d0 0x7f38f806d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3900036000 tx=0x7f390003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:45.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.639+0000 7f390a7fc640 1 -- 172.21.15.76:0/1846984260 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f390408f070 con 0x7f390c0ce3f0 2024-03-08T06:00:45.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.811+0000 7f391413a640 1 -- 172.21.15.76:0/1846984260 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f390c0b7960 con 0x7f390c0ce3f0 2024-03-08T06:00:45.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.812+0000 7f390a7fc640 1 -- 172.21.15.76:0/1846984260 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f39040591e0 con 0x7f390c0ce3f0 2024-03-08T06:00:45.813 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:45.813 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":26,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:42.202011+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:00:45.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.814+0000 7f391413a640 1 -- 172.21.15.76:0/1846984260 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f38f806b4d0 msgr2=0x7f38f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:45.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.814+0000 7f391413a640 1 --2- 172.21.15.76:0/1846984260 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f38f806b4d0 0x7f38f806d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3900036000 tx=0x7f390003d040 comp rx=0 tx=0).stop 2024-03-08T06:00:45.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.814+0000 7f391413a640 1 -- 172.21.15.76:0/1846984260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f390c0ce3f0 msgr2=0x7f390c1f7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:45.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.814+0000 7f391413a640 1 --2- 172.21.15.76:0/1846984260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f390c0ce3f0 0x7f390c1f7870 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f390400cf90 tx=0x7f390400a7d0 comp rx=0 tx=0).stop 2024-03-08T06:00:45.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.814+0000 7f391413a640 1 -- 172.21.15.76:0/1846984260 shutdown_connections 2024-03-08T06:00:45.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.814+0000 7f391413a640 1 --2- 172.21.15.76:0/1846984260 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f38f806b4d0 0x7f38f806d990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:45.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.814+0000 7f391413a640 1 --2- 172.21.15.76:0/1846984260 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f390c15dff0 0x7f390c1f0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:45.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.814+0000 7f391413a640 1 --2- 172.21.15.76:0/1846984260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f390c15dc10 0x7f390c14e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:45.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.814+0000 7f391413a640 1 --2- 172.21.15.76:0/1846984260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f390c0ce3f0 0x7f390c1f7870 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:45.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.814+0000 7f391413a640 1 -- 172.21.15.76:0/1846984260 >> 172.21.15.76:0/1846984260 conn(0x7f390c06f5e0 msgr2=0x7f390c0b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:45.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.814+0000 7f391413a640 1 -- 172.21.15.76:0/1846984260 shutdown_connections 2024-03-08T06:00:45.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.814+0000 7f391413a640 1 -- 172.21.15.76:0/1846984260 wait complete. 2024-03-08T06:00:45.825 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:00:33.926651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:00:34.474030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T06:00:38.207853+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T06:00:45.826 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:00:45.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.929+0000 7faa72447640 1 Processor -- start 2024-03-08T06:00:45.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.929+0000 7faa72447640 1 -- start start 2024-03-08T06:00:45.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.929+0000 7faa72447640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faa6c15e500 0x7faa6c0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:45.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.929+0000 7faa72447640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6c0b1550 con 0x7faa6c15e120 2024-03-08T06:00:45.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.929+0000 7faa72447640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6c0ae1d0 con 0x7faa6c0cf010 2024-03-08T06:00:45.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.929+0000 7faa72447640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6c0ae4b0 con 0x7faa6c15e500 2024-03-08T06:00:45.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.929+0000 7faa6bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7faa6c15e120 0x7faa6c0ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33816/0 (socket says 172.21.15.76:33816) 2024-03-08T06:00:45.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.930+0000 7faa6bfff640 1 -- 172.21.15.76:0/2486890237 learned_addr learned my addr 172.21.15.76:0/2486890237 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:45.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.930+0000 7faa6b7fe640 1 --2- 172.21.15.76:0/2486890237 >> v2:172.21.15.76:3300/0 conn(0x7faa6c15e500 0x7faa6c0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:45.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.930+0000 7faa6b7fe640 1 -- 172.21.15.76:0/2486890237 >> v1:172.21.15.81:6789/0 conn(0x7faa6c0cf010 legacy=0x7faa6c0aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:45.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.930+0000 7faa6b7fe640 1 -- 172.21.15.76:0/2486890237 >> v1:172.21.15.76:6789/0 conn(0x7faa6c15e120 legacy=0x7faa6c0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:45.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.930+0000 7faa6b7fe640 1 -- 172.21.15.76:0/2486890237 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c0ae790 con 0x7faa6c15e500 2024-03-08T06:00:45.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.930+0000 7faa6affd640 1 -- 172.21.15.76:0/2486890237 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181736050 0 0) 0x7faa6c0b1550 con 0x7faa6c15e120 2024-03-08T06:00:45.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.931+0000 7faa6b7fe640 1 --2- 172.21.15.76:0/2486890237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa6c15e500 0x7faa6c0cb270 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0097a0 tx=0x7faa5c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fc4604fec27b40 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:45.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.931+0000 7faa6affd640 1 -- 172.21.15.76:0/2486890237 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c034070 con 0x7faa6c15e500 2024-03-08T06:00:45.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.931+0000 7faa6affd640 1 -- 172.21.15.76:0/2486890237 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c030610 con 0x7faa6c15e500 2024-03-08T06:00:45.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.931+0000 7faa6affd640 1 -- 172.21.15.76:0/2486890237 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0309e0 con 0x7faa6c15e500 2024-03-08T06:00:45.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.931+0000 7faa72447640 1 -- 172.21.15.76:0/2486890237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa6c15e500 msgr2=0x7faa6c0cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:45.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.931+0000 7faa72447640 1 --2- 172.21.15.76:0/2486890237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa6c15e500 0x7faa6c0cb270 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0097a0 tx=0x7faa5c02b960 comp rx=0 tx=0).stop 2024-03-08T06:00:45.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.931+0000 7faa72447640 1 -- 172.21.15.76:0/2486890237 shutdown_connections 2024-03-08T06:00:45.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.931+0000 7faa72447640 1 --2- 172.21.15.76:0/2486890237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa6c15e500 0x7faa6c0cb270 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:45.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.931+0000 7faa72447640 1 -- 172.21.15.76:0/2486890237 >> 172.21.15.76:0/2486890237 conn(0x7faa6c0d0890 msgr2=0x7faa6c0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:45.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.931+0000 7faa72447640 1 -- 172.21.15.76:0/2486890237 shutdown_connections 2024-03-08T06:00:45.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.931+0000 7faa72447640 1 -- 172.21.15.76:0/2486890237 wait complete. 2024-03-08T06:00:45.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.932+0000 7faa72447640 1 Processor -- start 2024-03-08T06:00:45.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.933+0000 7faa72447640 1 -- start start 2024-03-08T06:00:45.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.933+0000 7faa72447640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa6c0cf010 0x7faa6c1efd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:45.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.933+0000 7faa72447640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa6c15e120 0x7faa6c09ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:45.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.933+0000 7faa72447640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa6c15e500 0x7faa6c1f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:45.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.933+0000 7faa72447640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0d1380 con 0x7faa6c0cf010 2024-03-08T06:00:45.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.933+0000 7faa72447640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0cf6b0 con 0x7faa6c15e120 2024-03-08T06:00:45.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.933+0000 7faa72447640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0d1500 con 0x7faa6c15e500 2024-03-08T06:00:45.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.933+0000 7faa6bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa6c0cf010 0x7faa6c1efd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:45.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.933+0000 7faa6bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa6c0cf010 0x7faa6c1efd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35114/0 (socket says 172.21.15.76:35114) 2024-03-08T06:00:45.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.933+0000 7faa6bfff640 1 -- 172.21.15.76:0/1508354700 learned_addr learned my addr 172.21.15.76:0/1508354700 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:45.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.934+0000 7faa6b7fe640 1 --2- 172.21.15.76:0/1508354700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa6c15e120 0x7faa6c09ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:45.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.934+0000 7faa709e4640 1 --2- 172.21.15.76:0/1508354700 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa6c15e500 0x7faa6c1f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:45.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.934+0000 7faa6bfff640 1 -- 172.21.15.76:0/1508354700 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa6c15e500 msgr2=0x7faa6c1f3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:45.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.934+0000 7faa6bfff640 1 --2- 172.21.15.76:0/1508354700 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa6c15e500 0x7faa6c1f3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:45.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.934+0000 7faa6bfff640 1 -- 172.21.15.76:0/1508354700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa6c15e120 msgr2=0x7faa6c09ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:45.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.934+0000 7faa6bfff640 1 --2- 172.21.15.76:0/1508354700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa6c15e120 0x7faa6c09ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:45.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.934+0000 7faa6bfff640 1 -- 172.21.15.76:0/1508354700 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c1f0250 con 0x7faa6c0cf010 2024-03-08T06:00:45.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.934+0000 7faa709e4640 1 --2- 172.21.15.76:0/1508354700 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa6c15e500 0x7faa6c1f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:00:45.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.934+0000 7faa6b7fe640 1 --2- 172.21.15.76:0/1508354700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa6c15e120 0x7faa6c09ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:00:45.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.934+0000 7faa6bfff640 1 --2- 172.21.15.76:0/1508354700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa6c0cf010 0x7faa6c1efd10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faa6000c4a0 tx=0x7faa6000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:45.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.934+0000 7faa68ff9640 1 -- 172.21.15.76:0/1508354700 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa60018070 con 0x7faa6c0cf010 2024-03-08T06:00:45.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.934+0000 7faa68ff9640 1 -- 172.21.15.76:0/1508354700 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa600074e0 con 0x7faa6c0cf010 2024-03-08T06:00:45.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.934+0000 7faa68ff9640 1 -- 172.21.15.76:0/1508354700 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa60004a20 con 0x7faa6c0cf010 2024-03-08T06:00:45.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.934+0000 7faa72447640 1 -- 172.21.15.76:0/1508354700 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6c09c450 con 0x7faa6c0cf010 2024-03-08T06:00:45.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.934+0000 7faa72447640 1 -- 172.21.15.76:0/1508354700 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa6c09ca10 con 0x7faa6c0cf010 2024-03-08T06:00:45.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.935+0000 7faa72447640 1 -- 172.21.15.76:0/1508354700 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6c04f030 con 0x7faa6c0cf010 2024-03-08T06:00:45.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.936+0000 7faa68ff9640 1 -- 172.21.15.76:0/1508354700 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa6001b070 con 0x7faa6c0cf010 2024-03-08T06:00:45.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.936+0000 7faa68ff9640 1 --2- 172.21.15.76:0/1508354700 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faa5806b500 0x7faa5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:45.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.936+0000 7faa6b7fe640 1 --2- 172.21.15.76:0/1508354700 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faa5806b500 0x7faa5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:45.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.936+0000 7faa68ff9640 1 -- 172.21.15.76:0/1508354700 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7faa600897f0 con 0x7faa6c0cf010 2024-03-08T06:00:45.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.937+0000 7faa6b7fe640 1 --2- 172.21.15.76:0/1508354700 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faa5806b500 0x7faa5806d9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faa5c036000 tx=0x7faa5c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:45.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:45.941+0000 7faa68ff9640 1 -- 172.21.15.76:0/1508354700 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa60058330 con 0x7faa6c0cf010 2024-03-08T06:00:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.112+0000 7faa72447640 1 -- 172.21.15.76:0/1508354700 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa6c15b950 con 0x7faa6c0cf010 2024-03-08T06:00:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.113+0000 7faa68ff9640 1 -- 172.21.15.76:0/1508354700 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa60057c30 con 0x7faa6c0cf010 2024-03-08T06:00:46.113 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:00:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.115+0000 7faa72447640 1 -- 172.21.15.76:0/1508354700 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faa5806b500 msgr2=0x7faa5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.115+0000 7faa72447640 1 --2- 172.21.15.76:0/1508354700 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faa5806b500 0x7faa5806d9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faa5c036000 tx=0x7faa5c008030 comp rx=0 tx=0).stop 2024-03-08T06:00:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.115+0000 7faa72447640 1 -- 172.21.15.76:0/1508354700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa6c0cf010 msgr2=0x7faa6c1efd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.115+0000 7faa72447640 1 --2- 172.21.15.76:0/1508354700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa6c0cf010 0x7faa6c1efd10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faa6000c4a0 tx=0x7faa6000c960 comp rx=0 tx=0).stop 2024-03-08T06:00:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.115+0000 7faa72447640 1 -- 172.21.15.76:0/1508354700 shutdown_connections 2024-03-08T06:00:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.115+0000 7faa72447640 1 --2- 172.21.15.76:0/1508354700 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faa5806b500 0x7faa5806d9c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.115+0000 7faa72447640 1 --2- 172.21.15.76:0/1508354700 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa6c15e500 0x7faa6c1f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.115+0000 7faa72447640 1 --2- 172.21.15.76:0/1508354700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa6c15e120 0x7faa6c09ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.115+0000 7faa72447640 1 --2- 172.21.15.76:0/1508354700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa6c0cf010 0x7faa6c1efd10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.115+0000 7faa72447640 1 -- 172.21.15.76:0/1508354700 >> 172.21.15.76:0/1508354700 conn(0x7faa6c0d0890 msgr2=0x7faa6c15ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.115+0000 7faa72447640 1 -- 172.21.15.76:0/1508354700 shutdown_connections 2024-03-08T06:00:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.115+0000 7faa72447640 1 -- 172.21.15.76:0/1508354700 wait complete. 2024-03-08T06:00:46.126 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:00:46.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.230+0000 7f30a23ae640 1 Processor -- start 2024-03-08T06:00:46.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.230+0000 7f30a23ae640 1 -- start start 2024-03-08T06:00:46.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.230+0000 7f30a23ae640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f309c159740 0x7f309c15c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:46.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.230+0000 7f30a23ae640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f309c06b390 0x7f309c0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:46.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.230+0000 7f30a23ae640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309c159b20 con 0x7f309c0cc4c0 2024-03-08T06:00:46.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.230+0000 7f30a23ae640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309c0ce160 con 0x7f309c159740 2024-03-08T06:00:46.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.230+0000 7f30a23ae640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0ccb80 con 0x7f309c06b390 2024-03-08T06:00:46.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.231+0000 7f30a094b640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f309c06b390 0x7f309c0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:46.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.231+0000 7f309bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f309c159740 0x7f309c15c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:46.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.231+0000 7f309b7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f309c0cc4c0 0x7f309c06ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:36530/0 (socket says 172.21.15.76:36530) 2024-03-08T06:00:46.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.231+0000 7f309b7fe640 1 -- 172.21.15.76:0/306673963 learned_addr learned my addr 172.21.15.76:0/306673963 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:46.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.231+0000 7f309bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f309c159740 0x7f309c15c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.76:39012/0 (socket says 172.21.15.76:39012) 2024-03-08T06:00:46.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.231+0000 7f30a094b640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f309c06b390 0x7f309c0cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:33886/0 (socket says 172.21.15.76:33886) 2024-03-08T06:00:46.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.231+0000 7f309bfff640 1 -- 172.21.15.76:0/306673963 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f309c06b390 msgr2=0x7f309c0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:46.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.231+0000 7f309bfff640 1 --2- 172.21.15.76:0/306673963 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f309c06b390 0x7f309c0cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.231+0000 7f309bfff640 1 -- 172.21.15.76:0/306673963 >> v1:172.21.15.81:6789/0 conn(0x7f309c0cc4c0 legacy=0x7f309c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.231+0000 7f309affd640 1 -- 172.21.15.76:0/306673963 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418318932 0 0) 0x7f309c159b20 con 0x7f309c0cc4c0 2024-03-08T06:00:46.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.231+0000 7f309bfff640 1 -- 172.21.15.76:0/306673963 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c0cce60 con 0x7f309c159740 2024-03-08T06:00:46.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.231+0000 7f30a094b640 1 --2- 172.21.15.76:0/306673963 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f309c06b390 0x7f309c0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:00:46.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.232+0000 7f309bfff640 1 --2- 172.21.15.76:0/306673963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f309c159740 0x7f309c15c0f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f308c0097a0 tx=0x7f308c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c063375f6206f44a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:46.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.232+0000 7f309affd640 1 -- 172.21.15.76:0/306673963 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c0315a0 con 0x7f309c159740 2024-03-08T06:00:46.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.232+0000 7f309affd640 1 -- 172.21.15.76:0/306673963 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308c037070 con 0x7f309c159740 2024-03-08T06:00:46.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.232+0000 7f309affd640 1 -- 172.21.15.76:0/306673963 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c031960 con 0x7f309c159740 2024-03-08T06:00:46.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.232+0000 7f30a23ae640 1 -- 172.21.15.76:0/306673963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f309c159740 msgr2=0x7f309c15c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.233+0000 7f30a23ae640 1 --2- 172.21.15.76:0/306673963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f309c159740 0x7f309c15c0f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f308c0097a0 tx=0x7f308c02b840 comp rx=0 tx=0).stop 2024-03-08T06:00:46.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.233+0000 7f30a23ae640 1 -- 172.21.15.76:0/306673963 shutdown_connections 2024-03-08T06:00:46.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.233+0000 7f30a23ae640 1 --2- 172.21.15.76:0/306673963 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f309c06b390 0x7f309c0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.233+0000 7f30a23ae640 1 --2- 172.21.15.76:0/306673963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f309c159740 0x7f309c15c0f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.233+0000 7f30a23ae640 1 -- 172.21.15.76:0/306673963 >> 172.21.15.76:0/306673963 conn(0x7f309c0d0890 msgr2=0x7f309c0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:46.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.233+0000 7f30a23ae640 1 -- 172.21.15.76:0/306673963 shutdown_connections 2024-03-08T06:00:46.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.233+0000 7f30a23ae640 1 -- 172.21.15.76:0/306673963 wait complete. 2024-03-08T06:00:46.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.234+0000 7f30a23ae640 1 Processor -- start 2024-03-08T06:00:46.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.234+0000 7f30a23ae640 1 -- start start 2024-03-08T06:00:46.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f30a23ae640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f309c06b390 0x7f309c0b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:46.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f30a23ae640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f309c0cc4c0 0x7f309c157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:46.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f30a23ae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f309c159740 0x7f309c06f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:46.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f30a23ae640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0cd700 con 0x7f309c0cc4c0 2024-03-08T06:00:46.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f30a23ae640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0cd420 con 0x7f309c159740 2024-03-08T06:00:46.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f30a23ae640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c06b9b0 con 0x7f309c06b390 2024-03-08T06:00:46.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f309b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f309c0cc4c0 0x7f309c157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:46.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f309b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f309c0cc4c0 0x7f309c157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35122/0 (socket says 172.21.15.76:35122) 2024-03-08T06:00:46.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f309b7fe640 1 -- 172.21.15.76:0/1714923568 learned_addr learned my addr 172.21.15.76:0/1714923568 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:46.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f309bfff640 1 --2- 172.21.15.76:0/1714923568 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f309c06b390 0x7f309c0b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:46.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f30a094b640 1 --2- 172.21.15.76:0/1714923568 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f309c159740 0x7f309c06f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:46.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f309b7fe640 1 -- 172.21.15.76:0/1714923568 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f309c06b390 msgr2=0x7f309c0b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f309b7fe640 1 --2- 172.21.15.76:0/1714923568 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f309c06b390 0x7f309c0b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f309b7fe640 1 -- 172.21.15.76:0/1714923568 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f309c159740 msgr2=0x7f309c06f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f309b7fe640 1 --2- 172.21.15.76:0/1714923568 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f309c159740 0x7f309c06f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f309b7fe640 1 -- 172.21.15.76:0/1714923568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c0b0930 con 0x7f309c0cc4c0 2024-03-08T06:00:46.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f309b7fe640 1 --2- 172.21.15.76:0/1714923568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f309c0cc4c0 0x7f309c157fc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f309000c4d0 tx=0x7f309000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:46.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f3098ff9640 1 -- 172.21.15.76:0/1714923568 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090018070 con 0x7f309c0cc4c0 2024-03-08T06:00:46.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f3098ff9640 1 -- 172.21.15.76:0/1714923568 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3090007500 con 0x7f309c0cc4c0 2024-03-08T06:00:46.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.235+0000 7f30a23ae640 1 -- 172.21.15.76:0/1714923568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309c0b0c10 con 0x7f309c0cc4c0 2024-03-08T06:00:46.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.236+0000 7f3098ff9640 1 -- 172.21.15.76:0/1714923568 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090004a20 con 0x7f309c0cc4c0 2024-03-08T06:00:46.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.236+0000 7f30a23ae640 1 -- 172.21.15.76:0/1714923568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c0b11d0 con 0x7f309c0cc4c0 2024-03-08T06:00:46.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.237+0000 7f30a23ae640 1 -- 172.21.15.76:0/1714923568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309c04f030 con 0x7f309c0cc4c0 2024-03-08T06:00:46.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.237+0000 7f3098ff9640 1 -- 172.21.15.76:0/1714923568 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f309001b070 con 0x7f309c0cc4c0 2024-03-08T06:00:46.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.237+0000 7f3098ff9640 1 --2- 172.21.15.76:0/1714923568 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f308806b4d0 0x7f308806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:46.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.237+0000 7f3098ff9640 1 -- 172.21.15.76:0/1714923568 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3090089760 con 0x7f309c0cc4c0 2024-03-08T06:00:46.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.237+0000 7f309bfff640 1 --2- 172.21.15.76:0/1714923568 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f308806b4d0 0x7f308806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:46.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.238+0000 7f309bfff640 1 --2- 172.21.15.76:0/1714923568 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f308806b4d0 0x7f308806d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f308c036000 tx=0x7f308c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:46.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.242+0000 7f3098ff9640 1 -- 172.21.15.76:0/1714923568 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3090057ce0 con 0x7f309c0cc4c0 2024-03-08T06:00:46.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.411+0000 7f30a23ae640 1 -- 172.21.15.76:0/1714923568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f309c0b6810 con 0x7f309c0cc4c0 2024-03-08T06:00:46.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.411+0000 7f3098ff9640 1 -- 172.21.15.76:0/1714923568 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f309005bd80 con 0x7f309c0cc4c0 2024-03-08T06:00:46.412 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:00:46.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.413+0000 7f30a23ae640 1 -- 172.21.15.76:0/1714923568 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f308806b4d0 msgr2=0x7f308806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.413+0000 7f30a23ae640 1 --2- 172.21.15.76:0/1714923568 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f308806b4d0 0x7f308806d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f308c036000 tx=0x7f308c008030 comp rx=0 tx=0).stop 2024-03-08T06:00:46.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.413+0000 7f30a23ae640 1 -- 172.21.15.76:0/1714923568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f309c0cc4c0 msgr2=0x7f309c157fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.413+0000 7f30a23ae640 1 --2- 172.21.15.76:0/1714923568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f309c0cc4c0 0x7f309c157fc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f309000c4d0 tx=0x7f309000c990 comp rx=0 tx=0).stop 2024-03-08T06:00:46.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.413+0000 7f30a23ae640 1 -- 172.21.15.76:0/1714923568 shutdown_connections 2024-03-08T06:00:46.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.413+0000 7f30a23ae640 1 --2- 172.21.15.76:0/1714923568 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f308806b4d0 0x7f308806d990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.413+0000 7f30a23ae640 1 --2- 172.21.15.76:0/1714923568 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f309c159740 0x7f309c06f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.413+0000 7f30a23ae640 1 --2- 172.21.15.76:0/1714923568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f309c0cc4c0 0x7f309c157fc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.413+0000 7f30a23ae640 1 --2- 172.21.15.76:0/1714923568 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f309c06b390 0x7f309c0b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.413+0000 7f30a23ae640 1 -- 172.21.15.76:0/1714923568 >> 172.21.15.76:0/1714923568 conn(0x7f309c0d0890 msgr2=0x7f309c0c7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:46.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.413+0000 7f30a23ae640 1 -- 172.21.15.76:0/1714923568 shutdown_connections 2024-03-08T06:00:46.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.413+0000 7f30a23ae640 1 -- 172.21.15.76:0/1714923568 wait complete. 2024-03-08T06:00:46.425 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:00:46.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.528+0000 7fabbc647640 1 Processor -- start 2024-03-08T06:00:46.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.528+0000 7fabbc647640 1 -- start start 2024-03-08T06:00:46.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.528+0000 7fabbc647640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fabb406d8a0 0x7fabb4162020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:46.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.529+0000 7fabbc647640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb4161840 con 0x7fabb40ce5e0 2024-03-08T06:00:46.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.529+0000 7fabbc647640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb415eb30 con 0x7fabb4162560 2024-03-08T06:00:46.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.529+0000 7fabbc647640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb4161b90 con 0x7fabb406d8a0 2024-03-08T06:00:46.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.529+0000 7fabb9be2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fabb406d8a0 0x7fabb4162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:46.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.529+0000 7fabba3e3640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fabb40ce5e0 0x7fabb40ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33820/0 (socket says 172.21.15.76:33820) 2024-03-08T06:00:46.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.529+0000 7fabba3e3640 1 -- 172.21.15.76:0/637534810 learned_addr learned my addr 172.21.15.76:0/637534810 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:46.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.529+0000 7fabb9be2640 1 -- 172.21.15.76:0/637534810 >> v1:172.21.15.81:6789/0 conn(0x7fabb4162560 legacy=0x7fabb415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:46.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.529+0000 7fabb9be2640 1 -- 172.21.15.76:0/637534810 >> v1:172.21.15.76:6789/0 conn(0x7fabb40ce5e0 legacy=0x7fabb40ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.529+0000 7fabb9be2640 1 -- 172.21.15.76:0/637534810 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb415ee10 con 0x7fabb406d8a0 2024-03-08T06:00:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.530+0000 7fabb9be2640 1 --2- 172.21.15.76:0/637534810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabb406d8a0 0x7fabb4162020 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faba40097a0 tx=0x7faba402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa46f4f587df2b51 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.530+0000 7fabb93e1640 1 -- 172.21.15.76:0/637534810 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4034070 con 0x7fabb406d8a0 2024-03-08T06:00:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.530+0000 7fabb93e1640 1 -- 172.21.15.76:0/637534810 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba4030610 con 0x7fabb406d8a0 2024-03-08T06:00:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.530+0000 7fabb93e1640 1 -- 172.21.15.76:0/637534810 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba40309e0 con 0x7fabb406d8a0 2024-03-08T06:00:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.530+0000 7fabbc647640 1 -- 172.21.15.76:0/637534810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabb406d8a0 msgr2=0x7fabb4162020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.530+0000 7fabbc647640 1 --2- 172.21.15.76:0/637534810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabb406d8a0 0x7fabb4162020 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faba40097a0 tx=0x7faba402b960 comp rx=0 tx=0).stop 2024-03-08T06:00:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.530+0000 7fabbc647640 1 -- 172.21.15.76:0/637534810 shutdown_connections 2024-03-08T06:00:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.530+0000 7fabbc647640 1 --2- 172.21.15.76:0/637534810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabb406d8a0 0x7fabb4162020 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.530+0000 7fabbc647640 1 -- 172.21.15.76:0/637534810 >> 172.21.15.76:0/637534810 conn(0x7fabb406ae60 msgr2=0x7fabb406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.531+0000 7fabbc647640 1 -- 172.21.15.76:0/637534810 shutdown_connections 2024-03-08T06:00:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.531+0000 7fabbc647640 1 -- 172.21.15.76:0/637534810 wait complete. 2024-03-08T06:00:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.532+0000 7fabbc647640 1 Processor -- start 2024-03-08T06:00:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.532+0000 7fabbc647640 1 -- start start 2024-03-08T06:00:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.532+0000 7fabbc647640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabb406d8a0 0x7fabb40c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.532+0000 7fabbc647640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabb40ce5e0 0x7fabb41e9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.532+0000 7fabbc647640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabb4162560 0x7fabb40b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.532+0000 7fabbc647640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb415e840 con 0x7fabb4162560 2024-03-08T06:00:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.532+0000 7fabbc647640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb415e6c0 con 0x7fabb40ce5e0 2024-03-08T06:00:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.532+0000 7fabbc647640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb406bfa0 con 0x7fabb406d8a0 2024-03-08T06:00:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.532+0000 7fabbabe4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabb4162560 0x7fabb40b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.532+0000 7fabbabe4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabb4162560 0x7fabb40b55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35132/0 (socket says 172.21.15.76:35132) 2024-03-08T06:00:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.533+0000 7fabbabe4640 1 -- 172.21.15.76:0/3645847644 learned_addr learned my addr 172.21.15.76:0/3645847644 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.533+0000 7fabba3e3640 1 --2- 172.21.15.76:0/3645847644 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabb406d8a0 0x7fabb40c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.533+0000 7fabb9be2640 1 --2- 172.21.15.76:0/3645847644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabb40ce5e0 0x7fabb41e9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.533+0000 7fabbabe4640 1 -- 172.21.15.76:0/3645847644 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabb406d8a0 msgr2=0x7fabb40c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.533+0000 7fabbabe4640 1 --2- 172.21.15.76:0/3645847644 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabb406d8a0 0x7fabb40c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.533+0000 7fabbabe4640 1 -- 172.21.15.76:0/3645847644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabb40ce5e0 msgr2=0x7fabb41e9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.533+0000 7fabbabe4640 1 --2- 172.21.15.76:0/3645847644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabb40ce5e0 0x7fabb41e9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.533+0000 7fabbabe4640 1 -- 172.21.15.76:0/3645847644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb40c80a0 con 0x7fabb4162560 2024-03-08T06:00:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.533+0000 7fabb9be2640 1 --2- 172.21.15.76:0/3645847644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabb40ce5e0 0x7fabb41e9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.533+0000 7fabbabe4640 1 --2- 172.21.15.76:0/3645847644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabb4162560 0x7fabb40b55a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fabac00b9e0 tx=0x7fabac00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.533+0000 7fabaaffd640 1 -- 172.21.15.76:0/3645847644 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac003b70 con 0x7fabb4162560 2024-03-08T06:00:46.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.533+0000 7fabaaffd640 1 -- 172.21.15.76:0/3645847644 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac003d10 con 0x7fabb4162560 2024-03-08T06:00:46.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.533+0000 7fabbc647640 1 -- 172.21.15.76:0/3645847644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb40c8380 con 0x7fabb4162560 2024-03-08T06:00:46.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.533+0000 7fabaaffd640 1 -- 172.21.15.76:0/3645847644 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac00cb60 con 0x7fabb4162560 2024-03-08T06:00:46.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.533+0000 7fabbc647640 1 -- 172.21.15.76:0/3645847644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb41e7b90 con 0x7fabb4162560 2024-03-08T06:00:46.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.534+0000 7fabbc647640 1 -- 172.21.15.76:0/3645847644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb404f030 con 0x7fabb4162560 2024-03-08T06:00:46.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.535+0000 7fabaaffd640 1 -- 172.21.15.76:0/3645847644 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fabac00cd00 con 0x7fabb4162560 2024-03-08T06:00:46.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.535+0000 7fabaaffd640 1 --2- 172.21.15.76:0/3645847644 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fab9806b4d0 0x7fab9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:46.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.535+0000 7fabba3e3640 1 --2- 172.21.15.76:0/3645847644 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fab9806b4d0 0x7fab9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:46.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.535+0000 7fabaaffd640 1 -- 172.21.15.76:0/3645847644 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fabac08a430 con 0x7fabb4162560 2024-03-08T06:00:46.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.535+0000 7fabba3e3640 1 --2- 172.21.15.76:0/3645847644 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fab9806b4d0 0x7fab9806d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fabb0000f60 tx=0x7fabb0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:46.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.540+0000 7fabaaffd640 1 -- 172.21.15.76:0/3645847644 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fabac058ee0 con 0x7fabb4162560 2024-03-08T06:00:46.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.708+0000 7fabbc647640 1 -- 172.21.15.76:0/3645847644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fabb41e7d60 con 0x7fabb4162560 2024-03-08T06:00:46.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.709+0000 7fabaaffd640 1 -- 172.21.15.76:0/3645847644 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fabac0587e0 con 0x7fabb4162560 2024-03-08T06:00:46.709 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:00:46.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.711+0000 7fabbc647640 1 -- 172.21.15.76:0/3645847644 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fab9806b4d0 msgr2=0x7fab9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.711+0000 7fabbc647640 1 --2- 172.21.15.76:0/3645847644 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fab9806b4d0 0x7fab9806d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fabb0000f60 tx=0x7fabb0009040 comp rx=0 tx=0).stop 2024-03-08T06:00:46.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.711+0000 7fabbc647640 1 -- 172.21.15.76:0/3645847644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabb4162560 msgr2=0x7fabb40b55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.711+0000 7fabbc647640 1 --2- 172.21.15.76:0/3645847644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabb4162560 0x7fabb40b55a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fabac00b9e0 tx=0x7fabac00bea0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.711+0000 7fabbc647640 1 -- 172.21.15.76:0/3645847644 shutdown_connections 2024-03-08T06:00:46.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.711+0000 7fabbc647640 1 --2- 172.21.15.76:0/3645847644 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fab9806b4d0 0x7fab9806d990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.711+0000 7fabbc647640 1 --2- 172.21.15.76:0/3645847644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabb4162560 0x7fabb40b55a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.711+0000 7fabbc647640 1 --2- 172.21.15.76:0/3645847644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabb40ce5e0 0x7fabb41e9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.711+0000 7fabbc647640 1 --2- 172.21.15.76:0/3645847644 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fabb406d8a0 0x7fabb40c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.711+0000 7fabbc647640 1 -- 172.21.15.76:0/3645847644 >> 172.21.15.76:0/3645847644 conn(0x7fabb406ae60 msgr2=0x7fabb40d1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:46.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.711+0000 7fabbc647640 1 -- 172.21.15.76:0/3645847644 shutdown_connections 2024-03-08T06:00:46.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.711+0000 7fabbc647640 1 -- 172.21.15.76:0/3645847644 wait complete. 2024-03-08T06:00:46.722 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T06:00:46.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.824+0000 7f826c02a640 1 Processor -- start 2024-03-08T06:00:46.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.824+0000 7f826c02a640 1 -- start start 2024-03-08T06:00:46.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.825+0000 7f826c02a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f82640ccf00 0x7f82640cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:46.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.825+0000 7f826c02a640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82640cfae0 con 0x7f826406c0d0 2024-03-08T06:00:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.825+0000 7f826c02a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82640cfdc0 con 0x7f8264156070 2024-03-08T06:00:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.825+0000 7f826c02a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8264156450 con 0x7f82640ccf00 2024-03-08T06:00:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.825+0000 7f826a5c7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f82640ccf00 0x7f82640cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.825+0000 7f82695c5640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f826406c0d0 0x7f82640cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33824/0 (socket says 172.21.15.76:33824) 2024-03-08T06:00:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.825+0000 7f82695c5640 1 -- 172.21.15.76:0/2696399992 learned_addr learned my addr 172.21.15.76:0/2696399992 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.825+0000 7f826a5c7640 1 -- 172.21.15.76:0/2696399992 >> v1:172.21.15.81:6789/0 conn(0x7f8264156070 legacy=0x7f826415c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.825+0000 7f826a5c7640 1 -- 172.21.15.76:0/2696399992 >> v1:172.21.15.76:6789/0 conn(0x7f826406c0d0 legacy=0x7f82640cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.825+0000 7f826a5c7640 1 -- 172.21.15.76:0/2696399992 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826415c130 con 0x7f82640ccf00 2024-03-08T06:00:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.825+0000 7f8268dc4640 1 -- 172.21.15.76:0/2696399992 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996338041 0 0) 0x7f82640cfae0 con 0x7f826406c0d0 2024-03-08T06:00:46.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.826+0000 7f826a5c7640 1 --2- 172.21.15.76:0/2696399992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82640ccf00 0x7f82640cf340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f825c009d90 tx=0x7f825c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65d98ce7c4395a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:46.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.826+0000 7f8268dc4640 1 -- 172.21.15.76:0/2696399992 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c039070 con 0x7f82640ccf00 2024-03-08T06:00:46.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.826+0000 7f8268dc4640 1 -- 172.21.15.76:0/2696399992 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c02cd00 con 0x7f82640ccf00 2024-03-08T06:00:46.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.826+0000 7f8268dc4640 1 -- 172.21.15.76:0/2696399992 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c034660 con 0x7f82640ccf00 2024-03-08T06:00:46.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.827+0000 7f826c02a640 1 -- 172.21.15.76:0/2696399992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82640ccf00 msgr2=0x7f82640cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.827+0000 7f826c02a640 1 --2- 172.21.15.76:0/2696399992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82640ccf00 0x7f82640cf340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f825c009d90 tx=0x7f825c02bf50 comp rx=0 tx=0).stop 2024-03-08T06:00:46.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.827+0000 7f826c02a640 1 -- 172.21.15.76:0/2696399992 shutdown_connections 2024-03-08T06:00:46.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.827+0000 7f826c02a640 1 --2- 172.21.15.76:0/2696399992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82640ccf00 0x7f82640cf340 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.827+0000 7f826c02a640 1 -- 172.21.15.76:0/2696399992 >> 172.21.15.76:0/2696399992 conn(0x7f826406dd40 msgr2=0x7f826406e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:46.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.827+0000 7f826c02a640 1 -- 172.21.15.76:0/2696399992 shutdown_connections 2024-03-08T06:00:46.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.827+0000 7f826c02a640 1 -- 172.21.15.76:0/2696399992 wait complete. 2024-03-08T06:00:46.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.828+0000 7f826c02a640 1 Processor -- start 2024-03-08T06:00:46.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.828+0000 7f826c02a640 1 -- start start 2024-03-08T06:00:46.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f826c02a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f826406c0d0 0x7f8264141f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:46.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f826c02a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82640ccf00 0x7f82641565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:46.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f826c02a640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8264156070 0x7f82640727d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:46.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f826c02a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82640cf880 con 0x7f82640ccf00 2024-03-08T06:00:46.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f826c02a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826415c4a0 con 0x7f826406c0d0 2024-03-08T06:00:46.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f826c02a640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826415f510 con 0x7f8264156070 2024-03-08T06:00:46.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f82695c5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82640ccf00 0x7f82641565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:46.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f82695c5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82640ccf00 0x7f82641565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35156/0 (socket says 172.21.15.76:35156) 2024-03-08T06:00:46.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f82695c5640 1 -- 172.21.15.76:0/307885830 learned_addr learned my addr 172.21.15.76:0/307885830 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:46.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f8269dc6640 1 --2- 172.21.15.76:0/307885830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f826406c0d0 0x7f8264141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:46.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f826a5c7640 1 --2- 172.21.15.76:0/307885830 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8264156070 0x7f82640727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:46.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f82695c5640 1 -- 172.21.15.76:0/307885830 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8264156070 msgr2=0x7f82640727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f82695c5640 1 --2- 172.21.15.76:0/307885830 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8264156070 0x7f82640727d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f82695c5640 1 -- 172.21.15.76:0/307885830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f826406c0d0 msgr2=0x7f8264141f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:46.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f82695c5640 1 --2- 172.21.15.76:0/307885830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f826406c0d0 0x7f8264141f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:46.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f82695c5640 1 -- 172.21.15.76:0/307885830 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8264156ca0 con 0x7f82640ccf00 2024-03-08T06:00:46.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f826a5c7640 1 --2- 172.21.15.76:0/307885830 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8264156070 0x7f82640727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:46.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f82695c5640 1 --2- 172.21.15.76:0/307885830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82640ccf00 0x7f82641565e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f825400c4a0 tx=0x7f825400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:46.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f825a7fc640 1 -- 172.21.15.76:0/307885830 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8254018070 con 0x7f82640ccf00 2024-03-08T06:00:46.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f825a7fc640 1 -- 172.21.15.76:0/307885830 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82540074e0 con 0x7f82640ccf00 2024-03-08T06:00:46.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f826c02a640 1 -- 172.21.15.76:0/307885830 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8264156f80 con 0x7f82640ccf00 2024-03-08T06:00:46.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f825a7fc640 1 -- 172.21.15.76:0/307885830 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8254004a20 con 0x7f82640ccf00 2024-03-08T06:00:46.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.829+0000 7f826c02a640 1 -- 172.21.15.76:0/307885830 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8264157540 con 0x7f82640ccf00 2024-03-08T06:00:46.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.830+0000 7f826c02a640 1 -- 172.21.15.76:0/307885830 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826404f030 con 0x7f82640ccf00 2024-03-08T06:00:46.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.831+0000 7f825a7fc640 1 -- 172.21.15.76:0/307885830 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f825401b070 con 0x7f82640ccf00 2024-03-08T06:00:46.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.831+0000 7f825a7fc640 1 --2- 172.21.15.76:0/307885830 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f824806b4d0 0x7f824806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:46.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.831+0000 7f825a7fc640 1 -- 172.21.15.76:0/307885830 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8254089850 con 0x7f82640ccf00 2024-03-08T06:00:46.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.831+0000 7f8269dc6640 1 --2- 172.21.15.76:0/307885830 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f824806b4d0 0x7f824806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:46.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.832+0000 7f8269dc6640 1 --2- 172.21.15.76:0/307885830 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f824806b4d0 0x7f824806d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f82641550c0 tx=0x7f8260009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:46.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:46.836+0000 7f825a7fc640 1 -- 172.21.15.76:0/307885830 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8254058390 con 0x7f82640ccf00 2024-03-08T06:00:47.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.004+0000 7f826c02a640 1 -- 172.21.15.76:0/307885830 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f826415a550 con 0x7f82640ccf00 2024-03-08T06:00:47.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.005+0000 7f825a7fc640 1 -- 172.21.15.76:0/307885830 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8254057c90 con 0x7f82640ccf00 2024-03-08T06:00:47.005 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:00:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.007+0000 7f826c02a640 1 -- 172.21.15.76:0/307885830 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f824806b4d0 msgr2=0x7f824806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.007+0000 7f826c02a640 1 --2- 172.21.15.76:0/307885830 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f824806b4d0 0x7f824806d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f82641550c0 tx=0x7f8260009040 comp rx=0 tx=0).stop 2024-03-08T06:00:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.007+0000 7f826c02a640 1 -- 172.21.15.76:0/307885830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82640ccf00 msgr2=0x7f82641565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.007+0000 7f826c02a640 1 --2- 172.21.15.76:0/307885830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82640ccf00 0x7f82641565e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f825400c4a0 tx=0x7f825400c960 comp rx=0 tx=0).stop 2024-03-08T06:00:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.007+0000 7f826c02a640 1 -- 172.21.15.76:0/307885830 shutdown_connections 2024-03-08T06:00:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.007+0000 7f826c02a640 1 --2- 172.21.15.76:0/307885830 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f824806b4d0 0x7f824806d990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.007+0000 7f826c02a640 1 --2- 172.21.15.76:0/307885830 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8264156070 0x7f82640727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.007+0000 7f826c02a640 1 --2- 172.21.15.76:0/307885830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82640ccf00 0x7f82641565e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.007+0000 7f826c02a640 1 --2- 172.21.15.76:0/307885830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f826406c0d0 0x7f8264141f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.007+0000 7f826c02a640 1 -- 172.21.15.76:0/307885830 >> 172.21.15.76:0/307885830 conn(0x7f826406dd40 msgr2=0x7f826415f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.007+0000 7f826c02a640 1 -- 172.21.15.76:0/307885830 shutdown_connections 2024-03-08T06:00:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.007+0000 7f826c02a640 1 -- 172.21.15.76:0/307885830 wait complete. 2024-03-08T06:00:47.018 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T06:00:47.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.119+0000 7fc386010640 1 Processor -- start 2024-03-08T06:00:47.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.119+0000 7fc386010640 1 -- start start 2024-03-08T06:00:47.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.119+0000 7fc386010640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc38015dbc0 0x7fc3800d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.119+0000 7fc386010640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3800d27d0 con 0x7fc3800b9840 2024-03-08T06:00:47.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.119+0000 7fc386010640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3800d2ab0 con 0x7fc3800b5cf0 2024-03-08T06:00:47.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.119+0000 7fc386010640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3800d2d90 con 0x7fc38015dbc0 2024-03-08T06:00:47.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.120+0000 7fc37f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc38015dbc0 0x7fc3800d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.120+0000 7fc37f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc38015dbc0 0x7fc3800d1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35160/0 (socket says 172.21.15.76:35160) 2024-03-08T06:00:47.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.120+0000 7fc37effd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc3800b9840 0x7fc3800b55e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33840/0 (socket says 172.21.15.76:33840) 2024-03-08T06:00:47.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.120+0000 7fc37f7fe640 1 -- 172.21.15.76:0/3682284104 learned_addr learned my addr 172.21.15.76:0/3682284104 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:47.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.120+0000 7fc37f7fe640 1 -- 172.21.15.76:0/3682284104 >> v1:172.21.15.81:6789/0 conn(0x7fc3800b5cf0 legacy=0x7fc3800b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.120+0000 7fc37f7fe640 1 -- 172.21.15.76:0/3682284104 >> v1:172.21.15.76:6789/0 conn(0x7fc3800b9840 legacy=0x7fc3800b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.120+0000 7fc37f7fe640 1 -- 172.21.15.76:0/3682284104 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3800d3070 con 0x7fc38015dbc0 2024-03-08T06:00:47.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.120+0000 7fc37e7fc640 1 -- 172.21.15.76:0/3682284104 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406979501 0 0) 0x7fc3800d27d0 con 0x7fc3800b9840 2024-03-08T06:00:47.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.121+0000 7fc37f7fe640 1 --2- 172.21.15.76:0/3682284104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc38015dbc0 0x7fc3800d1ef0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc370009870 tx=0x7fc37002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09ab7dd18d1eed0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.121+0000 7fc37e7fc640 1 -- 172.21.15.76:0/3682284104 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37002f840 con 0x7fc38015dbc0 2024-03-08T06:00:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.121+0000 7fc37e7fc640 1 -- 172.21.15.76:0/3682284104 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37002f9e0 con 0x7fc38015dbc0 2024-03-08T06:00:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.121+0000 7fc37e7fc640 1 -- 172.21.15.76:0/3682284104 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37002fdb0 con 0x7fc38015dbc0 2024-03-08T06:00:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.121+0000 7fc386010640 1 -- 172.21.15.76:0/3682284104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc38015dbc0 msgr2=0x7fc3800d1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.121+0000 7fc386010640 1 --2- 172.21.15.76:0/3682284104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc38015dbc0 0x7fc3800d1ef0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc370009870 tx=0x7fc37002ba20 comp rx=0 tx=0).stop 2024-03-08T06:00:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.121+0000 7fc386010640 1 -- 172.21.15.76:0/3682284104 shutdown_connections 2024-03-08T06:00:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.121+0000 7fc386010640 1 --2- 172.21.15.76:0/3682284104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc38015dbc0 0x7fc3800d1ef0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.121+0000 7fc386010640 1 -- 172.21.15.76:0/3682284104 >> 172.21.15.76:0/3682284104 conn(0x7fc3800670d0 msgr2=0x7fc3800d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.121+0000 7fc386010640 1 -- 172.21.15.76:0/3682284104 shutdown_connections 2024-03-08T06:00:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.121+0000 7fc386010640 1 -- 172.21.15.76:0/3682284104 wait complete. 2024-03-08T06:00:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.122+0000 7fc386010640 1 Processor -- start 2024-03-08T06:00:47.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.123+0000 7fc386010640 1 -- start start 2024-03-08T06:00:47.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.123+0000 7fc386010640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3800b5cf0 0x7fc3800cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.123+0000 7fc386010640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3800b9840 0x7fc3800cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.123+0000 7fc386010640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc38015dbc0 0x7fc3801f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.123+0000 7fc386010640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3800d25b0 con 0x7fc3800b5cf0 2024-03-08T06:00:47.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.123+0000 7fc386010640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3800d2430 con 0x7fc3800b9840 2024-03-08T06:00:47.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.123+0000 7fc386010640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38015dfa0 con 0x7fc38015dbc0 2024-03-08T06:00:47.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.123+0000 7fc37f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3800b5cf0 0x7fc3800cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.123+0000 7fc37f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3800b5cf0 0x7fc3800cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35164/0 (socket says 172.21.15.76:35164) 2024-03-08T06:00:47.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.123+0000 7fc37f7fe640 1 -- 172.21.15.76:0/273057806 learned_addr learned my addr 172.21.15.76:0/273057806 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:47.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.124+0000 7fc37f7fe640 1 -- 172.21.15.76:0/273057806 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc38015dbc0 msgr2=0x7fc3801f35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.124+0000 7fc37f7fe640 1 --2- 172.21.15.76:0/273057806 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc38015dbc0 0x7fc3801f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.124+0000 7fc37f7fe640 1 -- 172.21.15.76:0/273057806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3800b9840 msgr2=0x7fc3800cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.124+0000 7fc37f7fe640 1 --2- 172.21.15.76:0/273057806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3800b9840 0x7fc3800cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.124+0000 7fc37f7fe640 1 -- 172.21.15.76:0/273057806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3800cb6b0 con 0x7fc3800b5cf0 2024-03-08T06:00:47.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.124+0000 7fc37f7fe640 1 --2- 172.21.15.76:0/273057806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3800b5cf0 0x7fc3800cb0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc370009870 tx=0x7fc370004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:47.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.124+0000 7fc367fff640 1 -- 172.21.15.76:0/273057806 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc370041400 con 0x7fc3800b5cf0 2024-03-08T06:00:47.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.124+0000 7fc367fff640 1 -- 172.21.15.76:0/273057806 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc370046070 con 0x7fc3800b5cf0 2024-03-08T06:00:47.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.124+0000 7fc386010640 1 -- 172.21.15.76:0/273057806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3800cb990 con 0x7fc3800b5cf0 2024-03-08T06:00:47.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.124+0000 7fc367fff640 1 -- 172.21.15.76:0/273057806 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc370041800 con 0x7fc3800b5cf0 2024-03-08T06:00:47.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.124+0000 7fc386010640 1 -- 172.21.15.76:0/273057806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3800cbf50 con 0x7fc3800b5cf0 2024-03-08T06:00:47.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.125+0000 7fc386010640 1 -- 172.21.15.76:0/273057806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3800cc390 con 0x7fc3800b5cf0 2024-03-08T06:00:47.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.125+0000 7fc367fff640 1 -- 172.21.15.76:0/273057806 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc370040070 con 0x7fc3800b5cf0 2024-03-08T06:00:47.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.126+0000 7fc367fff640 1 --2- 172.21.15.76:0/273057806 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc36c06b4d0 0x7fc36c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.126+0000 7fc367fff640 1 -- 172.21.15.76:0/273057806 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc3700acd90 con 0x7fc3800b5cf0 2024-03-08T06:00:47.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.126+0000 7fc37effd640 1 --2- 172.21.15.76:0/273057806 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc36c06b4d0 0x7fc36c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.126+0000 7fc37effd640 1 --2- 172.21.15.76:0/273057806 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc36c06b4d0 0x7fc36c06d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc374007fd0 tx=0x7fc374007900 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:47.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.130+0000 7fc367fff640 1 -- 172.21.15.76:0/273057806 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc370033030 con 0x7fc3800b5cf0 2024-03-08T06:00:47.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.313+0000 7fc386010640 1 -- 172.21.15.76:0/273057806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc3800d1a00 con 0x7fc3800b5cf0 2024-03-08T06:00:47.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.313+0000 7fc367fff640 1 -- 172.21.15.76:0/273057806 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc37007b8d0 con 0x7fc3800b5cf0 2024-03-08T06:00:47.314 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-03-08T06:00:47.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.315+0000 7fc386010640 1 -- 172.21.15.76:0/273057806 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc36c06b4d0 msgr2=0x7fc36c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.315+0000 7fc386010640 1 --2- 172.21.15.76:0/273057806 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc36c06b4d0 0x7fc36c06d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc374007fd0 tx=0x7fc374007900 comp rx=0 tx=0).stop 2024-03-08T06:00:47.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.315+0000 7fc386010640 1 -- 172.21.15.76:0/273057806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3800b5cf0 msgr2=0x7fc3800cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.315+0000 7fc386010640 1 --2- 172.21.15.76:0/273057806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3800b5cf0 0x7fc3800cb0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc370009870 tx=0x7fc370004730 comp rx=0 tx=0).stop 2024-03-08T06:00:47.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.315+0000 7fc386010640 1 -- 172.21.15.76:0/273057806 shutdown_connections 2024-03-08T06:00:47.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.315+0000 7fc386010640 1 --2- 172.21.15.76:0/273057806 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc36c06b4d0 0x7fc36c06d990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.315+0000 7fc386010640 1 --2- 172.21.15.76:0/273057806 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc38015dbc0 0x7fc3801f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.315+0000 7fc386010640 1 --2- 172.21.15.76:0/273057806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3800b9840 0x7fc3800cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.315+0000 7fc386010640 1 --2- 172.21.15.76:0/273057806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3800b5cf0 0x7fc3800cb0d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.315+0000 7fc386010640 1 -- 172.21.15.76:0/273057806 >> 172.21.15.76:0/273057806 conn(0x7fc3800670d0 msgr2=0x7fc3800d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:47.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.315+0000 7fc386010640 1 -- 172.21.15.76:0/273057806 shutdown_connections 2024-03-08T06:00:47.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.315+0000 7fc386010640 1 -- 172.21.15.76:0/273057806 wait complete. 2024-03-08T06:00:47.326 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:00:47.326 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:00:47.342 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:00:47.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.481+0000 7fa8af5ac640 1 Processor -- start 2024-03-08T06:00:47.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.481+0000 7fa8af5ac640 1 -- start start 2024-03-08T06:00:47.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.482+0000 7fa8af5ac640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa8a80cc400 0x7fa8a815aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.482+0000 7fa8af5ac640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a815b570 con 0x7fa8a80cdc50 2024-03-08T06:00:47.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.482+0000 7fa8af5ac640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a815b850 con 0x7fa8a80ceee0 2024-03-08T06:00:47.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.482+0000 7fa8af5ac640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a815bea0 con 0x7fa8a80cc400 2024-03-08T06:00:47.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.482+0000 7fa8acb47640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa8a80cdc50 0x7fa8a80ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33846/0 (socket says 172.21.15.76:33846) 2024-03-08T06:00:47.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.482+0000 7fa8acb47640 1 -- 172.21.15.76:0/2079617951 learned_addr learned my addr 172.21.15.76:0/2079617951 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:47.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa8adb49640 1 --2- 172.21.15.76:0/2079617951 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa8a80cc400 0x7fa8a815aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 -- 172.21.15.76:0/2079617951 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265682066 0 0) 0x7fa8a815b570 con 0x7fa8a80cdc50 2024-03-08T06:00:47.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 -- 172.21.15.76:0/2079617951 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa894005730 con 0x7fa8a80cdc50 2024-03-08T06:00:47.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 -- 172.21.15.76:0/2079617951 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956274262 0 0) 0x7fa8a815b850 con 0x7fa8a80ceee0 2024-03-08T06:00:47.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 -- 172.21.15.76:0/2079617951 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8a815b570 con 0x7fa8a80ceee0 2024-03-08T06:00:47.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 -- 172.21.15.76:0/2079617951 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 758136613 0 0) 0x7fa894005730 con 0x7fa8a80cdc50 2024-03-08T06:00:47.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 -- 172.21.15.76:0/2079617951 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa8a80cc400 msgr2=0x7fa8a815aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:47.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 --2- 172.21.15.76:0/2079617951 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa8a80cc400 0x7fa8a815aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 -- 172.21.15.76:0/2079617951 >> v1:172.21.15.81:6789/0 conn(0x7fa8a80ceee0 legacy=0x7fa8a80cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 -- 172.21.15.76:0/2079617951 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a815bb30 con 0x7fa8a80cdc50 2024-03-08T06:00:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 -- 172.21.15.76:0/2079617951 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fa8980015f0 con 0x7fa8a80cdc50 2024-03-08T06:00:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 -- 172.21.15.76:0/2079617951 >> v1:172.21.15.76:6789/0 conn(0x7fa8a80cdc50 legacy=0x7fa8a80ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 --2- 172.21.15.76:0/2079617951 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa894005df0 0x7fa8940061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 --2- 172.21.15.76:0/2079617951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa894006a10 0x7fa894006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 --2- 172.21.15.76:0/2079617951 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa8a80cc400 0x7fa894007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 -- 172.21.15.76:0/2079617951 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a815bea0 con 0x7fa894005df0 2024-03-08T06:00:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 -- 172.21.15.76:0/2079617951 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89400dd30 con 0x7fa894006a10 2024-03-08T06:00:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa89ffff640 1 -- 172.21.15.76:0/2079617951 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89400df30 con 0x7fa8a80cc400 2024-03-08T06:00:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa8ad348640 1 --2- 172.21.15.76:0/2079617951 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa894005df0 0x7fa8940061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa8acb47640 1 --2- 172.21.15.76:0/2079617951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa894006a10 0x7fa894006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa8adb49640 1 --2- 172.21.15.76:0/2079617951 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa8a80cc400 0x7fa894007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa8ad348640 1 -- 172.21.15.76:0/2079617951 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa8a80cc400 msgr2=0x7fa894007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa8ad348640 1 --2- 172.21.15.76:0/2079617951 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa8a80cc400 0x7fa894007410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa8ad348640 1 -- 172.21.15.76:0/2079617951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa894006a10 msgr2=0x7fa894006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa8ad348640 1 --2- 172.21.15.76:0/2079617951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa894006a10 0x7fa894006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.483+0000 7fa8ad348640 1 -- 172.21.15.76:0/2079617951 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa894005730 con 0x7fa894005df0 2024-03-08T06:00:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.484+0000 7fa8ad348640 1 --2- 172.21.15.76:0/2079617951 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa894005df0 0x7fa8940061f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4001fd0 tx=0x7fa894007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5515f9e7d73d52f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.484+0000 7fa89ffff640 1 -- 172.21.15.76:0/2079617951 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4001bf0 con 0x7fa894005df0 2024-03-08T06:00:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.484+0000 7fa89ffff640 1 -- 172.21.15.76:0/2079617951 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a4003b70 con 0x7fa894005df0 2024-03-08T06:00:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.484+0000 7fa89ffff640 1 -- 172.21.15.76:0/2079617951 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a40050d0 con 0x7fa894005df0 2024-03-08T06:00:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.485+0000 7fa8af5ac640 1 -- 172.21.15.76:0/2079617951 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa894005df0 msgr2=0x7fa8940061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.485+0000 7fa8af5ac640 1 --2- 172.21.15.76:0/2079617951 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa894005df0 0x7fa8940061f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4001fd0 tx=0x7fa894007a40 comp rx=0 tx=0).stop 2024-03-08T06:00:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.485+0000 7fa8adb49640 1 -- 172.21.15.76:0/2079617951 reap_dead start 2024-03-08T06:00:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.485+0000 7fa8af5ac640 1 -- 172.21.15.76:0/2079617951 shutdown_connections 2024-03-08T06:00:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.485+0000 7fa8af5ac640 1 -- 172.21.15.76:0/2079617951 >> 172.21.15.76:0/2079617951 conn(0x7fa8a8099a60 msgr2=0x7fa8a80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.485+0000 7fa8af5ac640 1 -- 172.21.15.76:0/2079617951 shutdown_connections 2024-03-08T06:00:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.485+0000 7fa8af5ac640 1 -- 172.21.15.76:0/2079617951 wait complete. 2024-03-08T06:00:47.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.486+0000 7fa8af5ac640 1 Processor -- start 2024-03-08T06:00:47.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.486+0000 7fa8af5ac640 1 -- start start 2024-03-08T06:00:47.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8af5ac640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa894005df0 0x7fa8a8159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8af5ac640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8a80c5c10 0x7fa8a80ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8af5ac640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a8159060 0x7fa8a815c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8af5ac640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a815b320 con 0x7fa8a8159060 2024-03-08T06:00:47.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8af5ac640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a815b1a0 con 0x7fa8a80c5c10 2024-03-08T06:00:47.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8af5ac640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a80cec00 con 0x7fa894005df0 2024-03-08T06:00:47.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8adb49640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a8159060 0x7fa8a815c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8adb49640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a8159060 0x7fa8a815c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35194/0 (socket says 172.21.15.76:35194) 2024-03-08T06:00:47.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8adb49640 1 -- 172.21.15.76:0/1328632175 learned_addr learned my addr 172.21.15.76:0/1328632175 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:47.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8acb47640 1 --2- 172.21.15.76:0/1328632175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8a80c5c10 0x7fa8a80ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8ad348640 1 --2- 172.21.15.76:0/1328632175 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa894005df0 0x7fa8a8159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8adb49640 1 -- 172.21.15.76:0/1328632175 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa894005df0 msgr2=0x7fa8a8159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8adb49640 1 --2- 172.21.15.76:0/1328632175 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa894005df0 0x7fa8a8159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8adb49640 1 -- 172.21.15.76:0/1328632175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8a80c5c10 msgr2=0x7fa8a80ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8adb49640 1 --2- 172.21.15.76:0/1328632175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8a80c5c10 0x7fa8a80ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8adb49640 1 -- 172.21.15.76:0/1328632175 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a80c62a0 con 0x7fa8a8159060 2024-03-08T06:00:47.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8adb49640 1 --2- 172.21.15.76:0/1328632175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a8159060 0x7fa8a815c380 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa8a815e6f0 tx=0x7fa8a0011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:47.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa89dffb640 1 -- 172.21.15.76:0/1328632175 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0004240 con 0x7fa8a8159060 2024-03-08T06:00:47.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa89dffb640 1 -- 172.21.15.76:0/1328632175 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a001b070 con 0x7fa8a8159060 2024-03-08T06:00:47.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa8af5ac640 1 -- 172.21.15.76:0/1328632175 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a80c6580 con 0x7fa8a8159060 2024-03-08T06:00:47.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.487+0000 7fa89dffb640 1 -- 172.21.15.76:0/1328632175 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a000f770 con 0x7fa8a8159060 2024-03-08T06:00:47.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.488+0000 7fa8af5ac640 1 -- 172.21.15.76:0/1328632175 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a80c9a00 con 0x7fa8a8159060 2024-03-08T06:00:47.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.489+0000 7fa8af5ac640 1 -- 172.21.15.76:0/1328632175 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a804f030 con 0x7fa8a8159060 2024-03-08T06:00:47.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.489+0000 7fa89dffb640 1 -- 172.21.15.76:0/1328632175 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8a0012040 con 0x7fa8a8159060 2024-03-08T06:00:47.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.489+0000 7fa89dffb640 1 --2- 172.21.15.76:0/1328632175 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa88c06b4d0 0x7fa88c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.490+0000 7fa8ad348640 1 --2- 172.21.15.76:0/1328632175 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa88c06b4d0 0x7fa88c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.490+0000 7fa89dffb640 1 -- 172.21.15.76:0/1328632175 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa8a008a250 con 0x7fa8a8159060 2024-03-08T06:00:47.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.490+0000 7fa8ad348640 1 --2- 172.21.15.76:0/1328632175 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa88c06b4d0 0x7fa88c06d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4034000 tx=0x7fa8a4009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:47.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.494+0000 7fa89dffb640 1 -- 172.21.15.76:0/1328632175 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8a0058740 con 0x7fa8a8159060 2024-03-08T06:00:47.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.666+0000 7fa8af5ac640 1 -- 172.21.15.76:0/1328632175 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8a80b68a0 con 0x7fa8a8159060 2024-03-08T06:00:47.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.668+0000 7fa89dffb640 1 -- 172.21.15.76:0/1328632175 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18430 (secure 0 0 0) 0x7fa8a005c7e0 con 0x7fa8a8159060 2024-03-08T06:00:47.668 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:47.669 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":26,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:42.202011+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:00:47.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.670+0000 7fa8af5ac640 1 -- 172.21.15.76:0/1328632175 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa88c06b4d0 msgr2=0x7fa88c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.670+0000 7fa8af5ac640 1 --2- 172.21.15.76:0/1328632175 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa88c06b4d0 0x7fa88c06d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4034000 tx=0x7fa8a4009030 comp rx=0 tx=0).stop 2024-03-08T06:00:47.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.670+0000 7fa8af5ac640 1 -- 172.21.15.76:0/1328632175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a8159060 msgr2=0x7fa8a815c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.670+0000 7fa8af5ac640 1 --2- 172.21.15.76:0/1328632175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a8159060 0x7fa8a815c380 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa8a815e6f0 tx=0x7fa8a0011cf0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.670+0000 7fa8af5ac640 1 -- 172.21.15.76:0/1328632175 shutdown_connections 2024-03-08T06:00:47.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.670+0000 7fa8af5ac640 1 --2- 172.21.15.76:0/1328632175 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa88c06b4d0 0x7fa88c06d990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.670+0000 7fa8af5ac640 1 --2- 172.21.15.76:0/1328632175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a8159060 0x7fa8a815c380 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.670+0000 7fa8af5ac640 1 --2- 172.21.15.76:0/1328632175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8a80c5c10 0x7fa8a80ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.670+0000 7fa8af5ac640 1 --2- 172.21.15.76:0/1328632175 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa894005df0 0x7fa8a8159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.670+0000 7fa8af5ac640 1 -- 172.21.15.76:0/1328632175 >> 172.21.15.76:0/1328632175 conn(0x7fa8a8099a60 msgr2=0x7fa8a815ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:47.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.670+0000 7fa8af5ac640 1 -- 172.21.15.76:0/1328632175 shutdown_connections 2024-03-08T06:00:47.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.670+0000 7fa8af5ac640 1 -- 172.21.15.76:0/1328632175 wait complete. 2024-03-08T06:00:47.681 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:00:33.926651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:00:34.474030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T06:00:38.207853+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T06:00:47.682 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:00:47.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.784+0000 7f9587d8c640 1 Processor -- start 2024-03-08T06:00:47.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.785+0000 7f9587d8c640 1 -- start start 2024-03-08T06:00:47.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.785+0000 7f9587d8c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f958015dbc0 0x7f95800d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.785+0000 7f9587d8c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95800b9840 0x7f95800b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.785+0000 7f9587d8c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f95800b5b20 0x7f958006c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.785+0000 7f9587d8c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95800b5f90 con 0x7f95800b9840 2024-03-08T06:00:47.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.785+0000 7f9587d8c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95800d27d0 con 0x7f958015dbc0 2024-03-08T06:00:47.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.785+0000 7f9587d8c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95800d2ab0 con 0x7f95800b5b20 2024-03-08T06:00:47.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.786+0000 7f9585327640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95800b9840 0x7f95800b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.786+0000 7f9585327640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95800b9840 0x7f95800b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35200/0 (socket says 172.21.15.76:35200) 2024-03-08T06:00:47.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.786+0000 7f9585327640 1 -- 172.21.15.76:0/2285037147 learned_addr learned my addr 172.21.15.76:0/2285037147 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:47.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.786+0000 7f9586329640 1 --2- 172.21.15.76:0/2285037147 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f95800b5b20 0x7f958006c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.786+0000 7f9585b28640 1 --2- 172.21.15.76:0/2285037147 >> v2:172.21.15.81:3300/0 conn(0x7f958015dbc0 0x7f95800d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.786+0000 7f9585327640 1 -- 172.21.15.76:0/2285037147 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f95800b5b20 msgr2=0x7f958006c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:47.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.786+0000 7f9585327640 1 --2- 172.21.15.76:0/2285037147 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f95800b5b20 0x7f958006c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.786+0000 7f9585327640 1 -- 172.21.15.76:0/2285037147 >> v2:172.21.15.81:3300/0 conn(0x7f958015dbc0 msgr2=0x7f95800d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:47.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.786+0000 7f9585327640 1 --2- 172.21.15.76:0/2285037147 >> v2:172.21.15.81:3300/0 conn(0x7f958015dbc0 0x7f95800d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.786+0000 7f9585327640 1 -- 172.21.15.76:0/2285037147 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95800d2d90 con 0x7f95800b9840 2024-03-08T06:00:47.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.787+0000 7f9585327640 1 --2- 172.21.15.76:0/2285037147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95800b9840 0x7f95800b55e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f958006d280 tx=0x7f957002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c1fdb9eeead3eb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:47.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.787+0000 7f9584b26640 1 -- 172.21.15.76:0/2285037147 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570035020 con 0x7f95800b9840 2024-03-08T06:00:47.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.787+0000 7f9584b26640 1 -- 172.21.15.76:0/2285037147 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9570031870 con 0x7f95800b9840 2024-03-08T06:00:47.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.787+0000 7f9584b26640 1 -- 172.21.15.76:0/2285037147 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570031c40 con 0x7f95800b9840 2024-03-08T06:00:47.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.787+0000 7f9587d8c640 1 -- 172.21.15.76:0/2285037147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95800b9840 msgr2=0x7f95800b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.787+0000 7f9587d8c640 1 --2- 172.21.15.76:0/2285037147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95800b9840 0x7f95800b55e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f958006d280 tx=0x7f957002b910 comp rx=0 tx=0).stop 2024-03-08T06:00:47.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.787+0000 7f9587d8c640 1 -- 172.21.15.76:0/2285037147 shutdown_connections 2024-03-08T06:00:47.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.787+0000 7f9587d8c640 1 --2- 172.21.15.76:0/2285037147 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f95800b5b20 0x7f958006c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.787+0000 7f9587d8c640 1 --2- 172.21.15.76:0/2285037147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95800b9840 0x7f95800b55e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.787+0000 7f9587d8c640 1 --2- 172.21.15.76:0/2285037147 >> v2:172.21.15.81:3300/0 conn(0x7f958015dbc0 0x7f95800d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.787+0000 7f9587d8c640 1 -- 172.21.15.76:0/2285037147 >> 172.21.15.76:0/2285037147 conn(0x7f95800670d0 msgr2=0x7f95800d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:47.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.787+0000 7f9587d8c640 1 -- 172.21.15.76:0/2285037147 shutdown_connections 2024-03-08T06:00:47.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.787+0000 7f9587d8c640 1 -- 172.21.15.76:0/2285037147 wait complete. 2024-03-08T06:00:47.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.788+0000 7f9587d8c640 1 Processor -- start 2024-03-08T06:00:47.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.789+0000 7f9587d8c640 1 -- start start 2024-03-08T06:00:47.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.789+0000 7f9587d8c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95800b5b20 0x7f95801f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.789+0000 7f9587d8c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95800b9840 0x7f958014a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.789+0000 7f9587d8c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f958015dbc0 0x7f95800ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.789+0000 7f9587d8c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95800d2430 con 0x7f95800b5b20 2024-03-08T06:00:47.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.789+0000 7f9587d8c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95800d3870 con 0x7f95800b9840 2024-03-08T06:00:47.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.789+0000 7f9587d8c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95800d25b0 con 0x7f958015dbc0 2024-03-08T06:00:47.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.789+0000 7f9585b28640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95800b5b20 0x7f95801f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.789+0000 7f9585b28640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95800b5b20 0x7f95801f6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35214/0 (socket says 172.21.15.76:35214) 2024-03-08T06:00:47.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.789+0000 7f9585b28640 1 -- 172.21.15.76:0/270240087 learned_addr learned my addr 172.21.15.76:0/270240087 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:47.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.790+0000 7f9586329640 1 --2- 172.21.15.76:0/270240087 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f958015dbc0 0x7f95800ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.790+0000 7f9585327640 1 --2- 172.21.15.76:0/270240087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95800b9840 0x7f958014a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.790+0000 7f9585b28640 1 -- 172.21.15.76:0/270240087 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f958015dbc0 msgr2=0x7f95800ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.790+0000 7f9585b28640 1 --2- 172.21.15.76:0/270240087 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f958015dbc0 0x7f95800ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.790+0000 7f9585b28640 1 -- 172.21.15.76:0/270240087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95800b9840 msgr2=0x7f958014a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.790+0000 7f9585b28640 1 --2- 172.21.15.76:0/270240087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95800b9840 0x7f958014a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.790+0000 7f9585b28640 1 -- 172.21.15.76:0/270240087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95801f7b50 con 0x7f95800b5b20 2024-03-08T06:00:47.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.790+0000 7f9585b28640 1 --2- 172.21.15.76:0/270240087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95800b5b20 0x7f95801f6c90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f957c00e870 tx=0x7f957c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:47.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.790+0000 7f95767fc640 1 -- 172.21.15.76:0/270240087 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c017070 con 0x7f95800b5b20 2024-03-08T06:00:47.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.790+0000 7f95767fc640 1 -- 172.21.15.76:0/270240087 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957c00c940 con 0x7f95800b5b20 2024-03-08T06:00:47.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.790+0000 7f95767fc640 1 -- 172.21.15.76:0/270240087 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c00cd30 con 0x7f95800b5b20 2024-03-08T06:00:47.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.790+0000 7f9587d8c640 1 -- 172.21.15.76:0/270240087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95801f7590 con 0x7f95800b5b20 2024-03-08T06:00:47.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.790+0000 7f9587d8c640 1 -- 172.21.15.76:0/270240087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9580149810 con 0x7f95800b5b20 2024-03-08T06:00:47.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.791+0000 7f9587d8c640 1 -- 172.21.15.76:0/270240087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958004f030 con 0x7f95800b5b20 2024-03-08T06:00:47.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.791+0000 7f95767fc640 1 -- 172.21.15.76:0/270240087 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f957c01a070 con 0x7f95800b5b20 2024-03-08T06:00:47.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.792+0000 7f95767fc640 1 --2- 172.21.15.76:0/270240087 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f956406b4d0 0x7f956406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:47.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.792+0000 7f95767fc640 1 -- 172.21.15.76:0/270240087 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f957c08a6b0 con 0x7f95800b5b20 2024-03-08T06:00:47.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.792+0000 7f9585327640 1 --2- 172.21.15.76:0/270240087 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f956406b4d0 0x7f956406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:47.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.792+0000 7f9585327640 1 --2- 172.21.15.76:0/270240087 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f956406b4d0 0x7f956406d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9570036000 tx=0x7f957003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:47.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.796+0000 7f95767fc640 1 -- 172.21.15.76:0/270240087 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f957c058ba0 con 0x7f95800b5b20 2024-03-08T06:00:47.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.965+0000 7f9587d8c640 1 -- 172.21.15.76:0/270240087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95800cec20 con 0x7f95800b5b20 2024-03-08T06:00:47.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.966+0000 7f95767fc640 1 -- 172.21.15.76:0/270240087 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f957c05cc40 con 0x7f95800b5b20 2024-03-08T06:00:47.966 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:00:47.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.967+0000 7f9587d8c640 1 -- 172.21.15.76:0/270240087 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f956406b4d0 msgr2=0x7f956406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.967+0000 7f9587d8c640 1 --2- 172.21.15.76:0/270240087 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f956406b4d0 0x7f956406d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9570036000 tx=0x7f957003c040 comp rx=0 tx=0).stop 2024-03-08T06:00:47.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.967+0000 7f9587d8c640 1 -- 172.21.15.76:0/270240087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95800b5b20 msgr2=0x7f95801f6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:47.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.967+0000 7f9587d8c640 1 --2- 172.21.15.76:0/270240087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95800b5b20 0x7f95801f6c90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f957c00e870 tx=0x7f957c00ed30 comp rx=0 tx=0).stop 2024-03-08T06:00:47.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.967+0000 7f9587d8c640 1 -- 172.21.15.76:0/270240087 shutdown_connections 2024-03-08T06:00:47.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.967+0000 7f9587d8c640 1 --2- 172.21.15.76:0/270240087 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f956406b4d0 0x7f956406d990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.967+0000 7f9587d8c640 1 --2- 172.21.15.76:0/270240087 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f958015dbc0 0x7f95800ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.967+0000 7f9587d8c640 1 --2- 172.21.15.76:0/270240087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95800b9840 0x7f958014a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.967+0000 7f9587d8c640 1 --2- 172.21.15.76:0/270240087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95800b5b20 0x7f95801f6c90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:47.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.968+0000 7f9587d8c640 1 -- 172.21.15.76:0/270240087 >> 172.21.15.76:0/270240087 conn(0x7f95800670d0 msgr2=0x7f95800c9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:47.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.968+0000 7f9587d8c640 1 -- 172.21.15.76:0/270240087 shutdown_connections 2024-03-08T06:00:47.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:47.968+0000 7f9587d8c640 1 -- 172.21.15.76:0/270240087 wait complete. 2024-03-08T06:00:47.979 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:00:48.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.081+0000 7f8848684640 1 Processor -- start 2024-03-08T06:00:48.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.081+0000 7f8848684640 1 -- start start 2024-03-08T06:00:48.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.082+0000 7f8848684640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f88400cfbb0 0x7f884006b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.082+0000 7f8848684640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f88400caee0 0x7f88400cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.082+0000 7f8848684640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f884015e1b0 0x7f88400cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.082+0000 7f8848684640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88400d04d0 con 0x7f88400caee0 2024-03-08T06:00:48.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.082+0000 7f8848684640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88400cb360 con 0x7f884015e1b0 2024-03-08T06:00:48.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.082+0000 7f8848684640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884015e9d0 con 0x7f88400cfbb0 2024-03-08T06:00:48.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.083+0000 7f8845c1f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f88400caee0 0x7f88400cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.083+0000 7f8845c1f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f88400caee0 0x7f88400cff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35220/0 (socket says 172.21.15.76:35220) 2024-03-08T06:00:48.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.083+0000 7f8845c1f640 1 -- 172.21.15.76:0/3740451003 learned_addr learned my addr 172.21.15.76:0/3740451003 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:48.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.083+0000 7f8846420640 1 --2- 172.21.15.76:0/3740451003 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f88400cfbb0 0x7f884006b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.083+0000 7f8846c21640 1 --2- 172.21.15.76:0/3740451003 >> v2:172.21.15.81:3300/0 conn(0x7f884015e1b0 0x7f88400cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.083+0000 7f8845c1f640 1 -- 172.21.15.76:0/3740451003 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f88400cfbb0 msgr2=0x7f884006b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:48.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.083+0000 7f8845c1f640 1 --2- 172.21.15.76:0/3740451003 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f88400cfbb0 0x7f884006b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.083+0000 7f8845c1f640 1 -- 172.21.15.76:0/3740451003 >> v2:172.21.15.81:3300/0 conn(0x7f884015e1b0 msgr2=0x7f88400cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:48.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.083+0000 7f8845c1f640 1 --2- 172.21.15.76:0/3740451003 >> v2:172.21.15.81:3300/0 conn(0x7f884015e1b0 0x7f88400cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.083+0000 7f8845c1f640 1 -- 172.21.15.76:0/3740451003 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884015ecb0 con 0x7f88400caee0 2024-03-08T06:00:48.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.084+0000 7f8845c1f640 1 --2- 172.21.15.76:0/3740451003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88400caee0 0x7f88400cff90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f884006e440 tx=0x7f883002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159c076316e2236f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:48.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.084+0000 7f884541e640 1 -- 172.21.15.76:0/3740451003 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830035020 con 0x7f88400caee0 2024-03-08T06:00:48.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.084+0000 7f884541e640 1 -- 172.21.15.76:0/3740451003 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8830031870 con 0x7f88400caee0 2024-03-08T06:00:48.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.084+0000 7f884541e640 1 -- 172.21.15.76:0/3740451003 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830031c40 con 0x7f88400caee0 2024-03-08T06:00:48.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.084+0000 7f8848684640 1 -- 172.21.15.76:0/3740451003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88400caee0 msgr2=0x7f88400cff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.084+0000 7f8848684640 1 --2- 172.21.15.76:0/3740451003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88400caee0 0x7f88400cff90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f884006e440 tx=0x7f883002b910 comp rx=0 tx=0).stop 2024-03-08T06:00:48.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.084+0000 7f8848684640 1 -- 172.21.15.76:0/3740451003 shutdown_connections 2024-03-08T06:00:48.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.084+0000 7f8848684640 1 --2- 172.21.15.76:0/3740451003 >> v2:172.21.15.81:3300/0 conn(0x7f884015e1b0 0x7f88400cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.084+0000 7f8848684640 1 --2- 172.21.15.76:0/3740451003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88400caee0 0x7f88400cff90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.084+0000 7f8848684640 1 --2- 172.21.15.76:0/3740451003 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f88400cfbb0 0x7f884006b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.084+0000 7f8848684640 1 -- 172.21.15.76:0/3740451003 >> 172.21.15.76:0/3740451003 conn(0x7f88400d0890 msgr2=0x7f88400d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:48.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.084+0000 7f8848684640 1 -- 172.21.15.76:0/3740451003 shutdown_connections 2024-03-08T06:00:48.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.084+0000 7f8848684640 1 -- 172.21.15.76:0/3740451003 wait complete. 2024-03-08T06:00:48.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.085+0000 7f8848684640 1 Processor -- start 2024-03-08T06:00:48.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.086+0000 7f8848684640 1 -- start start 2024-03-08T06:00:48.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.086+0000 7f8848684640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88400caee0 0x7f884014eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.086+0000 7f8848684640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88400cfbb0 0x7f88400c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.086+0000 7f8848684640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f884015e1b0 0x7f88400cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.086+0000 7f8848684640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884015e610 con 0x7f88400caee0 2024-03-08T06:00:48.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.086+0000 7f8848684640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884015f270 con 0x7f88400cfbb0 2024-03-08T06:00:48.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.086+0000 7f8848684640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884015e790 con 0x7f884015e1b0 2024-03-08T06:00:48.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.086+0000 7f8846420640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88400caee0 0x7f884014eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.086+0000 7f8846420640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88400caee0 0x7f884014eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35226/0 (socket says 172.21.15.76:35226) 2024-03-08T06:00:48.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.086+0000 7f8846420640 1 -- 172.21.15.76:0/1186224708 learned_addr learned my addr 172.21.15.76:0/1186224708 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:48.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.086+0000 7f8846c21640 1 --2- 172.21.15.76:0/1186224708 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f884015e1b0 0x7f88400cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.086+0000 7f8845c1f640 1 --2- 172.21.15.76:0/1186224708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88400cfbb0 0x7f88400c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.087+0000 7f8846420640 1 -- 172.21.15.76:0/1186224708 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f884015e1b0 msgr2=0x7f88400cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.087+0000 7f8846420640 1 --2- 172.21.15.76:0/1186224708 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f884015e1b0 0x7f88400cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.087+0000 7f8846420640 1 -- 172.21.15.76:0/1186224708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88400cfbb0 msgr2=0x7f88400c43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.087+0000 7f8846420640 1 --2- 172.21.15.76:0/1186224708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88400cfbb0 0x7f88400c43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.087+0000 7f8846420640 1 -- 172.21.15.76:0/1186224708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88400c4900 con 0x7f88400caee0 2024-03-08T06:00:48.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.087+0000 7f8846420640 1 --2- 172.21.15.76:0/1186224708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88400caee0 0x7f884014eae0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f883c00eed0 tx=0x7f883c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:48.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.087+0000 7f8836ffd640 1 -- 172.21.15.76:0/1186224708 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c018020 con 0x7f88400caee0 2024-03-08T06:00:48.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.087+0000 7f8836ffd640 1 -- 172.21.15.76:0/1186224708 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c014400 con 0x7f88400caee0 2024-03-08T06:00:48.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.087+0000 7f8848684640 1 -- 172.21.15.76:0/1186224708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88400c6ce0 con 0x7f88400caee0 2024-03-08T06:00:48.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.087+0000 7f8836ffd640 1 -- 172.21.15.76:0/1186224708 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c0147d0 con 0x7f88400caee0 2024-03-08T06:00:48.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.087+0000 7f8848684640 1 -- 172.21.15.76:0/1186224708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88400c72a0 con 0x7f88400caee0 2024-03-08T06:00:48.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.088+0000 7f8848684640 1 -- 172.21.15.76:0/1186224708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88400ae500 con 0x7f88400caee0 2024-03-08T06:00:48.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.088+0000 7f8836ffd640 1 -- 172.21.15.76:0/1186224708 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f883c005d10 con 0x7f88400caee0 2024-03-08T06:00:48.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.089+0000 7f8836ffd640 1 --2- 172.21.15.76:0/1186224708 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f882406b4d0 0x7f882406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.089+0000 7f8836ffd640 1 -- 172.21.15.76:0/1186224708 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f883c08ada0 con 0x7f88400caee0 2024-03-08T06:00:48.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.089+0000 7f8845c1f640 1 --2- 172.21.15.76:0/1186224708 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f882406b4d0 0x7f882406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.089+0000 7f8845c1f640 1 --2- 172.21.15.76:0/1186224708 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f882406b4d0 0x7f882406d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8830036000 tx=0x7f883003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:48.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.093+0000 7f8836ffd640 1 -- 172.21.15.76:0/1186224708 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88400ae500 con 0x7f88400caee0 2024-03-08T06:00:48.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.268+0000 7f8848684640 1 -- 172.21.15.76:0/1186224708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f884004f030 con 0x7f88400caee0 2024-03-08T06:00:48.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.268+0000 7f8836ffd640 1 -- 172.21.15.76:0/1186224708 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f883c022030 con 0x7f88400caee0 2024-03-08T06:00:48.269 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:00:48.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.270+0000 7f8848684640 1 -- 172.21.15.76:0/1186224708 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f882406b4d0 msgr2=0x7f882406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.270+0000 7f8848684640 1 --2- 172.21.15.76:0/1186224708 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f882406b4d0 0x7f882406d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8830036000 tx=0x7f883003c040 comp rx=0 tx=0).stop 2024-03-08T06:00:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.270+0000 7f8848684640 1 -- 172.21.15.76:0/1186224708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88400caee0 msgr2=0x7f884014eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.270+0000 7f8848684640 1 --2- 172.21.15.76:0/1186224708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88400caee0 0x7f884014eae0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f883c00eed0 tx=0x7f883c00b480 comp rx=0 tx=0).stop 2024-03-08T06:00:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.270+0000 7f8848684640 1 -- 172.21.15.76:0/1186224708 shutdown_connections 2024-03-08T06:00:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.270+0000 7f8848684640 1 --2- 172.21.15.76:0/1186224708 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f882406b4d0 0x7f882406d990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.270+0000 7f8848684640 1 --2- 172.21.15.76:0/1186224708 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f884015e1b0 0x7f88400cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.270+0000 7f8848684640 1 --2- 172.21.15.76:0/1186224708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88400cfbb0 0x7f88400c43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.270+0000 7f8848684640 1 --2- 172.21.15.76:0/1186224708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88400caee0 0x7f884014eae0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.270+0000 7f8848684640 1 -- 172.21.15.76:0/1186224708 >> 172.21.15.76:0/1186224708 conn(0x7f88400d0890 msgr2=0x7f8840159dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.270+0000 7f8848684640 1 -- 172.21.15.76:0/1186224708 shutdown_connections 2024-03-08T06:00:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.270+0000 7f8848684640 1 -- 172.21.15.76:0/1186224708 wait complete. 2024-03-08T06:00:48.282 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:00:48.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.385+0000 7f6834884640 1 Processor -- start 2024-03-08T06:00:48.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.385+0000 7f6834884640 1 -- start start 2024-03-08T06:00:48.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.386+0000 7f6834884640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f68300b9840 0x7f68300b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.386+0000 7f6834884640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f68300b5b20 0x7f68300b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.386+0000 7f6834884640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68300d27d0 con 0x7f683015dbc0 2024-03-08T06:00:48.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.386+0000 7f6834884640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68300b5f90 con 0x7f68300b5b20 2024-03-08T06:00:48.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.386+0000 7f6834884640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68300d2ab0 con 0x7f68300b9840 2024-03-08T06:00:48.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.386+0000 7f682ed9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f68300b5b20 0x7f68300b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.386+0000 7f682ed9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f68300b5b20 0x7f68300b6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35232/0 (socket says 172.21.15.76:35232) 2024-03-08T06:00:48.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.386+0000 7f682ed9d640 1 -- 172.21.15.76:0/3259819154 learned_addr learned my addr 172.21.15.76:0/3259819154 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:48.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.386+0000 7f6827fff640 1 --2- 172.21.15.76:0/3259819154 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f68300b9840 0x7f68300b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.386+0000 7f682ed9d640 1 -- 172.21.15.76:0/3259819154 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f68300b9840 msgr2=0x7f68300b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:48.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.386+0000 7f682ed9d640 1 --2- 172.21.15.76:0/3259819154 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f68300b9840 0x7f68300b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.386+0000 7f682ed9d640 1 -- 172.21.15.76:0/3259819154 >> v1:172.21.15.81:6789/0 conn(0x7f683015dbc0 legacy=0x7f68300d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:48.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.386+0000 7f682ed9d640 1 -- 172.21.15.76:0/3259819154 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68300d2d90 con 0x7f68300b5b20 2024-03-08T06:00:48.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.386+0000 7f6827fff640 1 --2- 172.21.15.76:0/3259819154 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f68300b9840 0x7f68300b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:48.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.387+0000 7f682ed9d640 1 --2- 172.21.15.76:0/3259819154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68300b5b20 0x7f68300b6c90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6828009e50 tx=0x7f682802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85df34c2ea875e3b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:48.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.387+0000 7f682dd9b640 1 -- 172.21.15.76:0/3259819154 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682802ca70 con 0x7f68300b5b20 2024-03-08T06:00:48.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.387+0000 7f682dd9b640 1 -- 172.21.15.76:0/3259819154 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682802cc10 con 0x7f68300b5b20 2024-03-08T06:00:48.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.387+0000 7f682dd9b640 1 -- 172.21.15.76:0/3259819154 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68280343f0 con 0x7f68300b5b20 2024-03-08T06:00:48.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.387+0000 7f6834884640 1 -- 172.21.15.76:0/3259819154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68300b5b20 msgr2=0x7f68300b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.387+0000 7f6834884640 1 --2- 172.21.15.76:0/3259819154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68300b5b20 0x7f68300b6c90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6828009e50 tx=0x7f682802bed0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.387+0000 7f6834884640 1 -- 172.21.15.76:0/3259819154 shutdown_connections 2024-03-08T06:00:48.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.387+0000 7f6834884640 1 --2- 172.21.15.76:0/3259819154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68300b5b20 0x7f68300b6c90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.387+0000 7f6834884640 1 --2- 172.21.15.76:0/3259819154 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f68300b9840 0x7f68300b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.387+0000 7f6834884640 1 -- 172.21.15.76:0/3259819154 >> 172.21.15.76:0/3259819154 conn(0x7f68300670d0 msgr2=0x7f68300d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:48.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.387+0000 7f6834884640 1 -- 172.21.15.76:0/3259819154 shutdown_connections 2024-03-08T06:00:48.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.387+0000 7f6834884640 1 -- 172.21.15.76:0/3259819154 wait complete. 2024-03-08T06:00:48.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.389+0000 7f6834884640 1 Processor -- start 2024-03-08T06:00:48.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.389+0000 7f6834884640 1 -- start start 2024-03-08T06:00:48.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.389+0000 7f6834884640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f68300b5b20 0x7f6830159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.389+0000 7f6834884640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68300b9840 0x7f68301578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.389+0000 7f6834884640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f683015dbc0 0x7f6830152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.389+0000 7f6834884640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683015dfa0 con 0x7f68300b9840 2024-03-08T06:00:48.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.389+0000 7f6834884640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68300d3870 con 0x7f683015dbc0 2024-03-08T06:00:48.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.389+0000 7f6834884640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683015e120 con 0x7f68300b5b20 2024-03-08T06:00:48.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.390+0000 7f6827fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68300b9840 0x7f68301578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.390+0000 7f6827fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68300b9840 0x7f68301578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35242/0 (socket says 172.21.15.76:35242) 2024-03-08T06:00:48.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.390+0000 7f6827fff640 1 -- 172.21.15.76:0/1764438719 learned_addr learned my addr 172.21.15.76:0/1764438719 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:48.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.390+0000 7f682ed9d640 1 --2- 172.21.15.76:0/1764438719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f683015dbc0 0x7f6830152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.390+0000 7f682e59c640 1 --2- 172.21.15.76:0/1764438719 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f68300b5b20 0x7f6830159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.390+0000 7f6827fff640 1 -- 172.21.15.76:0/1764438719 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f68300b5b20 msgr2=0x7f6830159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.390+0000 7f6827fff640 1 --2- 172.21.15.76:0/1764438719 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f68300b5b20 0x7f6830159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.390+0000 7f6827fff640 1 -- 172.21.15.76:0/1764438719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f683015dbc0 msgr2=0x7f6830152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.390+0000 7f6827fff640 1 --2- 172.21.15.76:0/1764438719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f683015dbc0 0x7f6830152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.390+0000 7f6827fff640 1 -- 172.21.15.76:0/1764438719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6830158360 con 0x7f68300b9840 2024-03-08T06:00:48.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.390+0000 7f6827fff640 1 --2- 172.21.15.76:0/1764438719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68300b9840 0x7f68301578c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f681800c7f0 tx=0x7f681800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:48.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.390+0000 7f6826ffd640 1 -- 172.21.15.76:0/1764438719 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818015070 con 0x7f68300b9840 2024-03-08T06:00:48.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.390+0000 7f6826ffd640 1 -- 172.21.15.76:0/1764438719 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6818007a30 con 0x7f68300b9840 2024-03-08T06:00:48.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.390+0000 7f6834884640 1 -- 172.21.15.76:0/1764438719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6830158640 con 0x7f68300b9840 2024-03-08T06:00:48.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.390+0000 7f6826ffd640 1 -- 172.21.15.76:0/1764438719 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818007e00 con 0x7f68300b9840 2024-03-08T06:00:48.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.390+0000 7f6834884640 1 -- 172.21.15.76:0/1764438719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6830152350 con 0x7f68300b9840 2024-03-08T06:00:48.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.391+0000 7f6834884640 1 -- 172.21.15.76:0/1764438719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6830158810 con 0x7f68300b9840 2024-03-08T06:00:48.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.392+0000 7f6826ffd640 1 -- 172.21.15.76:0/1764438719 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6818018070 con 0x7f68300b9840 2024-03-08T06:00:48.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.392+0000 7f6826ffd640 1 --2- 172.21.15.76:0/1764438719 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f681006b4d0 0x7f681006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.392+0000 7f6826ffd640 1 -- 172.21.15.76:0/1764438719 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f681808a4f0 con 0x7f68300b9840 2024-03-08T06:00:48.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.392+0000 7f682e59c640 1 --2- 172.21.15.76:0/1764438719 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f681006b4d0 0x7f681006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.392+0000 7f682e59c640 1 --2- 172.21.15.76:0/1764438719 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f681006b4d0 0x7f681006d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6820006fd0 tx=0x7f6820008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:48.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.396+0000 7f6826ffd640 1 -- 172.21.15.76:0/1764438719 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6830158810 con 0x7f68300b9840 2024-03-08T06:00:48.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.564+0000 7f6834884640 1 -- 172.21.15.76:0/1764438719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6830151e60 con 0x7f68300b9840 2024-03-08T06:00:48.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.565+0000 7f6826ffd640 1 -- 172.21.15.76:0/1764438719 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6818059030 con 0x7f68300b9840 2024-03-08T06:00:48.565 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:00:48.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.566+0000 7f6834884640 1 -- 172.21.15.76:0/1764438719 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f681006b4d0 msgr2=0x7f681006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.566+0000 7f6834884640 1 --2- 172.21.15.76:0/1764438719 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f681006b4d0 0x7f681006d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6820006fd0 tx=0x7f6820008040 comp rx=0 tx=0).stop 2024-03-08T06:00:48.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.566+0000 7f6834884640 1 -- 172.21.15.76:0/1764438719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68300b9840 msgr2=0x7f68301578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.566+0000 7f6834884640 1 --2- 172.21.15.76:0/1764438719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68300b9840 0x7f68301578c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f681800c7f0 tx=0x7f681800cbf0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.566+0000 7f6834884640 1 -- 172.21.15.76:0/1764438719 shutdown_connections 2024-03-08T06:00:48.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.566+0000 7f6834884640 1 --2- 172.21.15.76:0/1764438719 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f681006b4d0 0x7f681006d990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.566+0000 7f6834884640 1 --2- 172.21.15.76:0/1764438719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f683015dbc0 0x7f6830152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.566+0000 7f6834884640 1 --2- 172.21.15.76:0/1764438719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68300b9840 0x7f68301578c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.566+0000 7f6834884640 1 --2- 172.21.15.76:0/1764438719 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f68300b5b20 0x7f6830159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.566+0000 7f6834884640 1 -- 172.21.15.76:0/1764438719 >> 172.21.15.76:0/1764438719 conn(0x7f68300670d0 msgr2=0x7f68300c7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:48.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.567+0000 7f6834884640 1 -- 172.21.15.76:0/1764438719 shutdown_connections 2024-03-08T06:00:48.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.567+0000 7f6834884640 1 -- 172.21.15.76:0/1764438719 wait complete. 2024-03-08T06:00:48.578 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T06:00:48.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.681+0000 7f5a564d4640 1 Processor -- start 2024-03-08T06:00:48.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.681+0000 7f5a564d4640 1 -- start start 2024-03-08T06:00:48.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.681+0000 7f5a564d4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5a500ce5e0 0x7f5a5006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.681+0000 7f5a564d4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5a5015dfb0 0x7f5a5015e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.681+0000 7f5a564d4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a50161840 con 0x7f5a500ce9c0 2024-03-08T06:00:48.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.681+0000 7f5a564d4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a50161b90 con 0x7f5a500ce5e0 2024-03-08T06:00:48.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.681+0000 7f5a564d4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5015eb30 con 0x7f5a5015dfb0 2024-03-08T06:00:48.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.682+0000 7f5a4ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5a500ce5e0 0x7f5a5006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.682+0000 7f5a4ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5a500ce5e0 0x7f5a5006d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41160/0 (socket says 172.21.15.76:41160) 2024-03-08T06:00:48.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.682+0000 7f5a4ffff640 1 -- 172.21.15.76:0/4059618682 learned_addr learned my addr 172.21.15.76:0/4059618682 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:48.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.682+0000 7f5a54a71640 1 --2- 172.21.15.76:0/4059618682 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5a5015dfb0 0x7f5a5015e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.682+0000 7f5a4ffff640 1 -- 172.21.15.76:0/4059618682 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5a5015dfb0 msgr2=0x7f5a5015e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:48.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.682+0000 7f5a4ffff640 1 --2- 172.21.15.76:0/4059618682 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5a5015dfb0 0x7f5a5015e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.682+0000 7f5a4ffff640 1 -- 172.21.15.76:0/4059618682 >> v1:172.21.15.81:6789/0 conn(0x7f5a500ce9c0 legacy=0x7f5a50162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.682+0000 7f5a4ffff640 1 -- 172.21.15.76:0/4059618682 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5015ee10 con 0x7f5a500ce5e0 2024-03-08T06:00:48.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.682+0000 7f5a54a71640 1 --2- 172.21.15.76:0/4059618682 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5a5015dfb0 0x7f5a5015e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:00:48.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.683+0000 7f5a4ffff640 1 --2- 172.21.15.76:0/4059618682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a500ce5e0 0x7f5a5006d8a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009870 tx=0x7f5a4402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbf478d90e96cb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:48.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.683+0000 7f5a4effd640 1 -- 172.21.15.76:0/4059618682 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4402f840 con 0x7f5a500ce5e0 2024-03-08T06:00:48.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.683+0000 7f5a4effd640 1 -- 172.21.15.76:0/4059618682 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4402f9e0 con 0x7f5a500ce5e0 2024-03-08T06:00:48.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.683+0000 7f5a4effd640 1 -- 172.21.15.76:0/4059618682 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4402fdb0 con 0x7f5a500ce5e0 2024-03-08T06:00:48.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.683+0000 7f5a564d4640 1 -- 172.21.15.76:0/4059618682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a500ce5e0 msgr2=0x7f5a5006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.683+0000 7f5a564d4640 1 --2- 172.21.15.76:0/4059618682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a500ce5e0 0x7f5a5006d8a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009870 tx=0x7f5a4402ba20 comp rx=0 tx=0).stop 2024-03-08T06:00:48.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.683+0000 7f5a564d4640 1 -- 172.21.15.76:0/4059618682 shutdown_connections 2024-03-08T06:00:48.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.683+0000 7f5a564d4640 1 --2- 172.21.15.76:0/4059618682 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5a5015dfb0 0x7f5a5015e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.683+0000 7f5a564d4640 1 --2- 172.21.15.76:0/4059618682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a500ce5e0 0x7f5a5006d8a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.683+0000 7f5a564d4640 1 -- 172.21.15.76:0/4059618682 >> 172.21.15.76:0/4059618682 conn(0x7f5a5006ae60 msgr2=0x7f5a5006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:48.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.683+0000 7f5a564d4640 1 -- 172.21.15.76:0/4059618682 shutdown_connections 2024-03-08T06:00:48.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.683+0000 7f5a564d4640 1 -- 172.21.15.76:0/4059618682 wait complete. 2024-03-08T06:00:48.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.685+0000 7f5a564d4640 1 Processor -- start 2024-03-08T06:00:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.685+0000 7f5a564d4640 1 -- start start 2024-03-08T06:00:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.685+0000 7f5a564d4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a500ce5e0 0x7f5a501e7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.685+0000 7f5a564d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a500ce9c0 0x7f5a5015c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.685+0000 7f5a564d4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5a5015dfb0 0x7f5a50158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.685+0000 7f5a564d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a500cee20 con 0x7f5a500ce9c0 2024-03-08T06:00:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.685+0000 7f5a564d4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a50162730 con 0x7f5a500ce5e0 2024-03-08T06:00:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.685+0000 7f5a564d4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5006bfa0 con 0x7f5a5015dfb0 2024-03-08T06:00:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a4f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a500ce9c0 0x7f5a5015c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a4f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a500ce9c0 0x7f5a5015c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41172/0 (socket says 172.21.15.76:41172) 2024-03-08T06:00:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a4f7fe640 1 -- 172.21.15.76:0/2807432447 learned_addr learned my addr 172.21.15.76:0/2807432447 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a54a71640 1 --2- 172.21.15.76:0/2807432447 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5a5015dfb0 0x7f5a50158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a4ffff640 1 --2- 172.21.15.76:0/2807432447 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a500ce5e0 0x7f5a501e7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a4f7fe640 1 -- 172.21.15.76:0/2807432447 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5a5015dfb0 msgr2=0x7f5a50158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a4f7fe640 1 --2- 172.21.15.76:0/2807432447 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5a5015dfb0 0x7f5a50158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a4f7fe640 1 -- 172.21.15.76:0/2807432447 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a500ce5e0 msgr2=0x7f5a501e7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a4f7fe640 1 --2- 172.21.15.76:0/2807432447 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a500ce5e0 0x7f5a501e7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a4f7fe640 1 -- 172.21.15.76:0/2807432447 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a501e8920 con 0x7f5a500ce9c0 2024-03-08T06:00:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a54a71640 1 --2- 172.21.15.76:0/2807432447 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5a5015dfb0 0x7f5a50158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a4ffff640 1 --2- 172.21.15.76:0/2807432447 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a500ce5e0 0x7f5a501e7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a4f7fe640 1 --2- 172.21.15.76:0/2807432447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a500ce9c0 0x7f5a5015c970 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000c470 tx=0x7f5a40001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a4cff9640 1 -- 172.21.15.76:0/2807432447 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a400052d0 con 0x7f5a500ce9c0 2024-03-08T06:00:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a4cff9640 1 -- 172.21.15.76:0/2807432447 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a40005470 con 0x7f5a500ce9c0 2024-03-08T06:00:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a564d4640 1 -- 172.21.15.76:0/2807432447 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a501e8c00 con 0x7f5a500ce9c0 2024-03-08T06:00:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a4cff9640 1 -- 172.21.15.76:0/2807432447 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40007260 con 0x7f5a500ce9c0 2024-03-08T06:00:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.686+0000 7f5a564d4640 1 -- 172.21.15.76:0/2807432447 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a501e91c0 con 0x7f5a500ce9c0 2024-03-08T06:00:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.687+0000 7f5a564d4640 1 -- 172.21.15.76:0/2807432447 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a500b7710 con 0x7f5a500ce9c0 2024-03-08T06:00:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.688+0000 7f5a4cff9640 1 -- 172.21.15.76:0/2807432447 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a40007400 con 0x7f5a500ce9c0 2024-03-08T06:00:48.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.688+0000 7f5a4cff9640 1 --2- 172.21.15.76:0/2807432447 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5a3c06b4d0 0x7f5a3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.688+0000 7f5a4cff9640 1 -- 172.21.15.76:0/2807432447 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5a400899c0 con 0x7f5a500ce9c0 2024-03-08T06:00:48.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.688+0000 7f5a4ffff640 1 --2- 172.21.15.76:0/2807432447 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5a3c06b4d0 0x7f5a3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.688+0000 7f5a4ffff640 1 --2- 172.21.15.76:0/2807432447 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5a3c06b4d0 0x7f5a3c06d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5a44036000 tx=0x7f5a44039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:48.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.692+0000 7f5a4cff9640 1 -- 172.21.15.76:0/2807432447 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a4008be20 con 0x7f5a500ce9c0 2024-03-08T06:00:48.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.864+0000 7f5a564d4640 1 -- 172.21.15.76:0/2807432447 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5a5006e5d0 con 0x7f5a500ce9c0 2024-03-08T06:00:48.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.865+0000 7f5a4cff9640 1 -- 172.21.15.76:0/2807432447 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5a40058590 con 0x7f5a500ce9c0 2024-03-08T06:00:48.865 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:00:48.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.867+0000 7f5a564d4640 1 -- 172.21.15.76:0/2807432447 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5a3c06b4d0 msgr2=0x7f5a3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.867+0000 7f5a564d4640 1 --2- 172.21.15.76:0/2807432447 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5a3c06b4d0 0x7f5a3c06d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5a44036000 tx=0x7f5a44039040 comp rx=0 tx=0).stop 2024-03-08T06:00:48.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.867+0000 7f5a564d4640 1 -- 172.21.15.76:0/2807432447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a500ce9c0 msgr2=0x7f5a5015c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.867+0000 7f5a564d4640 1 --2- 172.21.15.76:0/2807432447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a500ce9c0 0x7f5a5015c970 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000c470 tx=0x7f5a40001e70 comp rx=0 tx=0).stop 2024-03-08T06:00:48.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.867+0000 7f5a564d4640 1 -- 172.21.15.76:0/2807432447 shutdown_connections 2024-03-08T06:00:48.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.867+0000 7f5a564d4640 1 --2- 172.21.15.76:0/2807432447 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5a3c06b4d0 0x7f5a3c06d990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.867+0000 7f5a564d4640 1 --2- 172.21.15.76:0/2807432447 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5a5015dfb0 0x7f5a50158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.867+0000 7f5a564d4640 1 --2- 172.21.15.76:0/2807432447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a500ce9c0 0x7f5a5015c970 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.867+0000 7f5a564d4640 1 --2- 172.21.15.76:0/2807432447 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a500ce5e0 0x7f5a501e7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.867+0000 7f5a564d4640 1 -- 172.21.15.76:0/2807432447 >> 172.21.15.76:0/2807432447 conn(0x7f5a5006ae60 msgr2=0x7f5a50160860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:48.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.867+0000 7f5a564d4640 1 -- 172.21.15.76:0/2807432447 shutdown_connections 2024-03-08T06:00:48.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.867+0000 7f5a564d4640 1 -- 172.21.15.76:0/2807432447 wait complete. 2024-03-08T06:00:48.878 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T06:00:48.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.980+0000 7fbc6f676640 1 Processor -- start 2024-03-08T06:00:48.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.980+0000 7fbc6f676640 1 -- start start 2024-03-08T06:00:48.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.981+0000 7fbc6f676640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbc6815dbc0 0x7fbc680d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.981+0000 7fbc6f676640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc680b5cf0 0x7fbc680b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.981+0000 7fbc6f676640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc680d27d0 con 0x7fbc680b9840 2024-03-08T06:00:48.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.981+0000 7fbc6f676640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc680d2ab0 con 0x7fbc6815dbc0 2024-03-08T06:00:48.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.981+0000 7fbc6f676640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc680d2d90 con 0x7fbc680b5cf0 2024-03-08T06:00:48.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.981+0000 7fbc6dc13640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc680b5cf0 0x7fbc680b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.981+0000 7fbc6dc13640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc680b5cf0 0x7fbc680b6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:43908/0 (socket says 172.21.15.76:43908) 2024-03-08T06:00:48.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.981+0000 7fbc6dc13640 1 -- 172.21.15.76:0/2715737250 learned_addr learned my addr 172.21.15.76:0/2715737250 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:48.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.981+0000 7fbc6d412640 1 --2- 172.21.15.76:0/2715737250 >> v2:172.21.15.81:3300/0 conn(0x7fbc6815dbc0 0x7fbc680d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.982+0000 7fbc6dc13640 1 -- 172.21.15.76:0/2715737250 >> v2:172.21.15.81:3300/0 conn(0x7fbc6815dbc0 msgr2=0x7fbc680d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:48.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.982+0000 7fbc6dc13640 1 --2- 172.21.15.76:0/2715737250 >> v2:172.21.15.81:3300/0 conn(0x7fbc6815dbc0 0x7fbc680d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.982+0000 7fbc6dc13640 1 -- 172.21.15.76:0/2715737250 >> v1:172.21.15.81:6789/0 conn(0x7fbc680b9840 legacy=0x7fbc680b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.982+0000 7fbc6dc13640 1 -- 172.21.15.76:0/2715737250 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc680d3070 con 0x7fbc680b5cf0 2024-03-08T06:00:48.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.983+0000 7fbc6dc13640 1 --2- 172.21.15.76:0/2715737250 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc680b5cf0 0x7fbc680b6c90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009e50 tx=0x7fbc6002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=781f4c3a3d0fbbdf server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:48.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.983+0000 7fbc5ffff640 1 -- 172.21.15.76:0/2715737250 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6002cb00 con 0x7fbc680b5cf0 2024-03-08T06:00:48.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.983+0000 7fbc5ffff640 1 -- 172.21.15.76:0/2715737250 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6002cca0 con 0x7fbc680b5cf0 2024-03-08T06:00:48.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.983+0000 7fbc5ffff640 1 -- 172.21.15.76:0/2715737250 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc60034610 con 0x7fbc680b5cf0 2024-03-08T06:00:48.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.983+0000 7fbc6f676640 1 -- 172.21.15.76:0/2715737250 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc680b5cf0 msgr2=0x7fbc680b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.983+0000 7fbc6f676640 1 --2- 172.21.15.76:0/2715737250 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc680b5cf0 0x7fbc680b6c90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009e50 tx=0x7fbc6002bf60 comp rx=0 tx=0).stop 2024-03-08T06:00:48.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.983+0000 7fbc6f676640 1 -- 172.21.15.76:0/2715737250 shutdown_connections 2024-03-08T06:00:48.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.983+0000 7fbc6f676640 1 --2- 172.21.15.76:0/2715737250 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc680b5cf0 0x7fbc680b6c90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.983+0000 7fbc6f676640 1 --2- 172.21.15.76:0/2715737250 >> v2:172.21.15.81:3300/0 conn(0x7fbc6815dbc0 0x7fbc680d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.983+0000 7fbc6f676640 1 -- 172.21.15.76:0/2715737250 >> 172.21.15.76:0/2715737250 conn(0x7fbc680670d0 msgr2=0x7fbc680d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:48.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.983+0000 7fbc6f676640 1 -- 172.21.15.76:0/2715737250 shutdown_connections 2024-03-08T06:00:48.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.983+0000 7fbc6f676640 1 -- 172.21.15.76:0/2715737250 wait complete. 2024-03-08T06:00:48.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.985+0000 7fbc6f676640 1 Processor -- start 2024-03-08T06:00:48.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.985+0000 7fbc6f676640 1 -- start start 2024-03-08T06:00:48.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.985+0000 7fbc6f676640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc680b5cf0 0x7fbc681f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.985+0000 7fbc6f676640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc680b9840 0x7fbc681e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.985+0000 7fbc6f676640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc6815dbc0 0x7fbc681e34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.985+0000 7fbc6f676640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc680d25b0 con 0x7fbc680b5cf0 2024-03-08T06:00:48.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.985+0000 7fbc6f676640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc680d2430 con 0x7fbc6815dbc0 2024-03-08T06:00:48.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.985+0000 7fbc6f676640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6815dfa0 con 0x7fbc680b9840 2024-03-08T06:00:48.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.985+0000 7fbc6d412640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc680b5cf0 0x7fbc681f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.985+0000 7fbc6d412640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc680b5cf0 0x7fbc681f30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41184/0 (socket says 172.21.15.76:41184) 2024-03-08T06:00:48.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.985+0000 7fbc6d412640 1 -- 172.21.15.76:0/1109148535 learned_addr learned my addr 172.21.15.76:0/1109148535 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:48.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.985+0000 7fbc6dc13640 1 --2- 172.21.15.76:0/1109148535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc6815dbc0 0x7fbc681e34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.985+0000 7fbc6cc11640 1 --2- 172.21.15.76:0/1109148535 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc680b9840 0x7fbc681e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.986+0000 7fbc6d412640 1 -- 172.21.15.76:0/1109148535 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc680b9840 msgr2=0x7fbc681e2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.986+0000 7fbc6d412640 1 --2- 172.21.15.76:0/1109148535 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc680b9840 0x7fbc681e2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.986+0000 7fbc6d412640 1 -- 172.21.15.76:0/1109148535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc6815dbc0 msgr2=0x7fbc681e34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:48.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.986+0000 7fbc6d412640 1 --2- 172.21.15.76:0/1109148535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc6815dbc0 0x7fbc681e34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:48.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.986+0000 7fbc6d412640 1 -- 172.21.15.76:0/1109148535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc680cc730 con 0x7fbc680b5cf0 2024-03-08T06:00:48.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.986+0000 7fbc6d412640 1 --2- 172.21.15.76:0/1109148535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc680b5cf0 0x7fbc681f30f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400e8a0 tx=0x7fbc6400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:48.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.986+0000 7fbc5dffb640 1 -- 172.21.15.76:0/1109148535 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64017070 con 0x7fbc680b5cf0 2024-03-08T06:00:48.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.986+0000 7fbc5dffb640 1 -- 172.21.15.76:0/1109148535 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6400c940 con 0x7fbc680b5cf0 2024-03-08T06:00:48.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.986+0000 7fbc6f676640 1 -- 172.21.15.76:0/1109148535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc680cca10 con 0x7fbc680b5cf0 2024-03-08T06:00:48.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.986+0000 7fbc5dffb640 1 -- 172.21.15.76:0/1109148535 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6400cd30 con 0x7fbc680b5cf0 2024-03-08T06:00:48.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.986+0000 7fbc6f676640 1 -- 172.21.15.76:0/1109148535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc680ccfd0 con 0x7fbc680b5cf0 2024-03-08T06:00:48.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.987+0000 7fbc6f676640 1 -- 172.21.15.76:0/1109148535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6804f030 con 0x7fbc680b5cf0 2024-03-08T06:00:48.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.988+0000 7fbc5dffb640 1 -- 172.21.15.76:0/1109148535 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc6401a070 con 0x7fbc680b5cf0 2024-03-08T06:00:48.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.988+0000 7fbc5dffb640 1 --2- 172.21.15.76:0/1109148535 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbc4c06b4d0 0x7fbc4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:48.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.988+0000 7fbc5dffb640 1 -- 172.21.15.76:0/1109148535 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbc6408a630 con 0x7fbc680b5cf0 2024-03-08T06:00:48.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.988+0000 7fbc6cc11640 1 --2- 172.21.15.76:0/1109148535 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbc4c06b4d0 0x7fbc4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:48.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.988+0000 7fbc6cc11640 1 --2- 172.21.15.76:0/1109148535 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbc4c06b4d0 0x7fbc4c06d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbc68156270 tx=0x7fbc58009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:48.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:48.992+0000 7fbc5dffb640 1 -- 172.21.15.76:0/1109148535 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc64058bb0 con 0x7fbc680b5cf0 2024-03-08T06:00:49.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:49.173+0000 7fbc6f676640 1 -- 172.21.15.76:0/1109148535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbc680d4820 con 0x7fbc680b5cf0 2024-03-08T06:00:49.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:49.173+0000 7fbc5dffb640 1 -- 172.21.15.76:0/1109148535 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbc6405cc50 con 0x7fbc680b5cf0 2024-03-08T06:00:49.173 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-03-08T06:00:49.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:49.175+0000 7fbc6f676640 1 -- 172.21.15.76:0/1109148535 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbc4c06b4d0 msgr2=0x7fbc4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:49.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:49.175+0000 7fbc6f676640 1 --2- 172.21.15.76:0/1109148535 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbc4c06b4d0 0x7fbc4c06d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbc68156270 tx=0x7fbc58009040 comp rx=0 tx=0).stop 2024-03-08T06:00:49.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:49.175+0000 7fbc6f676640 1 -- 172.21.15.76:0/1109148535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc680b5cf0 msgr2=0x7fbc681f30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:49.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:49.175+0000 7fbc6f676640 1 --2- 172.21.15.76:0/1109148535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc680b5cf0 0x7fbc681f30f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400e8a0 tx=0x7fbc6400ed60 comp rx=0 tx=0).stop 2024-03-08T06:00:49.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:49.175+0000 7fbc6f676640 1 -- 172.21.15.76:0/1109148535 shutdown_connections 2024-03-08T06:00:49.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:49.175+0000 7fbc6f676640 1 --2- 172.21.15.76:0/1109148535 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbc4c06b4d0 0x7fbc4c06d990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:49.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:49.175+0000 7fbc6f676640 1 --2- 172.21.15.76:0/1109148535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc6815dbc0 0x7fbc681e34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:49.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:49.175+0000 7fbc6f676640 1 --2- 172.21.15.76:0/1109148535 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc680b9840 0x7fbc681e2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:49.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:49.175+0000 7fbc6f676640 1 --2- 172.21.15.76:0/1109148535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc680b5cf0 0x7fbc681f30f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:49.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:49.175+0000 7fbc6f676640 1 -- 172.21.15.76:0/1109148535 >> 172.21.15.76:0/1109148535 conn(0x7fbc680670d0 msgr2=0x7fbc680cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:49.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:49.175+0000 7fbc6f676640 1 -- 172.21.15.76:0/1109148535 shutdown_connections 2024-03-08T06:00:49.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:49.175+0000 7fbc6f676640 1 -- 172.21.15.76:0/1109148535 wait complete. 2024-03-08T06:00:49.186 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2024-03-08T06:00:49.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:49.187 DEBUG:teuthology.orchestra.run.smithi076:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-08T06:00:49.212 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi076.stderr:ceph-fuse[44901]: fuse finished with error 0 and tester_r 0 2024-03-08T06:00:49.213 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:00:55.315 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-03-08T06:00:55.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:00:55.316 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-08T06:00:55.457 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:00:55.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.560+0000 7f1c6cd93640 1 Processor -- start 2024-03-08T06:00:55.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.560+0000 7f1c6cd93640 1 -- start start 2024-03-08T06:00:55.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.561+0000 7f1c6cd93640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1c680b6d00 0x7f1c680d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:55.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.561+0000 7f1c6cd93640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c680cf180 con 0x7f1c6815dde0 2024-03-08T06:00:55.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.561+0000 7f1c6cd93640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c680d1ed0 con 0x7f1c680cec10 2024-03-08T06:00:55.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.561+0000 7f1c6cd93640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c680d21b0 con 0x7f1c680b6d00 2024-03-08T06:00:55.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.561+0000 7f1c6659c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1c6815dde0 0x7f1c68162250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51080/0 (socket says 172.21.15.76:51080) 2024-03-08T06:00:55.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.561+0000 7f1c6659c640 1 -- 172.21.15.76:0/4109818114 learned_addr learned my addr 172.21.15.76:0/4109818114 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:55.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.561+0000 7f1c65d9b640 1 --2- 172.21.15.76:0/4109818114 >> v2:172.21.15.76:3300/0 conn(0x7f1c680b6d00 0x7f1c680d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:55.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.562+0000 7f1c6559a640 1 -- 172.21.15.76:0/4109818114 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028086977 0 0) 0x7f1c680cf180 con 0x7f1c6815dde0 2024-03-08T06:00:55.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.562+0000 7f1c6559a640 1 -- 172.21.15.76:0/4109818114 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c50003520 con 0x7f1c6815dde0 2024-03-08T06:00:55.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.562+0000 7f1c65d9b640 1 -- 172.21.15.76:0/4109818114 >> v1:172.21.15.81:6789/0 conn(0x7f1c680cec10 legacy=0x7f1c680d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:55.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.562+0000 7f1c65d9b640 1 -- 172.21.15.76:0/4109818114 >> v1:172.21.15.76:6789/0 conn(0x7f1c6815dde0 legacy=0x7f1c68162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:55.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.562+0000 7f1c65d9b640 1 -- 172.21.15.76:0/4109818114 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c680d09c0 con 0x7f1c680b6d00 2024-03-08T06:00:55.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.562+0000 7f1c65d9b640 1 --2- 172.21.15.76:0/4109818114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c680b6d00 0x7f1c680d0480 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0097a0 tx=0x7f1c5c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa4f998d16d4997 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:55.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.563+0000 7f1c6559a640 1 -- 172.21.15.76:0/4109818114 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0315a0 con 0x7f1c680b6d00 2024-03-08T06:00:55.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.563+0000 7f1c6559a640 1 -- 172.21.15.76:0/4109818114 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c037070 con 0x7f1c680b6d00 2024-03-08T06:00:55.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.563+0000 7f1c6559a640 1 -- 172.21.15.76:0/4109818114 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c031960 con 0x7f1c680b6d00 2024-03-08T06:00:55.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.563+0000 7f1c6cd93640 1 -- 172.21.15.76:0/4109818114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c680b6d00 msgr2=0x7f1c680d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:55.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.563+0000 7f1c6cd93640 1 --2- 172.21.15.76:0/4109818114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c680b6d00 0x7f1c680d0480 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0097a0 tx=0x7f1c5c02b840 comp rx=0 tx=0).stop 2024-03-08T06:00:55.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.563+0000 7f1c6cd93640 1 -- 172.21.15.76:0/4109818114 shutdown_connections 2024-03-08T06:00:55.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.563+0000 7f1c6cd93640 1 --2- 172.21.15.76:0/4109818114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c680b6d00 0x7f1c680d0480 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:55.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.563+0000 7f1c6cd93640 1 -- 172.21.15.76:0/4109818114 >> 172.21.15.76:0/4109818114 conn(0x7f1c680b56e0 msgr2=0x7f1c680b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:55.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.563+0000 7f1c6cd93640 1 -- 172.21.15.76:0/4109818114 shutdown_connections 2024-03-08T06:00:55.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.563+0000 7f1c6cd93640 1 -- 172.21.15.76:0/4109818114 wait complete. 2024-03-08T06:00:55.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.565+0000 7f1c6cd93640 1 Processor -- start 2024-03-08T06:00:55.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.565+0000 7f1c6cd93640 1 -- start start 2024-03-08T06:00:55.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c6cd93640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1c680b6d00 0x7f1c68155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:55.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c6cd93640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c680cec10 0x7f1c68153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:55.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c6cd93640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6815dde0 0x7f1c68154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:55.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c6cd93640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c680d3240 con 0x7f1c680cec10 2024-03-08T06:00:55.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c6cd93640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c680d30c0 con 0x7f1c6815dde0 2024-03-08T06:00:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c6cd93640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c680d33c0 con 0x7f1c680b6d00 2024-03-08T06:00:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c65d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c680cec10 0x7f1c68153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c65d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c680cec10 0x7f1c68153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41200/0 (socket says 172.21.15.76:41200) 2024-03-08T06:00:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c65d9b640 1 -- 172.21.15.76:0/219620244 learned_addr learned my addr 172.21.15.76:0/219620244 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c66d9d640 1 --2- 172.21.15.76:0/219620244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6815dde0 0x7f1c68154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c6659c640 1 --2- 172.21.15.76:0/219620244 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1c680b6d00 0x7f1c68155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c65d9b640 1 -- 172.21.15.76:0/219620244 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1c680b6d00 msgr2=0x7f1c68155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c65d9b640 1 --2- 172.21.15.76:0/219620244 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1c680b6d00 0x7f1c68155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c65d9b640 1 -- 172.21.15.76:0/219620244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6815dde0 msgr2=0x7f1c68154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c65d9b640 1 --2- 172.21.15.76:0/219620244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6815dde0 0x7f1c68154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c65d9b640 1 -- 172.21.15.76:0/219620244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c681563b0 con 0x7f1c680cec10 2024-03-08T06:00:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.566+0000 7f1c6659c640 1 --2- 172.21.15.76:0/219620244 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1c680b6d00 0x7f1c68155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.567+0000 7f1c65d9b640 1 --2- 172.21.15.76:0/219620244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c680cec10 0x7f1c68153ef0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c034040 tx=0x7f1c5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.567+0000 7f1c56ffd640 1 -- 172.21.15.76:0/219620244 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c032630 con 0x7f1c680cec10 2024-03-08T06:00:55.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.567+0000 7f1c56ffd640 1 -- 172.21.15.76:0/219620244 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c0327f0 con 0x7f1c680cec10 2024-03-08T06:00:55.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.567+0000 7f1c56ffd640 1 -- 172.21.15.76:0/219620244 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c032be0 con 0x7f1c680cec10 2024-03-08T06:00:55.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.567+0000 7f1c6cd93640 1 -- 172.21.15.76:0/219620244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c68156630 con 0x7f1c680cec10 2024-03-08T06:00:55.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.567+0000 7f1c6cd93640 1 -- 172.21.15.76:0/219620244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c68156bf0 con 0x7f1c680cec10 2024-03-08T06:00:55.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.568+0000 7f1c6cd93640 1 -- 172.21.15.76:0/219620244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c68072f60 con 0x7f1c680cec10 2024-03-08T06:00:55.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.568+0000 7f1c56ffd640 1 -- 172.21.15.76:0/219620244 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c5c03b3f0 con 0x7f1c680cec10 2024-03-08T06:00:55.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.569+0000 7f1c56ffd640 1 --2- 172.21.15.76:0/219620244 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1c4806b500 0x7f1c4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:55.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.569+0000 7f1c56ffd640 1 -- 172.21.15.76:0/219620244 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1c5c02f030 con 0x7f1c680cec10 2024-03-08T06:00:55.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.569+0000 7f1c6659c640 1 --2- 172.21.15.76:0/219620244 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1c4806b500 0x7f1c4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:55.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.569+0000 7f1c6659c640 1 --2- 172.21.15.76:0/219620244 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1c4806b500 0x7f1c4806d9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1c58005c00 tx=0x7f1c58009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:55.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.573+0000 7f1c56ffd640 1 -- 172.21.15.76:0/219620244 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c68072f60 con 0x7f1c680cec10 2024-03-08T06:00:55.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.744+0000 7f1c6cd93640 1 -- 172.21.15.76:0/219620244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c680d1870 con 0x7f1c680cec10 2024-03-08T06:00:55.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.746+0000 7f1c56ffd640 1 -- 172.21.15.76:0/219620244 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f1c681563b0 con 0x7f1c680cec10 2024-03-08T06:00:55.747 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:55.747 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":26,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:42.202011+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:00:55.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.748+0000 7f1c6cd93640 1 -- 172.21.15.76:0/219620244 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1c4806b500 msgr2=0x7f1c4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:55.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.748+0000 7f1c6cd93640 1 --2- 172.21.15.76:0/219620244 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1c4806b500 0x7f1c4806d9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1c58005c00 tx=0x7f1c58009040 comp rx=0 tx=0).stop 2024-03-08T06:00:55.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.748+0000 7f1c6cd93640 1 -- 172.21.15.76:0/219620244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c680cec10 msgr2=0x7f1c68153ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:55.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.748+0000 7f1c6cd93640 1 --2- 172.21.15.76:0/219620244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c680cec10 0x7f1c68153ef0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c034040 tx=0x7f1c5c0046c0 comp rx=0 tx=0).stop 2024-03-08T06:00:55.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.748+0000 7f1c6cd93640 1 -- 172.21.15.76:0/219620244 shutdown_connections 2024-03-08T06:00:55.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.748+0000 7f1c6cd93640 1 --2- 172.21.15.76:0/219620244 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1c4806b500 0x7f1c4806d9c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:55.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.748+0000 7f1c6cd93640 1 --2- 172.21.15.76:0/219620244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6815dde0 0x7f1c68154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:55.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.748+0000 7f1c6cd93640 1 --2- 172.21.15.76:0/219620244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c680cec10 0x7f1c68153ef0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:55.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.748+0000 7f1c6cd93640 1 --2- 172.21.15.76:0/219620244 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1c680b6d00 0x7f1c68155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:55.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.748+0000 7f1c6cd93640 1 -- 172.21.15.76:0/219620244 >> 172.21.15.76:0/219620244 conn(0x7f1c680b56e0 msgr2=0x7f1c680d3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:55.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.748+0000 7f1c6cd93640 1 -- 172.21.15.76:0/219620244 shutdown_connections 2024-03-08T06:00:55.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.748+0000 7f1c6cd93640 1 -- 172.21.15.76:0/219620244 wait complete. 2024-03-08T06:00:55.760 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:00:55.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.860+0000 7fac24bb5640 1 Processor -- start 2024-03-08T06:00:55.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.860+0000 7fac24bb5640 1 -- start start 2024-03-08T06:00:55.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.861+0000 7fac24bb5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fac2015eab0 0x7fac2006c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:55.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.861+0000 7fac24bb5640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fac2006c750 0x7fac200d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.861+0000 7fac24bb5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2015ef30 con 0x7fac2015dfc0 2024-03-08T06:00:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.861+0000 7fac24bb5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac200b6c10 con 0x7fac2015eab0 2024-03-08T06:00:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.861+0000 7fac24bb5640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2015f210 con 0x7fac2006c750 2024-03-08T06:00:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.861+0000 7fac17fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fac2015eab0 0x7fac2006c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.861+0000 7fac17fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fac2015eab0 0x7fac2006c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41210/0 (socket says 172.21.15.76:41210) 2024-03-08T06:00:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.861+0000 7fac17fff640 1 -- 172.21.15.76:0/3513040030 learned_addr learned my addr 172.21.15.76:0/3513040030 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.861+0000 7fac1ed9d640 1 --2- 172.21.15.76:0/3513040030 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fac2006c750 0x7fac200d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.861+0000 7fac17fff640 1 -- 172.21.15.76:0/3513040030 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fac2006c750 msgr2=0x7fac200d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.861+0000 7fac17fff640 1 --2- 172.21.15.76:0/3513040030 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fac2006c750 0x7fac200d2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.861+0000 7fac17fff640 1 -- 172.21.15.76:0/3513040030 >> v1:172.21.15.81:6789/0 conn(0x7fac2015dfc0 legacy=0x7fac2015e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:55.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.861+0000 7fac17fff640 1 -- 172.21.15.76:0/3513040030 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac200d1f00 con 0x7fac2015eab0 2024-03-08T06:00:55.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.862+0000 7fac17fff640 1 --2- 172.21.15.76:0/3513040030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac2015eab0 0x7fac2006c210 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fac08009850 tx=0x7fac0802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6402a05840d11044 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:55.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.862+0000 7fac1dd9b640 1 -- 172.21.15.76:0/3513040030 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08004230 con 0x7fac2015eab0 2024-03-08T06:00:55.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.862+0000 7fac1dd9b640 1 -- 172.21.15.76:0/3513040030 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac080043d0 con 0x7fac2015eab0 2024-03-08T06:00:55.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.862+0000 7fac1dd9b640 1 -- 172.21.15.76:0/3513040030 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08003810 con 0x7fac2015eab0 2024-03-08T06:00:55.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.863+0000 7fac24bb5640 1 -- 172.21.15.76:0/3513040030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac2015eab0 msgr2=0x7fac2006c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:55.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.863+0000 7fac24bb5640 1 --2- 172.21.15.76:0/3513040030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac2015eab0 0x7fac2006c210 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fac08009850 tx=0x7fac0802b9f0 comp rx=0 tx=0).stop 2024-03-08T06:00:55.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.863+0000 7fac24bb5640 1 -- 172.21.15.76:0/3513040030 shutdown_connections 2024-03-08T06:00:55.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.863+0000 7fac24bb5640 1 --2- 172.21.15.76:0/3513040030 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fac2006c750 0x7fac200d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:55.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.863+0000 7fac24bb5640 1 --2- 172.21.15.76:0/3513040030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac2015eab0 0x7fac2006c210 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:55.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.863+0000 7fac24bb5640 1 -- 172.21.15.76:0/3513040030 >> 172.21.15.76:0/3513040030 conn(0x7fac20066de0 msgr2=0x7fac200671f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:55.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.863+0000 7fac24bb5640 1 -- 172.21.15.76:0/3513040030 shutdown_connections 2024-03-08T06:00:55.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.863+0000 7fac24bb5640 1 -- 172.21.15.76:0/3513040030 wait complete. 2024-03-08T06:00:55.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.864+0000 7fac24bb5640 1 Processor -- start 2024-03-08T06:00:55.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.864+0000 7fac24bb5640 1 -- start start 2024-03-08T06:00:55.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac24bb5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac2006c750 0x7fac201e52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:55.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac24bb5640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fac2015dfc0 0x7fac200cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:55.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac24bb5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac2015eab0 0x7fac200cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:55.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac24bb5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac200d2c40 con 0x7fac2006c750 2024-03-08T06:00:55.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac24bb5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac200b9720 con 0x7fac2015eab0 2024-03-08T06:00:55.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac24bb5640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac200d2dc0 con 0x7fac2015dfc0 2024-03-08T06:00:55.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac1e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac2006c750 0x7fac201e52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:55.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac1e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac2006c750 0x7fac201e52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41226/0 (socket says 172.21.15.76:41226) 2024-03-08T06:00:55.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac1e59c640 1 -- 172.21.15.76:0/3714669776 learned_addr learned my addr 172.21.15.76:0/3714669776 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:55.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac1ed9d640 1 --2- 172.21.15.76:0/3714669776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac2015eab0 0x7fac200cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:55.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac17fff640 1 --2- 172.21.15.76:0/3714669776 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fac2015dfc0 0x7fac200cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:55.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac1e59c640 1 -- 172.21.15.76:0/3714669776 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fac2015dfc0 msgr2=0x7fac200cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:55.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac1e59c640 1 --2- 172.21.15.76:0/3714669776 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fac2015dfc0 0x7fac200cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:55.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac1e59c640 1 -- 172.21.15.76:0/3714669776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac2015eab0 msgr2=0x7fac200cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:55.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac1e59c640 1 --2- 172.21.15.76:0/3714669776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac2015eab0 0x7fac200cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:55.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac1e59c640 1 -- 172.21.15.76:0/3714669776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac201e4200 con 0x7fac2006c750 2024-03-08T06:00:55.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac1ed9d640 1 --2- 172.21.15.76:0/3714669776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac2015eab0 0x7fac200cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:00:55.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac1e59c640 1 --2- 172.21.15.76:0/3714669776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac2006c750 0x7fac201e52a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fac1000c400 tx=0x7fac1000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:55.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac16ffd640 1 -- 172.21.15.76:0/3714669776 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac100052d0 con 0x7fac2006c750 2024-03-08T06:00:55.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac16ffd640 1 -- 172.21.15.76:0/3714669776 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac10005490 con 0x7fac2006c750 2024-03-08T06:00:55.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac24bb5640 1 -- 172.21.15.76:0/3714669776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac201e44e0 con 0x7fac2006c750 2024-03-08T06:00:55.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac16ffd640 1 -- 172.21.15.76:0/3714669776 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10007260 con 0x7fac2006c750 2024-03-08T06:00:55.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.865+0000 7fac24bb5640 1 -- 172.21.15.76:0/3714669776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac201e4aa0 con 0x7fac2006c750 2024-03-08T06:00:55.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.866+0000 7fac24bb5640 1 -- 172.21.15.76:0/3714669776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac200d1d20 con 0x7fac2006c750 2024-03-08T06:00:55.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.867+0000 7fac16ffd640 1 -- 172.21.15.76:0/3714669776 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac10007400 con 0x7fac2006c750 2024-03-08T06:00:55.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.867+0000 7fac16ffd640 1 --2- 172.21.15.76:0/3714669776 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fac0006b4d0 0x7fac0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:55.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.867+0000 7fac16ffd640 1 -- 172.21.15.76:0/3714669776 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fac10089a00 con 0x7fac2006c750 2024-03-08T06:00:55.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.867+0000 7fac17fff640 1 --2- 172.21.15.76:0/3714669776 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fac0006b4d0 0x7fac0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:55.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.868+0000 7fac17fff640 1 --2- 172.21.15.76:0/3714669776 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fac0006b4d0 0x7fac0006d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fac08035000 tx=0x7fac08008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:55.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:55.872+0000 7fac16ffd640 1 -- 172.21.15.76:0/3714669776 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac200d1d20 con 0x7fac2006c750 2024-03-08T06:00:56.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.045+0000 7fac24bb5640 1 -- 172.21.15.76:0/3714669776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac200ca480 con 0x7fac2006c750 2024-03-08T06:00:56.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.046+0000 7fac16ffd640 1 -- 172.21.15.76:0/3714669776 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18430 (secure 0 0 0) 0x7fac100585d0 con 0x7fac2006c750 2024-03-08T06:00:56.047 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:56.047 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":26,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:42.202011+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:00:56.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.048+0000 7fac24bb5640 1 -- 172.21.15.76:0/3714669776 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fac0006b4d0 msgr2=0x7fac0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:56.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.048+0000 7fac24bb5640 1 --2- 172.21.15.76:0/3714669776 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fac0006b4d0 0x7fac0006d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fac08035000 tx=0x7fac08008030 comp rx=0 tx=0).stop 2024-03-08T06:00:56.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.048+0000 7fac24bb5640 1 -- 172.21.15.76:0/3714669776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac2006c750 msgr2=0x7fac201e52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:56.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.048+0000 7fac24bb5640 1 --2- 172.21.15.76:0/3714669776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac2006c750 0x7fac201e52a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fac1000c400 tx=0x7fac1000c8c0 comp rx=0 tx=0).stop 2024-03-08T06:00:56.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.048+0000 7fac24bb5640 1 -- 172.21.15.76:0/3714669776 shutdown_connections 2024-03-08T06:00:56.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.048+0000 7fac24bb5640 1 --2- 172.21.15.76:0/3714669776 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fac0006b4d0 0x7fac0006d990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:56.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.048+0000 7fac24bb5640 1 --2- 172.21.15.76:0/3714669776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac2015eab0 0x7fac200cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:56.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.048+0000 7fac24bb5640 1 --2- 172.21.15.76:0/3714669776 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fac2015dfc0 0x7fac200cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:56.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.048+0000 7fac24bb5640 1 --2- 172.21.15.76:0/3714669776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac2006c750 0x7fac201e52a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:56.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.048+0000 7fac24bb5640 1 -- 172.21.15.76:0/3714669776 >> 172.21.15.76:0/3714669776 conn(0x7fac20066de0 msgr2=0x7fac200cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:56.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.048+0000 7fac24bb5640 1 -- 172.21.15.76:0/3714669776 shutdown_connections 2024-03-08T06:00:56.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.048+0000 7fac24bb5640 1 -- 172.21.15.76:0/3714669776 wait complete. 2024-03-08T06:00:56.060 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-08T06:00:56.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.163+0000 7f0dcecb5640 1 Processor -- start 2024-03-08T06:00:56.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.163+0000 7f0dcecb5640 1 -- start start 2024-03-08T06:00:56.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.164+0000 7f0dcecb5640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dc80b2f20 0x7f0dc806b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:56.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.164+0000 7f0dcecb5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc806b5a0 con 0x7f0dc815bd70 2024-03-08T06:00:56.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.164+0000 7f0dcecb5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc80cc5b0 con 0x7f0dc806ac80 2024-03-08T06:00:56.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.164+0000 7f0dcecb5640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc80add10 con 0x7f0dc80b2f20 2024-03-08T06:00:56.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.164+0000 7f0dcd252640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0dc815bd70 0x7f0dc80ad600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51092/0 (socket says 172.21.15.76:51092) 2024-03-08T06:00:56.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.164+0000 7f0dcd252640 1 -- 172.21.15.76:0/435476366 learned_addr learned my addr 172.21.15.76:0/435476366 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:56.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.164+0000 7f0dc7fff640 1 --2- 172.21.15.76:0/435476366 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dc80b2f20 0x7f0dc806b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:56.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.164+0000 7f0dc77fe640 1 -- 172.21.15.76:0/435476366 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490987356 0 0) 0x7f0dc806b5a0 con 0x7f0dc815bd70 2024-03-08T06:00:56.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.164+0000 7f0dc77fe640 1 -- 172.21.15.76:0/435476366 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0db8003520 con 0x7f0dc815bd70 2024-03-08T06:00:56.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.164+0000 7f0dc7fff640 1 -- 172.21.15.76:0/435476366 >> v1:172.21.15.81:6789/0 conn(0x7f0dc806ac80 legacy=0x7f0dc80d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:56.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.164+0000 7f0dc77fe640 1 -- 172.21.15.76:0/435476366 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1964596933 0 0) 0x7f0dc80cc5b0 con 0x7f0dc806ac80 2024-03-08T06:00:56.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.164+0000 7f0dc7fff640 1 -- 172.21.15.76:0/435476366 >> v1:172.21.15.76:6789/0 conn(0x7f0dc815bd70 legacy=0x7f0dc80ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:56.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.164+0000 7f0dc7fff640 1 -- 172.21.15.76:0/435476366 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc80cc890 con 0x7f0dc80b2f20 2024-03-08T06:00:56.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.165+0000 7f0dc7fff640 1 --2- 172.21.15.76:0/435476366 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dc80b2f20 0x7f0dc806b060 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0db40097d0 tx=0x7f0db402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8565de55b0713a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:56.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.165+0000 7f0dc77fe640 1 -- 172.21.15.76:0/435476366 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4035020 con 0x7f0dc80b2f20 2024-03-08T06:00:56.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.165+0000 7f0dc77fe640 1 -- 172.21.15.76:0/435476366 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db4031890 con 0x7f0dc80b2f20 2024-03-08T06:00:56.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.165+0000 7f0dc77fe640 1 -- 172.21.15.76:0/435476366 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4031c40 con 0x7f0dc80b2f20 2024-03-08T06:00:56.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.166+0000 7f0dcecb5640 1 -- 172.21.15.76:0/435476366 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dc80b2f20 msgr2=0x7f0dc806b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:56.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.166+0000 7f0dcecb5640 1 --2- 172.21.15.76:0/435476366 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dc80b2f20 0x7f0dc806b060 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0db40097d0 tx=0x7f0db402b930 comp rx=0 tx=0).stop 2024-03-08T06:00:56.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.166+0000 7f0dcecb5640 1 -- 172.21.15.76:0/435476366 shutdown_connections 2024-03-08T06:00:56.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.166+0000 7f0dcecb5640 1 --2- 172.21.15.76:0/435476366 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dc80b2f20 0x7f0dc806b060 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:56.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.166+0000 7f0dcecb5640 1 -- 172.21.15.76:0/435476366 >> 172.21.15.76:0/435476366 conn(0x7f0dc80b2320 msgr2=0x7f0dc80b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:56.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.166+0000 7f0dcecb5640 1 -- 172.21.15.76:0/435476366 shutdown_connections 2024-03-08T06:00:56.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.166+0000 7f0dcecb5640 1 -- 172.21.15.76:0/435476366 wait complete. 2024-03-08T06:00:56.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.167+0000 7f0dcecb5640 1 Processor -- start 2024-03-08T06:00:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.167+0000 7f0dcecb5640 1 -- start start 2024-03-08T06:00:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.168+0000 7f0dcecb5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dc806ac80 0x7f0dc80c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.168+0000 7f0dcecb5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dc80b2f20 0x7f0dc8149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.168+0000 7f0dcecb5640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dc815bd70 0x7f0dc81514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.168+0000 7f0dcecb5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc80cd130 con 0x7f0dc806ac80 2024-03-08T06:00:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.168+0000 7f0dcecb5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc80cce50 con 0x7f0dc80b2f20 2024-03-08T06:00:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.168+0000 7f0dcecb5640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc80cc1d0 con 0x7f0dc815bd70 2024-03-08T06:00:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.168+0000 7f0dcca51640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dc806ac80 0x7f0dc80c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.168+0000 7f0dcca51640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dc806ac80 0x7f0dc80c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41234/0 (socket says 172.21.15.76:41234) 2024-03-08T06:00:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.168+0000 7f0dcca51640 1 -- 172.21.15.76:0/3377854878 learned_addr learned my addr 172.21.15.76:0/3377854878 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.168+0000 7f0dc7fff640 1 --2- 172.21.15.76:0/3377854878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dc80b2f20 0x7f0dc8149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.168+0000 7f0dcd252640 1 --2- 172.21.15.76:0/3377854878 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dc815bd70 0x7f0dc81514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:56.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.168+0000 7f0dcca51640 1 -- 172.21.15.76:0/3377854878 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dc815bd70 msgr2=0x7f0dc81514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:56.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.168+0000 7f0dcca51640 1 --2- 172.21.15.76:0/3377854878 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dc815bd70 0x7f0dc81514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:56.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.168+0000 7f0dcca51640 1 -- 172.21.15.76:0/3377854878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dc80b2f20 msgr2=0x7f0dc8149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:56.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.168+0000 7f0dcca51640 1 --2- 172.21.15.76:0/3377854878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dc80b2f20 0x7f0dc8149b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:56.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.168+0000 7f0dcca51640 1 -- 172.21.15.76:0/3377854878 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc80c5ee0 con 0x7f0dc806ac80 2024-03-08T06:00:56.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.169+0000 7f0dcca51640 1 --2- 172.21.15.76:0/3377854878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dc806ac80 0x7f0dc80c56b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00c490 tx=0x7f0dbc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:56.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.169+0000 7f0dc57fa640 1 -- 172.21.15.76:0/3377854878 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc005480 con 0x7f0dc806ac80 2024-03-08T06:00:56.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.169+0000 7f0dc57fa640 1 -- 172.21.15.76:0/3377854878 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dbc0035f0 con 0x7f0dc806ac80 2024-03-08T06:00:56.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.169+0000 7f0dcecb5640 1 -- 172.21.15.76:0/3377854878 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc80c61c0 con 0x7f0dc806ac80 2024-03-08T06:00:56.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.169+0000 7f0dc57fa640 1 -- 172.21.15.76:0/3377854878 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0039c0 con 0x7f0dc806ac80 2024-03-08T06:00:56.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.169+0000 7f0dcecb5640 1 -- 172.21.15.76:0/3377854878 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc80c6780 con 0x7f0dc806ac80 2024-03-08T06:00:56.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.170+0000 7f0dcecb5640 1 -- 172.21.15.76:0/3377854878 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc804f030 con 0x7f0dc806ac80 2024-03-08T06:00:56.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.170+0000 7f0dc57fa640 1 -- 172.21.15.76:0/3377854878 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0dbc003b60 con 0x7f0dc806ac80 2024-03-08T06:00:56.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.170+0000 7f0dc57fa640 1 --2- 172.21.15.76:0/3377854878 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0dac06b6a0 0x7f0dac06db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:56.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.171+0000 7f0dc57fa640 1 -- 172.21.15.76:0/3377854878 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0dbc08a230 con 0x7f0dc806ac80 2024-03-08T06:00:56.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.171+0000 7f0dc7fff640 1 --2- 172.21.15.76:0/3377854878 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0dac06b6a0 0x7f0dac06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:56.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.171+0000 7f0dc7fff640 1 --2- 172.21.15.76:0/3377854878 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0dac06b6a0 0x7f0dac06db60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0db4036000 tx=0x7f0db4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:56.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.175+0000 7f0dc57fa640 1 -- 172.21.15.76:0/3377854878 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0dbc0182b0 con 0x7f0dc806ac80 2024-03-08T06:00:56.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:56.384+0000 7f0dcecb5640 1 -- 172.21.15.76:0/3377854878 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0dc80b0070 con 0x7f0dc806ac80 2024-03-08T06:00:57.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.249+0000 7f0dc57fa640 1 -- 172.21.15.76:0/3377854878 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0dbc058ce0 con 0x7f0dc806ac80 2024-03-08T06:00:57.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.251+0000 7f0dcecb5640 1 -- 172.21.15.76:0/3377854878 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0dac06b6a0 msgr2=0x7f0dac06db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:57.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.251+0000 7f0dcecb5640 1 --2- 172.21.15.76:0/3377854878 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0dac06b6a0 0x7f0dac06db60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0db4036000 tx=0x7f0db4008030 comp rx=0 tx=0).stop 2024-03-08T06:00:57.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.251+0000 7f0dcecb5640 1 -- 172.21.15.76:0/3377854878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dc806ac80 msgr2=0x7f0dc80c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:57.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.251+0000 7f0dcecb5640 1 --2- 172.21.15.76:0/3377854878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dc806ac80 0x7f0dc80c56b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00c490 tx=0x7f0dbc00c950 comp rx=0 tx=0).stop 2024-03-08T06:00:57.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.251+0000 7f0dcecb5640 1 -- 172.21.15.76:0/3377854878 shutdown_connections 2024-03-08T06:00:57.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.251+0000 7f0dcecb5640 1 --2- 172.21.15.76:0/3377854878 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0dac06b6a0 0x7f0dac06db60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:57.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.251+0000 7f0dcecb5640 1 --2- 172.21.15.76:0/3377854878 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dc815bd70 0x7f0dc81514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:57.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.252+0000 7f0dcecb5640 1 --2- 172.21.15.76:0/3377854878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dc80b2f20 0x7f0dc8149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:57.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.252+0000 7f0dcecb5640 1 --2- 172.21.15.76:0/3377854878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dc806ac80 0x7f0dc80c56b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:57.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.252+0000 7f0dcecb5640 1 -- 172.21.15.76:0/3377854878 >> 172.21.15.76:0/3377854878 conn(0x7f0dc80b2320 msgr2=0x7f0dc80d0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:57.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.252+0000 7f0dcecb5640 1 -- 172.21.15.76:0/3377854878 shutdown_connections 2024-03-08T06:00:57.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.252+0000 7f0dcecb5640 1 -- 172.21.15.76:0/3377854878 wait complete. 2024-03-08T06:00:57.265 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-03-08T06:00:57.265 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-08T06:00:57.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.373+0000 7f8b55780640 1 Processor -- start 2024-03-08T06:00:57.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.373+0000 7f8b55780640 1 -- start start 2024-03-08T06:00:57.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.374+0000 7f8b55780640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8b500b9840 0x7f8b500b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:57.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.374+0000 7f8b55780640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8b500b5b20 0x7f8b500b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:57.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.374+0000 7f8b55780640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b500b5f40 con 0x7f8b5015dbc0 2024-03-08T06:00:57.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.374+0000 7f8b55780640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b500d27d0 con 0x7f8b500b5b20 2024-03-08T06:00:57.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.374+0000 7f8b55780640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b500d2ab0 con 0x7f8b500b9840 2024-03-08T06:00:57.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.374+0000 7f8b4f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8b500b5b20 0x7f8b500b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:57.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.374+0000 7f8b4f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8b500b5b20 0x7f8b500b6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41248/0 (socket says 172.21.15.76:41248) 2024-03-08T06:00:57.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.374+0000 7f8b4f7fe640 1 -- 172.21.15.76:0/3675505530 learned_addr learned my addr 172.21.15.76:0/3675505530 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:57.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.374+0000 7f8b4f7fe640 1 -- 172.21.15.76:0/3675505530 >> v2:172.21.15.81:3300/0 conn(0x7f8b500b9840 msgr2=0x7f8b500b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:57.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.374+0000 7f8b4f7fe640 1 --2- 172.21.15.76:0/3675505530 >> v2:172.21.15.81:3300/0 conn(0x7f8b500b9840 0x7f8b500b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:57.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.374+0000 7f8b4f7fe640 1 -- 172.21.15.76:0/3675505530 >> v1:172.21.15.81:6789/0 conn(0x7f8b5015dbc0 legacy=0x7f8b500d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:57.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.374+0000 7f8b4f7fe640 1 -- 172.21.15.76:0/3675505530 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b500d2d90 con 0x7f8b500b5b20 2024-03-08T06:00:57.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.375+0000 7f8b4f7fe640 1 --2- 172.21.15.76:0/3675505530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b500b5b20 0x7f8b500b6c90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8b500d19d0 tx=0x7f8b4802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48162652ac049be3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:57.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.375+0000 7f8b4dffb640 1 -- 172.21.15.76:0/3675505530 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4802ca80 con 0x7f8b500b5b20 2024-03-08T06:00:57.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.375+0000 7f8b4dffb640 1 -- 172.21.15.76:0/3675505530 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4802cc20 con 0x7f8b500b5b20 2024-03-08T06:00:57.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.375+0000 7f8b4dffb640 1 -- 172.21.15.76:0/3675505530 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b480343f0 con 0x7f8b500b5b20 2024-03-08T06:00:57.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.376+0000 7f8b55780640 1 -- 172.21.15.76:0/3675505530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b500b5b20 msgr2=0x7f8b500b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:57.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.376+0000 7f8b55780640 1 --2- 172.21.15.76:0/3675505530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b500b5b20 0x7f8b500b6c90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8b500d19d0 tx=0x7f8b4802bee0 comp rx=0 tx=0).stop 2024-03-08T06:00:57.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.376+0000 7f8b55780640 1 -- 172.21.15.76:0/3675505530 shutdown_connections 2024-03-08T06:00:57.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.376+0000 7f8b55780640 1 --2- 172.21.15.76:0/3675505530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b500b5b20 0x7f8b500b6c90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:57.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.376+0000 7f8b55780640 1 --2- 172.21.15.76:0/3675505530 >> v2:172.21.15.81:3300/0 conn(0x7f8b500b9840 0x7f8b500b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:57.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.376+0000 7f8b55780640 1 -- 172.21.15.76:0/3675505530 >> 172.21.15.76:0/3675505530 conn(0x7f8b500670d0 msgr2=0x7f8b500d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:57.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.376+0000 7f8b55780640 1 -- 172.21.15.76:0/3675505530 shutdown_connections 2024-03-08T06:00:57.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.376+0000 7f8b55780640 1 -- 172.21.15.76:0/3675505530 wait complete. 2024-03-08T06:00:57.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.377+0000 7f8b55780640 1 Processor -- start 2024-03-08T06:00:57.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.377+0000 7f8b55780640 1 -- start start 2024-03-08T06:00:57.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b55780640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b500b5b20 0x7f8b50146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:57.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b55780640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b500b9840 0x7f8b5015be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:57.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b55780640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8b5015dbc0 0x7f8b500c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:57.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b55780640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5015dfa0 con 0x7f8b500b9840 2024-03-08T06:00:57.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b55780640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b500d3870 con 0x7f8b500b5b20 2024-03-08T06:00:57.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b55780640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5015e120 con 0x7f8b5015dbc0 2024-03-08T06:00:57.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b4e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b500b9840 0x7f8b5015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:57.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b4e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b500b9840 0x7f8b5015be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41252/0 (socket says 172.21.15.76:41252) 2024-03-08T06:00:57.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b4e7fc640 1 -- 172.21.15.76:0/1961979344 learned_addr learned my addr 172.21.15.76:0/1961979344 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:57.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b4f7fe640 1 --2- 172.21.15.76:0/1961979344 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8b5015dbc0 0x7f8b500c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:57.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b4effd640 1 --2- 172.21.15.76:0/1961979344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b500b5b20 0x7f8b50146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:57.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b4e7fc640 1 -- 172.21.15.76:0/1961979344 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8b5015dbc0 msgr2=0x7f8b500c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:57.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b4e7fc640 1 --2- 172.21.15.76:0/1961979344 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8b5015dbc0 0x7f8b500c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:57.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b4e7fc640 1 -- 172.21.15.76:0/1961979344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b500b5b20 msgr2=0x7f8b50146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:57.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b4e7fc640 1 --2- 172.21.15.76:0/1961979344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b500b5b20 0x7f8b50146f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:57.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b4e7fc640 1 -- 172.21.15.76:0/1961979344 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b501f6be0 con 0x7f8b500b9840 2024-03-08T06:00:57.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b4effd640 1 --2- 172.21.15.76:0/1961979344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b500b5b20 0x7f8b50146f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:57.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b4f7fe640 1 --2- 172.21.15.76:0/1961979344 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8b5015dbc0 0x7f8b500c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:57.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b4e7fc640 1 --2- 172.21.15.76:0/1961979344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b500b9840 0x7f8b5015be50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400b4d0 tx=0x7f8b4400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:57.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b377fe640 1 -- 172.21.15.76:0/1961979344 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44014070 con 0x7f8b500b9840 2024-03-08T06:00:57.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b377fe640 1 -- 172.21.15.76:0/1961979344 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b44010870 con 0x7f8b500b9840 2024-03-08T06:00:57.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b55780640 1 -- 172.21.15.76:0/1961979344 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b501f6ec0 con 0x7f8b500b9840 2024-03-08T06:00:57.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.378+0000 7f8b377fe640 1 -- 172.21.15.76:0/1961979344 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44010c60 con 0x7f8b500b9840 2024-03-08T06:00:57.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.379+0000 7f8b55780640 1 -- 172.21.15.76:0/1961979344 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b501f7480 con 0x7f8b500b9840 2024-03-08T06:00:57.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.379+0000 7f8b55780640 1 -- 172.21.15.76:0/1961979344 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b5004f030 con 0x7f8b500b9840 2024-03-08T06:00:57.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.380+0000 7f8b377fe640 1 -- 172.21.15.76:0/1961979344 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b44017070 con 0x7f8b500b9840 2024-03-08T06:00:57.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.380+0000 7f8b377fe640 1 --2- 172.21.15.76:0/1961979344 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8b3c06b4d0 0x7f8b3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:57.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.380+0000 7f8b377fe640 1 -- 172.21.15.76:0/1961979344 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8b4408a350 con 0x7f8b500b9840 2024-03-08T06:00:57.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.380+0000 7f8b4effd640 1 --2- 172.21.15.76:0/1961979344 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8b3c06b4d0 0x7f8b3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:57.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.381+0000 7f8b4effd640 1 --2- 172.21.15.76:0/1961979344 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8b3c06b4d0 0x7f8b3c06d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000a810 tx=0x7f8b40008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:57.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.385+0000 7f8b377fe640 1 -- 172.21.15.76:0/1961979344 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b44058e90 con 0x7f8b500b9840 2024-03-08T06:00:57.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:57.592+0000 7f8b55780640 1 -- 172.21.15.76:0/1961979344 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8b5006b2b0 con 0x7f8b500b9840 2024-03-08T06:00:58.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.254+0000 7f8b377fe640 1 -- 172.21.15.76:0/1961979344 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8b44058790 con 0x7f8b500b9840 2024-03-08T06:00:58.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.256+0000 7f8b55780640 1 -- 172.21.15.76:0/1961979344 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8b3c06b4d0 msgr2=0x7f8b3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:58.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.256+0000 7f8b55780640 1 --2- 172.21.15.76:0/1961979344 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8b3c06b4d0 0x7f8b3c06d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000a810 tx=0x7f8b40008040 comp rx=0 tx=0).stop 2024-03-08T06:00:58.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.256+0000 7f8b55780640 1 -- 172.21.15.76:0/1961979344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b500b9840 msgr2=0x7f8b5015be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:58.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.256+0000 7f8b55780640 1 --2- 172.21.15.76:0/1961979344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b500b9840 0x7f8b5015be50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400b4d0 tx=0x7f8b4400b990 comp rx=0 tx=0).stop 2024-03-08T06:00:58.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.256+0000 7f8b55780640 1 -- 172.21.15.76:0/1961979344 shutdown_connections 2024-03-08T06:00:58.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.257+0000 7f8b55780640 1 --2- 172.21.15.76:0/1961979344 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8b3c06b4d0 0x7f8b3c06d990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.257+0000 7f8b55780640 1 --2- 172.21.15.76:0/1961979344 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8b5015dbc0 0x7f8b500c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.257+0000 7f8b55780640 1 --2- 172.21.15.76:0/1961979344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b500b9840 0x7f8b5015be50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.257+0000 7f8b55780640 1 --2- 172.21.15.76:0/1961979344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b500b5b20 0x7f8b50146f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.257+0000 7f8b55780640 1 -- 172.21.15.76:0/1961979344 >> 172.21.15.76:0/1961979344 conn(0x7f8b500670d0 msgr2=0x7f8b500c79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:58.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.257+0000 7f8b55780640 1 -- 172.21.15.76:0/1961979344 shutdown_connections 2024-03-08T06:00:58.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.257+0000 7f8b55780640 1 -- 172.21.15.76:0/1961979344 wait complete. 2024-03-08T06:00:58.270 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-08T06:00:58.270 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:00:58.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.375+0000 7fa1c85b5640 1 Processor -- start 2024-03-08T06:00:58.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.375+0000 7fa1c85b5640 1 -- start start 2024-03-08T06:00:58.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.376+0000 7fa1c85b5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1c006ac80 0x7fa1c00cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:58.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.376+0000 7fa1c85b5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa1c00af9c0 0x7fa1c006b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:58.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.376+0000 7fa1c85b5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c006b5a0 con 0x7fa1c006eee0 2024-03-08T06:00:58.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.376+0000 7fa1c85b5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c00d0150 con 0x7fa1c006ac80 2024-03-08T06:00:58.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.376+0000 7fa1c85b5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c00d0430 con 0x7fa1c00af9c0 2024-03-08T06:00:58.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.376+0000 7fa1c6351640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1c006ac80 0x7fa1c00cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:58.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.376+0000 7fa1c6351640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1c006ac80 0x7fa1c00cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41258/0 (socket says 172.21.15.76:41258) 2024-03-08T06:00:58.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.376+0000 7fa1c6351640 1 -- 172.21.15.76:0/3034990753 learned_addr learned my addr 172.21.15.76:0/3034990753 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:58.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.377+0000 7fa1c5b50640 1 --2- 172.21.15.76:0/3034990753 >> v2:172.21.15.81:3300/0 conn(0x7fa1c00af9c0 0x7fa1c006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:58.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.377+0000 7fa1c6351640 1 -- 172.21.15.76:0/3034990753 >> v2:172.21.15.81:3300/0 conn(0x7fa1c00af9c0 msgr2=0x7fa1c006b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:58.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.377+0000 7fa1c6351640 1 --2- 172.21.15.76:0/3034990753 >> v2:172.21.15.81:3300/0 conn(0x7fa1c00af9c0 0x7fa1c006b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.377+0000 7fa1c6351640 1 -- 172.21.15.76:0/3034990753 >> v1:172.21.15.81:6789/0 conn(0x7fa1c006eee0 legacy=0x7fa1c00caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:58.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.377+0000 7fa1c6351640 1 -- 172.21.15.76:0/3034990753 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c006e6e0 con 0x7fa1c006ac80 2024-03-08T06:00:58.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.377+0000 7fa1c6351640 1 --2- 172.21.15.76:0/3034990753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1c006ac80 0x7fa1c00cfa00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00ce310 tx=0x7fa1bc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2ca077f9047e60 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:58.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.378+0000 7fa1c534f640 1 -- 172.21.15.76:0/3034990753 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc034070 con 0x7fa1c006ac80 2024-03-08T06:00:58.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.378+0000 7fa1c534f640 1 -- 172.21.15.76:0/3034990753 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1bc030860 con 0x7fa1c006ac80 2024-03-08T06:00:58.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.378+0000 7fa1c534f640 1 -- 172.21.15.76:0/3034990753 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc030c30 con 0x7fa1c006ac80 2024-03-08T06:00:58.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.378+0000 7fa1c85b5640 1 -- 172.21.15.76:0/3034990753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1c006ac80 msgr2=0x7fa1c00cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:58.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.378+0000 7fa1c85b5640 1 --2- 172.21.15.76:0/3034990753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1c006ac80 0x7fa1c00cfa00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00ce310 tx=0x7fa1bc02b9a0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.378+0000 7fa1c85b5640 1 -- 172.21.15.76:0/3034990753 shutdown_connections 2024-03-08T06:00:58.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.378+0000 7fa1c85b5640 1 --2- 172.21.15.76:0/3034990753 >> v2:172.21.15.81:3300/0 conn(0x7fa1c00af9c0 0x7fa1c006b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.378+0000 7fa1c85b5640 1 --2- 172.21.15.76:0/3034990753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1c006ac80 0x7fa1c00cfa00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.378+0000 7fa1c85b5640 1 -- 172.21.15.76:0/3034990753 >> 172.21.15.76:0/3034990753 conn(0x7fa1c0099a60 msgr2=0x7fa1c00d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:58.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.378+0000 7fa1c85b5640 1 -- 172.21.15.76:0/3034990753 shutdown_connections 2024-03-08T06:00:58.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.378+0000 7fa1c85b5640 1 -- 172.21.15.76:0/3034990753 wait complete. 2024-03-08T06:00:58.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.379+0000 7fa1c85b5640 1 Processor -- start 2024-03-08T06:00:58.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.379+0000 7fa1c85b5640 1 -- start start 2024-03-08T06:00:58.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c85b5640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa1c006ac80 0x7fa1c01f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:58.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c85b5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1c006eee0 0x7fa1c015a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:58.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c85b5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1c00af9c0 0x7fa1c01f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:58.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c85b5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c015e280 con 0x7fa1c006eee0 2024-03-08T06:00:58.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c85b5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c00afe20 con 0x7fa1c00af9c0 2024-03-08T06:00:58.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c85b5640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c00cb5b0 con 0x7fa1c006ac80 2024-03-08T06:00:58.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c5b50640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1c006eee0 0x7fa1c015a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:58.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c5b50640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1c006eee0 0x7fa1c015a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41260/0 (socket says 172.21.15.76:41260) 2024-03-08T06:00:58.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c5b50640 1 -- 172.21.15.76:0/3773322520 learned_addr learned my addr 172.21.15.76:0/3773322520 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:58.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c6351640 1 --2- 172.21.15.76:0/3773322520 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa1c006ac80 0x7fa1c01f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:58.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c6b52640 1 --2- 172.21.15.76:0/3773322520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1c00af9c0 0x7fa1c01f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:58.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c5b50640 1 -- 172.21.15.76:0/3773322520 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa1c006ac80 msgr2=0x7fa1c01f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:58.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c5b50640 1 --2- 172.21.15.76:0/3773322520 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa1c006ac80 0x7fa1c01f4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c5b50640 1 -- 172.21.15.76:0/3773322520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1c00af9c0 msgr2=0x7fa1c01f78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:58.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c5b50640 1 --2- 172.21.15.76:0/3773322520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1c00af9c0 0x7fa1c01f78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c5b50640 1 -- 172.21.15.76:0/3773322520 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c01f8360 con 0x7fa1c006eee0 2024-03-08T06:00:58.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c6b52640 1 --2- 172.21.15.76:0/3773322520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1c00af9c0 0x7fa1c01f78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:58.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.380+0000 7fa1c6351640 1 --2- 172.21.15.76:0/3773322520 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa1c006ac80 0x7fa1c01f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:58.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.381+0000 7fa1c5b50640 1 --2- 172.21.15.76:0/3773322520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1c006eee0 0x7fa1c015a460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000d5f0 tx=0x7fa1b000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:58.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.381+0000 7fa1b6ffd640 1 -- 172.21.15.76:0/3773322520 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00125f0 con 0x7fa1c006eee0 2024-03-08T06:00:58.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.381+0000 7fa1b6ffd640 1 -- 172.21.15.76:0/3773322520 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b0019070 con 0x7fa1c006eee0 2024-03-08T06:00:58.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.381+0000 7fa1c85b5640 1 -- 172.21.15.76:0/3773322520 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c01f8640 con 0x7fa1c006eee0 2024-03-08T06:00:58.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.381+0000 7fa1b6ffd640 1 -- 172.21.15.76:0/3773322520 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00129d0 con 0x7fa1c006eee0 2024-03-08T06:00:58.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.381+0000 7fa1c85b5640 1 -- 172.21.15.76:0/3773322520 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c01f42d0 con 0x7fa1c006eee0 2024-03-08T06:00:58.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.382+0000 7fa1c85b5640 1 -- 172.21.15.76:0/3773322520 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c004f030 con 0x7fa1c006eee0 2024-03-08T06:00:58.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.382+0000 7fa1b6ffd640 1 -- 172.21.15.76:0/3773322520 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1b0012b70 con 0x7fa1c006eee0 2024-03-08T06:00:58.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.382+0000 7fa1b6ffd640 1 --2- 172.21.15.76:0/3773322520 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa1a406b4d0 0x7fa1a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:58.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.383+0000 7fa1b6ffd640 1 -- 172.21.15.76:0/3773322520 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa1b008a8c0 con 0x7fa1c006eee0 2024-03-08T06:00:58.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.383+0000 7fa1c6351640 1 --2- 172.21.15.76:0/3773322520 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa1a406b4d0 0x7fa1a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:58.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.383+0000 7fa1c6351640 1 --2- 172.21.15.76:0/3773322520 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa1a406b4d0 0x7fa1a406d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc036000 tx=0x7fa1bc03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:58.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.387+0000 7fa1b6ffd640 1 -- 172.21.15.76:0/3773322520 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1b0059490 con 0x7fa1c006eee0 2024-03-08T06:00:58.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.594+0000 7fa1c85b5640 1 -- 172.21.15.76:0/3773322520 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1c00af7a0 con 0x7fa1c006eee0 2024-03-08T06:00:58.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.594+0000 7fa1b6ffd640 1 -- 172.21.15.76:0/3773322520 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7fa1b0058d90 con 0x7fa1c006eee0 2024-03-08T06:00:58.595 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:58.595 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4249563566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4249563566},{"type":"v1","addr":"172.21.15.76:6835","nonce":4249563566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1514641617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1514641617},{"type":"v1","addr":"172.21.15.76:6837","nonce":1514641617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6834/1332442802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1332442802},{"type":"v1","addr":"172.21.15.81:6834","nonce":1332442802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:00:35.504231+0000","modified":"2024-03-08T06:00:57.594023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6835/2357927814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":2357927814},{"type":"v1","addr":"172.21.15.81:6835","nonce":2357927814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T06:00:58.595 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-03-08T06:00:58.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.596+0000 7fa1c85b5640 1 -- 172.21.15.76:0/3773322520 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa1a406b4d0 msgr2=0x7fa1a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:58.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.596+0000 7fa1c85b5640 1 --2- 172.21.15.76:0/3773322520 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa1a406b4d0 0x7fa1a406d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc036000 tx=0x7fa1bc03b040 comp rx=0 tx=0).stop 2024-03-08T06:00:58.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.596+0000 7fa1c85b5640 1 -- 172.21.15.76:0/3773322520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1c006eee0 msgr2=0x7fa1c015a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:58.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.596+0000 7fa1c85b5640 1 --2- 172.21.15.76:0/3773322520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1c006eee0 0x7fa1c015a460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000d5f0 tx=0x7fa1b000dab0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.596+0000 7fa1c85b5640 1 -- 172.21.15.76:0/3773322520 shutdown_connections 2024-03-08T06:00:58.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.596+0000 7fa1c85b5640 1 --2- 172.21.15.76:0/3773322520 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa1a406b4d0 0x7fa1a406d990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.596+0000 7fa1c85b5640 1 --2- 172.21.15.76:0/3773322520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1c00af9c0 0x7fa1c01f78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.596+0000 7fa1c85b5640 1 --2- 172.21.15.76:0/3773322520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1c006eee0 0x7fa1c015a460 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.596+0000 7fa1c85b5640 1 --2- 172.21.15.76:0/3773322520 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa1c006ac80 0x7fa1c01f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.596+0000 7fa1c85b5640 1 -- 172.21.15.76:0/3773322520 >> 172.21.15.76:0/3773322520 conn(0x7fa1c0099a60 msgr2=0x7fa1c0158090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:58.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.596+0000 7fa1c85b5640 1 -- 172.21.15.76:0/3773322520 shutdown_connections 2024-03-08T06:00:58.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.596+0000 7fa1c85b5640 1 -- 172.21.15.76:0/3773322520 wait complete. 2024-03-08T06:00:58.608 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:00:58.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.708+0000 7f76eadc7640 1 Processor -- start 2024-03-08T06:00:58.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.708+0000 7f76eadc7640 1 -- start start 2024-03-08T06:00:58.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.709+0000 7f76eadc7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f76e4071390 0x7f76e40cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:58.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.709+0000 7f76eadc7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76e406d3f0 0x7f76e4071770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:58.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.709+0000 7f76eadc7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e4071cb0 con 0x7f76e40b3360 2024-03-08T06:00:58.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.709+0000 7f76eadc7640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e40ce320 con 0x7f76e4071390 2024-03-08T06:00:58.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.709+0000 7f76eadc7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e406d880 con 0x7f76e406d3f0 2024-03-08T06:00:58.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.709+0000 7f76e3fff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76e406d3f0 0x7f76e4071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:58.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.709+0000 7f76e8b63640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f76e4071390 0x7f76e40cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:58.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.710+0000 7f76e9364640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f76e40b3360 0x7f76e40b3810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:40022/0 (socket says 172.21.15.76:40022) 2024-03-08T06:00:58.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.710+0000 7f76e3fff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76e406d3f0 0x7f76e4071770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:36992/0 (socket says 172.21.15.76:36992) 2024-03-08T06:00:58.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.710+0000 7f76e9364640 1 -- 172.21.15.76:0/3765668834 learned_addr learned my addr 172.21.15.76:0/3765668834 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:58.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.710+0000 7f76e8b63640 1 -- 172.21.15.76:0/3765668834 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76e406d3f0 msgr2=0x7f76e4071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:58.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.710+0000 7f76e8b63640 1 --2- 172.21.15.76:0/3765668834 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76e406d3f0 0x7f76e4071770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.710+0000 7f76e8b63640 1 -- 172.21.15.76:0/3765668834 >> v1:172.21.15.81:6789/0 conn(0x7f76e40b3360 legacy=0x7f76e40b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:58.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.710+0000 7f76e8b63640 1 -- 172.21.15.76:0/3765668834 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e406e010 con 0x7f76e4071390 2024-03-08T06:00:58.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.710+0000 7f76e37fe640 1 -- 172.21.15.76:0/3765668834 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661667177 0 0) 0x7f76e4071cb0 con 0x7f76e40b3360 2024-03-08T06:00:58.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.711+0000 7f76e8b63640 1 --2- 172.21.15.76:0/3765668834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76e4071390 0x7f76e40cf0e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f76d40097a0 tx=0x7f76d402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c918db7fd985101e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:58.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.711+0000 7f76e37fe640 1 -- 172.21.15.76:0/3765668834 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4035020 con 0x7f76e4071390 2024-03-08T06:00:58.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.711+0000 7f76e37fe640 1 -- 172.21.15.76:0/3765668834 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d4031830 con 0x7f76e4071390 2024-03-08T06:00:58.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.711+0000 7f76e37fe640 1 -- 172.21.15.76:0/3765668834 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4031c00 con 0x7f76e4071390 2024-03-08T06:00:58.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.711+0000 7f76eadc7640 1 -- 172.21.15.76:0/3765668834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76e4071390 msgr2=0x7f76e40cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:58.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.711+0000 7f76eadc7640 1 --2- 172.21.15.76:0/3765668834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76e4071390 0x7f76e40cf0e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f76d40097a0 tx=0x7f76d402b8d0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.711+0000 7f76eadc7640 1 -- 172.21.15.76:0/3765668834 shutdown_connections 2024-03-08T06:00:58.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.711+0000 7f76eadc7640 1 --2- 172.21.15.76:0/3765668834 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76e406d3f0 0x7f76e4071770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.711+0000 7f76eadc7640 1 --2- 172.21.15.76:0/3765668834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76e4071390 0x7f76e40cf0e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.711+0000 7f76eadc7640 1 -- 172.21.15.76:0/3765668834 >> 172.21.15.76:0/3765668834 conn(0x7f76e40cf740 msgr2=0x7f76e40cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:58.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.711+0000 7f76eadc7640 1 -- 172.21.15.76:0/3765668834 shutdown_connections 2024-03-08T06:00:58.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.711+0000 7f76eadc7640 1 -- 172.21.15.76:0/3765668834 wait complete. 2024-03-08T06:00:58.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.712+0000 7f76eadc7640 1 Processor -- start 2024-03-08T06:00:58.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.713+0000 7f76eadc7640 1 -- start start 2024-03-08T06:00:58.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.713+0000 7f76eadc7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76e4071390 0x7f76e41561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:58.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.713+0000 7f76eadc7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76e40b3360 0x7f76e4157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:58.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.713+0000 7f76eadc7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e4158200 0x7f76e4140ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:58.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.713+0000 7f76eadc7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e40ce7b0 con 0x7f76e4158200 2024-03-08T06:00:58.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.713+0000 7f76eadc7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e40ced70 con 0x7f76e40b3360 2024-03-08T06:00:58.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.713+0000 7f76eadc7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e4158780 con 0x7f76e4071390 2024-03-08T06:00:58.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.713+0000 7f76e9364640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e4158200 0x7f76e4140ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:58.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.714+0000 7f76e9364640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e4158200 0x7f76e4140ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51200/0 (socket says 172.21.15.76:51200) 2024-03-08T06:00:58.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.714+0000 7f76e9364640 1 -- 172.21.15.76:0/3327574271 learned_addr learned my addr 172.21.15.76:0/3327574271 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:58.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.714+0000 7f76e3fff640 1 --2- 172.21.15.76:0/3327574271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76e40b3360 0x7f76e4157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:58.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.714+0000 7f76e8b63640 1 --2- 172.21.15.76:0/3327574271 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76e4071390 0x7f76e41561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:58.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.714+0000 7f76e9364640 1 -- 172.21.15.76:0/3327574271 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76e4071390 msgr2=0x7f76e41561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:58.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.714+0000 7f76e9364640 1 --2- 172.21.15.76:0/3327574271 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76e4071390 0x7f76e41561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.714+0000 7f76e9364640 1 -- 172.21.15.76:0/3327574271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76e40b3360 msgr2=0x7f76e4157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:58.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.714+0000 7f76e9364640 1 --2- 172.21.15.76:0/3327574271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76e40b3360 0x7f76e4157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.714+0000 7f76e9364640 1 -- 172.21.15.76:0/3327574271 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e4158a60 con 0x7f76e4158200 2024-03-08T06:00:58.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.714+0000 7f76e3fff640 1 --2- 172.21.15.76:0/3327574271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76e40b3360 0x7f76e4157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:00:58.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.714+0000 7f76e9364640 1 --2- 172.21.15.76:0/3327574271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e4158200 0x7f76e4140ce0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00c780 tx=0x7f76dc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:58.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.714+0000 7f76e17fa640 1 -- 172.21.15.76:0/3327574271 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc018070 con 0x7f76e4158200 2024-03-08T06:00:58.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.714+0000 7f76e17fa640 1 -- 172.21.15.76:0/3327574271 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76dc00a640 con 0x7f76e4158200 2024-03-08T06:00:58.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.714+0000 7f76eadc7640 1 -- 172.21.15.76:0/3327574271 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e4153910 con 0x7f76e4158200 2024-03-08T06:00:58.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.714+0000 7f76e17fa640 1 -- 172.21.15.76:0/3327574271 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc00aa30 con 0x7f76e4158200 2024-03-08T06:00:58.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.714+0000 7f76eadc7640 1 -- 172.21.15.76:0/3327574271 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e41531a0 con 0x7f76e4158200 2024-03-08T06:00:58.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.715+0000 7f76eadc7640 1 -- 172.21.15.76:0/3327574271 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e404f030 con 0x7f76e4158200 2024-03-08T06:00:58.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.716+0000 7f76e17fa640 1 -- 172.21.15.76:0/3327574271 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76dc01b070 con 0x7f76e4158200 2024-03-08T06:00:58.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.716+0000 7f76e17fa640 1 --2- 172.21.15.76:0/3327574271 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f76d006b4d0 0x7f76d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:58.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.716+0000 7f76e17fa640 1 -- 172.21.15.76:0/3327574271 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f76dc089b40 con 0x7f76e4158200 2024-03-08T06:00:58.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.716+0000 7f76e8b63640 1 --2- 172.21.15.76:0/3327574271 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f76d006b4d0 0x7f76d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:58.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.716+0000 7f76e8b63640 1 --2- 172.21.15.76:0/3327574271 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f76d006b4d0 0x7f76d006d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f76d4036000 tx=0x7f76d4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:58.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.720+0000 7f76e17fa640 1 -- 172.21.15.76:0/3327574271 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76dc058680 con 0x7f76e4158200 2024-03-08T06:00:58.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.892+0000 7f76eadc7640 1 -- 172.21.15.76:0/3327574271 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76e40b3fe0 con 0x7f76e4158200 2024-03-08T06:00:58.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.893+0000 7f76e17fa640 1 -- 172.21.15.76:0/3327574271 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f76dc057f80 con 0x7f76e4158200 2024-03-08T06:00:58.894 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:58.894 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":26,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:42.202011+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:00:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.895+0000 7f76eadc7640 1 -- 172.21.15.76:0/3327574271 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f76d006b4d0 msgr2=0x7f76d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.895+0000 7f76eadc7640 1 --2- 172.21.15.76:0/3327574271 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f76d006b4d0 0x7f76d006d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f76d4036000 tx=0x7f76d4008030 comp rx=0 tx=0).stop 2024-03-08T06:00:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.895+0000 7f76eadc7640 1 -- 172.21.15.76:0/3327574271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e4158200 msgr2=0x7f76e4140ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.895+0000 7f76eadc7640 1 --2- 172.21.15.76:0/3327574271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e4158200 0x7f76e4140ce0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00c780 tx=0x7f76dc00cc40 comp rx=0 tx=0).stop 2024-03-08T06:00:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.895+0000 7f76eadc7640 1 -- 172.21.15.76:0/3327574271 shutdown_connections 2024-03-08T06:00:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.895+0000 7f76eadc7640 1 --2- 172.21.15.76:0/3327574271 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f76d006b4d0 0x7f76d006d990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.895+0000 7f76eadc7640 1 --2- 172.21.15.76:0/3327574271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e4158200 0x7f76e4140ce0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.895+0000 7f76eadc7640 1 --2- 172.21.15.76:0/3327574271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76e40b3360 0x7f76e4157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.895+0000 7f76eadc7640 1 --2- 172.21.15.76:0/3327574271 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76e4071390 0x7f76e41561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.895+0000 7f76eadc7640 1 -- 172.21.15.76:0/3327574271 >> 172.21.15.76:0/3327574271 conn(0x7f76e40cf740 msgr2=0x7f76e40cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.895+0000 7f76eadc7640 1 -- 172.21.15.76:0/3327574271 shutdown_connections 2024-03-08T06:00:58.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:58.895+0000 7f76eadc7640 1 -- 172.21.15.76:0/3327574271 wait complete. 2024-03-08T06:00:58.907 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-08T06:00:58.907 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:00:59.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.006+0000 7f197dfe2640 1 Processor -- start 2024-03-08T06:00:59.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.007+0000 7f197dfe2640 1 -- start start 2024-03-08T06:00:59.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.007+0000 7f197dfe2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f197815bb40 0x7f19780b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.007+0000 7f197dfe2640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f197815bf20 0x7f19780b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.007+0000 7f197dfe2640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f19780cf5b0 0x7f19780ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.007+0000 7f197dfe2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19780cfb50 con 0x7f197815bb40 2024-03-08T06:00:59.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.007+0000 7f197dfe2640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19780cfe30 con 0x7f19780cf5b0 2024-03-08T06:00:59.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.007+0000 7f197dfe2640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19780d0110 con 0x7f197815bf20 2024-03-08T06:00:59.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.007+0000 7f19777fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f197815bb40 0x7f19780b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.008+0000 7f19777fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f197815bb40 0x7f19780b0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51206/0 (socket says 172.21.15.76:51206) 2024-03-08T06:00:59.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.008+0000 7f19777fe640 1 -- 172.21.15.76:0/3396429086 learned_addr learned my addr 172.21.15.76:0/3396429086 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:59.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.008+0000 7f1976ffd640 1 --2- 172.21.15.76:0/3396429086 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f197815bf20 0x7f19780b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.008+0000 7f1977fff640 1 --2- 172.21.15.76:0/3396429086 >> v2:172.21.15.81:3300/0 conn(0x7f19780cf5b0 0x7f19780ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.008+0000 7f19777fe640 1 -- 172.21.15.76:0/3396429086 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f197815bf20 msgr2=0x7f19780b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:59.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.008+0000 7f19777fe640 1 --2- 172.21.15.76:0/3396429086 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f197815bf20 0x7f19780b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.008+0000 7f19777fe640 1 -- 172.21.15.76:0/3396429086 >> v2:172.21.15.81:3300/0 conn(0x7f19780cf5b0 msgr2=0x7f19780ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:59.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.008+0000 7f19777fe640 1 --2- 172.21.15.76:0/3396429086 >> v2:172.21.15.81:3300/0 conn(0x7f19780cf5b0 0x7f19780ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.008+0000 7f19777fe640 1 -- 172.21.15.76:0/3396429086 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19780d03f0 con 0x7f197815bb40 2024-03-08T06:00:59.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.009+0000 7f19777fe640 1 --2- 172.21.15.76:0/3396429086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197815bb40 0x7f19780b0c90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1978070b10 tx=0x7f196c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43c653a0145bd6d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:59.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.009+0000 7f19767fc640 1 -- 172.21.15.76:0/3396429086 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c035020 con 0x7f197815bb40 2024-03-08T06:00:59.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.009+0000 7f19767fc640 1 -- 172.21.15.76:0/3396429086 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c031870 con 0x7f197815bb40 2024-03-08T06:00:59.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.009+0000 7f19767fc640 1 -- 172.21.15.76:0/3396429086 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c031c40 con 0x7f197815bb40 2024-03-08T06:00:59.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.009+0000 7f197dfe2640 1 -- 172.21.15.76:0/3396429086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197815bb40 msgr2=0x7f19780b0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.009+0000 7f197dfe2640 1 --2- 172.21.15.76:0/3396429086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197815bb40 0x7f19780b0c90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1978070b10 tx=0x7f196c02b910 comp rx=0 tx=0).stop 2024-03-08T06:00:59.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.009+0000 7f197dfe2640 1 -- 172.21.15.76:0/3396429086 shutdown_connections 2024-03-08T06:00:59.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.009+0000 7f197dfe2640 1 --2- 172.21.15.76:0/3396429086 >> v2:172.21.15.81:3300/0 conn(0x7f19780cf5b0 0x7f19780ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.009+0000 7f197dfe2640 1 --2- 172.21.15.76:0/3396429086 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f197815bf20 0x7f19780b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.009+0000 7f197dfe2640 1 --2- 172.21.15.76:0/3396429086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197815bb40 0x7f19780b0c90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.009+0000 7f197dfe2640 1 -- 172.21.15.76:0/3396429086 >> 172.21.15.76:0/3396429086 conn(0x7f197806f5e0 msgr2=0x7f197806f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:59.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.009+0000 7f197dfe2640 1 -- 172.21.15.76:0/3396429086 shutdown_connections 2024-03-08T06:00:59.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.009+0000 7f197dfe2640 1 -- 172.21.15.76:0/3396429086 wait complete. 2024-03-08T06:00:59.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.010+0000 7f197dfe2640 1 Processor -- start 2024-03-08T06:00:59.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.011+0000 7f197dfe2640 1 -- start start 2024-03-08T06:00:59.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.011+0000 7f197dfe2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19780cf5b0 0x7f197815e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.011+0000 7f197dfe2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197815bb40 0x7f19780b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.011+0000 7f197dfe2640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f197815bf20 0x7f19780b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.011+0000 7f197dfe2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19780b4e30 con 0x7f197815bb40 2024-03-08T06:00:59.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.011+0000 7f197dfe2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19780b11d0 con 0x7f19780cf5b0 2024-03-08T06:00:59.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.011+0000 7f197dfe2640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19780b4fb0 con 0x7f197815bf20 2024-03-08T06:00:59.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.011+0000 7f1976ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197815bb40 0x7f19780b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.011+0000 7f1976ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197815bb40 0x7f19780b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51218/0 (socket says 172.21.15.76:51218) 2024-03-08T06:00:59.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.011+0000 7f1976ffd640 1 -- 172.21.15.76:0/3281255534 learned_addr learned my addr 172.21.15.76:0/3281255534 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:59.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.012+0000 7f1977fff640 1 --2- 172.21.15.76:0/3281255534 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f197815bf20 0x7f19780b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.012+0000 7f19777fe640 1 --2- 172.21.15.76:0/3281255534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19780cf5b0 0x7f197815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.012+0000 7f1976ffd640 1 -- 172.21.15.76:0/3281255534 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f197815bf20 msgr2=0x7f19780b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.012+0000 7f1976ffd640 1 --2- 172.21.15.76:0/3281255534 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f197815bf20 0x7f19780b3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.012+0000 7f1976ffd640 1 -- 172.21.15.76:0/3281255534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19780cf5b0 msgr2=0x7f197815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.012+0000 7f1976ffd640 1 --2- 172.21.15.76:0/3281255534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19780cf5b0 0x7f197815e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.012+0000 7f1976ffd640 1 -- 172.21.15.76:0/3281255534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19780b3cf0 con 0x7f197815bb40 2024-03-08T06:00:59.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.012+0000 7f1976ffd640 1 --2- 172.21.15.76:0/3281255534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197815bb40 0x7f19780b30e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f196800e870 tx=0x7f196800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:59.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.012+0000 7f195ffff640 1 -- 172.21.15.76:0/3281255534 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968017070 con 0x7f197815bb40 2024-03-08T06:00:59.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.012+0000 7f195ffff640 1 -- 172.21.15.76:0/3281255534 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196800c940 con 0x7f197815bb40 2024-03-08T06:00:59.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.012+0000 7f195ffff640 1 -- 172.21.15.76:0/3281255534 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196800cd30 con 0x7f197815bb40 2024-03-08T06:00:59.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.012+0000 7f197dfe2640 1 -- 172.21.15.76:0/3281255534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19780b3fd0 con 0x7f197815bb40 2024-03-08T06:00:59.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.012+0000 7f197dfe2640 1 -- 172.21.15.76:0/3281255534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197806c1f0 con 0x7f197815bb40 2024-03-08T06:00:59.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.013+0000 7f197dfe2640 1 -- 172.21.15.76:0/3281255534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1978069940 con 0x7f197815bb40 2024-03-08T06:00:59.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.013+0000 7f195ffff640 1 -- 172.21.15.76:0/3281255534 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f196801a070 con 0x7f197815bb40 2024-03-08T06:00:59.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.014+0000 7f195ffff640 1 --2- 172.21.15.76:0/3281255534 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f196406b4d0 0x7f196406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.014+0000 7f195ffff640 1 -- 172.21.15.76:0/3281255534 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f196808a730 con 0x7f197815bb40 2024-03-08T06:00:59.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.014+0000 7f19777fe640 1 --2- 172.21.15.76:0/3281255534 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f196406b4d0 0x7f196406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.014+0000 7f19777fe640 1 --2- 172.21.15.76:0/3281255534 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f196406b4d0 0x7f196406d990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f196c036000 tx=0x7f196c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:59.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.018+0000 7f195ffff640 1 -- 172.21.15.76:0/3281255534 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1978069940 con 0x7f197815bb40 2024-03-08T06:00:59.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.188+0000 7f197dfe2640 1 -- 172.21.15.76:0/3281255534 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f197804f030 con 0x7f196406b4d0 2024-03-08T06:00:59.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.196+0000 7f195ffff640 1 -- 172.21.15.76:0/3281255534 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f197804f030 con 0x7f196406b4d0 2024-03-08T06:00:59.200 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:59.202 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-08T06:00:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.199+0000 7f197dfe2640 1 -- 172.21.15.76:0/3281255534 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f196406b4d0 msgr2=0x7f196406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.199+0000 7f197dfe2640 1 --2- 172.21.15.76:0/3281255534 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f196406b4d0 0x7f196406d990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f196c036000 tx=0x7f196c008030 comp rx=0 tx=0).stop 2024-03-08T06:00:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.199+0000 7f197dfe2640 1 -- 172.21.15.76:0/3281255534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197815bb40 msgr2=0x7f19780b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.199+0000 7f197dfe2640 1 --2- 172.21.15.76:0/3281255534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197815bb40 0x7f19780b30e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f196800e870 tx=0x7f196800ed30 comp rx=0 tx=0).stop 2024-03-08T06:00:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.199+0000 7f197dfe2640 1 -- 172.21.15.76:0/3281255534 shutdown_connections 2024-03-08T06:00:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.199+0000 7f197dfe2640 1 --2- 172.21.15.76:0/3281255534 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f196406b4d0 0x7f196406d990 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.199+0000 7f197dfe2640 1 --2- 172.21.15.76:0/3281255534 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f197815bf20 0x7f19780b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.199+0000 7f197dfe2640 1 --2- 172.21.15.76:0/3281255534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197815bb40 0x7f19780b30e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.199+0000 7f197dfe2640 1 --2- 172.21.15.76:0/3281255534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19780cf5b0 0x7f197815e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.199+0000 7f197dfe2640 1 -- 172.21.15.76:0/3281255534 >> 172.21.15.76:0/3281255534 conn(0x7f197806f5e0 msgr2=0x7f19780ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.199+0000 7f197dfe2640 1 -- 172.21.15.76:0/3281255534 shutdown_connections 2024-03-08T06:00:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.199+0000 7f197dfe2640 1 -- 172.21.15.76:0/3281255534 wait complete. 2024-03-08T06:00:59.210 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-03-08T06:00:57.925087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219560,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755160,"statfs":{"total":773094113280,"available":772869283840,"internally_reserved":0,"allocated":3481600,"data_stored":2089616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003253"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.250953+0000","last_change":"2024-03-08T06:00:35.165081+0000","last_active":"2024-03-08T06:00:43.250953+0000","last_peered":"2024-03-08T06:00:43.250953+0000","last_clean":"2024-03-08T06:00:43.250953+0000","last_became_active":"2024-03-08T06:00:35.164818+0000","last_became_peered":"2024-03-08T06:00:35.164818+0000","last_unstale":"2024-03-08T06:00:43.250953+0000","last_undegraded":"2024-03-08T06:00:43.250953+0000","last_fullsized":"2024-03-08T06:00:43.250953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:59:03.457310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205755+0000","last_change":"2024-03-08T06:00:36.182716+0000","last_active":"2024-03-08T06:00:42.205755+0000","last_peered":"2024-03-08T06:00:42.205755+0000","last_clean":"2024-03-08T06:00:42.205755+0000","last_became_active":"2024-03-08T06:00:36.182377+0000","last_became_peered":"2024-03-08T06:00:36.182377+0000","last_unstale":"2024-03-08T06:00:42.205755+0000","last_undegraded":"2024-03-08T06:00:42.205755+0000","last_fullsized":"2024-03-08T06:00:42.205755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:59:39.518589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.204982+0000","last_change":"2024-03-08T06:00:40.773175+0000","last_active":"2024-03-08T06:00:42.204982+0000","last_peered":"2024-03-08T06:00:42.204982+0000","last_clean":"2024-03-08T06:00:42.204982+0000","last_became_active":"2024-03-08T06:00:40.772777+0000","last_became_peered":"2024-03-08T06:00:40.772777+0000","last_unstale":"2024-03-08T06:00:42.204982+0000","last_undegraded":"2024-03-08T06:00:42.204982+0000","last_fullsized":"2024-03-08T06:00:42.204982+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:51:44.068190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205870+0000","last_change":"2024-03-08T06:00:35.177115+0000","last_active":"2024-03-08T06:00:42.205870+0000","last_peered":"2024-03-08T06:00:42.205870+0000","last_clean":"2024-03-08T06:00:42.205870+0000","last_became_active":"2024-03-08T06:00:35.176827+0000","last_became_peered":"2024-03-08T06:00:35.176827+0000","last_unstale":"2024-03-08T06:00:42.205870+0000","last_undegraded":"2024-03-08T06:00:42.205870+0000","last_fullsized":"2024-03-08T06:00:42.205870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:57:55.324372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251714+0000","last_change":"2024-03-08T06:00:36.191516+0000","last_active":"2024-03-08T06:00:43.251714+0000","last_peered":"2024-03-08T06:00:43.251714+0000","last_clean":"2024-03-08T06:00:43.251714+0000","last_became_active":"2024-03-08T06:00:36.191258+0000","last_became_peered":"2024-03-08T06:00:36.191258+0000","last_unstale":"2024-03-08T06:00:43.251714+0000","last_undegraded":"2024-03-08T06:00:43.251714+0000","last_fullsized":"2024-03-08T06:00:43.251714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206166+0000","last_change":"2024-03-08T06:00:40.219312+0000","last_active":"2024-03-08T06:00:42.206166+0000","last_peered":"2024-03-08T06:00:42.206166+0000","last_clean":"2024-03-08T06:00:42.206166+0000","last_became_active":"2024-03-08T06:00:40.218891+0000","last_became_peered":"2024-03-08T06:00:40.218891+0000","last_unstale":"2024-03-08T06:00:42.206166+0000","last_undegraded":"2024-03-08T06:00:42.206166+0000","last_fullsized":"2024-03-08T06:00:42.206166+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:19:30.099147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752093+0000","last_change":"2024-03-08T06:00:35.163380+0000","last_active":"2024-03-08T06:00:42.752093+0000","last_peered":"2024-03-08T06:00:42.752093+0000","last_clean":"2024-03-08T06:00:42.752093+0000","last_became_active":"2024-03-08T06:00:35.163123+0000","last_became_peered":"2024-03-08T06:00:35.163123+0000","last_unstale":"2024-03-08T06:00:42.752093+0000","last_undegraded":"2024-03-08T06:00:42.752093+0000","last_fullsized":"2024-03-08T06:00:42.752093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252109+0000","last_change":"2024-03-08T06:00:36.194135+0000","last_active":"2024-03-08T06:00:43.252109+0000","last_peered":"2024-03-08T06:00:43.252109+0000","last_clean":"2024-03-08T06:00:43.252109+0000","last_became_active":"2024-03-08T06:00:36.193870+0000","last_became_peered":"2024-03-08T06:00:36.193870+0000","last_unstale":"2024-03-08T06:00:43.252109+0000","last_undegraded":"2024-03-08T06:00:43.252109+0000","last_fullsized":"2024-03-08T06:00:43.252109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:02:15.458747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206553+0000","last_change":"2024-03-08T06:00:40.221366+0000","last_active":"2024-03-08T06:00:42.206553+0000","last_peered":"2024-03-08T06:00:42.206553+0000","last_clean":"2024-03-08T06:00:42.206553+0000","last_became_active":"2024-03-08T06:00:40.220967+0000","last_became_peered":"2024-03-08T06:00:40.220967+0000","last_unstale":"2024-03-08T06:00:42.206553+0000","last_undegraded":"2024-03-08T06:00:42.206553+0000","last_fullsized":"2024-03-08T06:00:42.206553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:16:51.281060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751868+0000","last_change":"2024-03-08T06:00:35.173354+0000","last_active":"2024-03-08T06:00:42.751868+0000","last_peered":"2024-03-08T06:00:42.751868+0000","last_clean":"2024-03-08T06:00:42.751868+0000","last_became_active":"2024-03-08T06:00:35.173105+0000","last_became_peered":"2024-03-08T06:00:35.173105+0000","last_unstale":"2024-03-08T06:00:42.751868+0000","last_undegraded":"2024-03-08T06:00:42.751868+0000","last_fullsized":"2024-03-08T06:00:42.751868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252178+0000","last_change":"2024-03-08T06:00:36.193424+0000","last_active":"2024-03-08T06:00:43.252178+0000","last_peered":"2024-03-08T06:00:43.252178+0000","last_clean":"2024-03-08T06:00:43.252178+0000","last_became_active":"2024-03-08T06:00:36.191212+0000","last_became_peered":"2024-03-08T06:00:36.191212+0000","last_unstale":"2024-03-08T06:00:43.252178+0000","last_undegraded":"2024-03-08T06:00:43.252178+0000","last_fullsized":"2024-03-08T06:00:43.252178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:02.331109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206122+0000","last_change":"2024-03-08T06:00:40.221344+0000","last_active":"2024-03-08T06:00:42.206122+0000","last_peered":"2024-03-08T06:00:42.206122+0000","last_clean":"2024-03-08T06:00:42.206122+0000","last_became_active":"2024-03-08T06:00:40.220925+0000","last_became_peered":"2024-03-08T06:00:40.220925+0000","last_unstale":"2024-03-08T06:00:42.206122+0000","last_undegraded":"2024-03-08T06:00:42.206122+0000","last_fullsized":"2024-03-08T06:00:42.206122+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:06.372126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254602+0000","last_change":"2024-03-08T06:00:36.186043+0000","last_active":"2024-03-08T06:00:43.254602+0000","last_peered":"2024-03-08T06:00:43.254602+0000","last_clean":"2024-03-08T06:00:43.254602+0000","last_became_active":"2024-03-08T06:00:36.185793+0000","last_became_peered":"2024-03-08T06:00:36.185793+0000","last_unstale":"2024-03-08T06:00:43.254602+0000","last_undegraded":"2024-03-08T06:00:43.254602+0000","last_fullsized":"2024-03-08T06:00:43.254602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:01:37.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751980+0000","last_change":"2024-03-08T06:00:35.163406+0000","last_active":"2024-03-08T06:00:42.751980+0000","last_peered":"2024-03-08T06:00:42.751980+0000","last_clean":"2024-03-08T06:00:42.751980+0000","last_became_active":"2024-03-08T06:00:35.163138+0000","last_became_peered":"2024-03-08T06:00:35.163138+0000","last_unstale":"2024-03-08T06:00:42.751980+0000","last_undegraded":"2024-03-08T06:00:42.751980+0000","last_fullsized":"2024-03-08T06:00:42.751980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:38.366092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206808+0000","last_change":"2024-03-08T06:00:40.219654+0000","last_active":"2024-03-08T06:00:42.206808+0000","last_peered":"2024-03-08T06:00:42.206808+0000","last_clean":"2024-03-08T06:00:42.206808+0000","last_became_active":"2024-03-08T06:00:40.219225+0000","last_became_peered":"2024-03-08T06:00:40.219225+0000","last_unstale":"2024-03-08T06:00:42.206808+0000","last_undegraded":"2024-03-08T06:00:42.206808+0000","last_fullsized":"2024-03-08T06:00:42.206808+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:34:22.301392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205304+0000","last_change":"2024-03-08T06:00:36.184886+0000","last_active":"2024-03-08T06:00:42.205304+0000","last_peered":"2024-03-08T06:00:42.205304+0000","last_clean":"2024-03-08T06:00:42.205304+0000","last_became_active":"2024-03-08T06:00:36.184582+0000","last_became_peered":"2024-03-08T06:00:36.184582+0000","last_unstale":"2024-03-08T06:00:42.205304+0000","last_undegraded":"2024-03-08T06:00:42.205304+0000","last_fullsized":"2024-03-08T06:00:42.205304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251830+0000","last_change":"2024-03-08T06:00:35.171210+0000","last_active":"2024-03-08T06:00:43.251830+0000","last_peered":"2024-03-08T06:00:43.251830+0000","last_clean":"2024-03-08T06:00:43.251830+0000","last_became_active":"2024-03-08T06:00:35.170784+0000","last_became_peered":"2024-03-08T06:00:35.170784+0000","last_unstale":"2024-03-08T06:00:43.251830+0000","last_undegraded":"2024-03-08T06:00:43.251830+0000","last_fullsized":"2024-03-08T06:00:43.251830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206300+0000","last_change":"2024-03-08T06:00:40.221460+0000","last_active":"2024-03-08T06:00:42.206300+0000","last_peered":"2024-03-08T06:00:42.206300+0000","last_clean":"2024-03-08T06:00:42.206300+0000","last_became_active":"2024-03-08T06:00:40.221046+0000","last_became_peered":"2024-03-08T06:00:40.221046+0000","last_unstale":"2024-03-08T06:00:42.206300+0000","last_undegraded":"2024-03-08T06:00:42.206300+0000","last_fullsized":"2024-03-08T06:00:42.206300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:35.686997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205125+0000","last_change":"2024-03-08T06:00:36.181765+0000","last_active":"2024-03-08T06:00:42.205125+0000","last_peered":"2024-03-08T06:00:42.205125+0000","last_clean":"2024-03-08T06:00:42.205125+0000","last_became_active":"2024-03-08T06:00:36.181503+0000","last_became_peered":"2024-03-08T06:00:36.181503+0000","last_unstale":"2024-03-08T06:00:42.205125+0000","last_undegraded":"2024-03-08T06:00:42.205125+0000","last_fullsized":"2024-03-08T06:00:42.205125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:01:37.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.250999+0000","last_change":"2024-03-08T06:00:35.164505+0000","last_active":"2024-03-08T06:00:43.250999+0000","last_peered":"2024-03-08T06:00:43.250999+0000","last_clean":"2024-03-08T06:00:43.250999+0000","last_became_active":"2024-03-08T06:00:35.164203+0000","last_became_peered":"2024-03-08T06:00:35.164203+0000","last_unstale":"2024-03-08T06:00:43.250999+0000","last_undegraded":"2024-03-08T06:00:43.250999+0000","last_fullsized":"2024-03-08T06:00:43.250999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:38.366092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253073+0000","last_change":"2024-03-08T06:00:40.772590+0000","last_active":"2024-03-08T06:00:43.253073+0000","last_peered":"2024-03-08T06:00:43.253073+0000","last_clean":"2024-03-08T06:00:43.253073+0000","last_became_active":"2024-03-08T06:00:40.772198+0000","last_became_peered":"2024-03-08T06:00:40.772198+0000","last_unstale":"2024-03-08T06:00:43.253073+0000","last_undegraded":"2024-03-08T06:00:43.253073+0000","last_fullsized":"2024-03-08T06:00:43.253073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:24:20.856936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206949+0000","last_change":"2024-03-08T06:00:36.180323+0000","last_active":"2024-03-08T06:00:42.206949+0000","last_peered":"2024-03-08T06:00:42.206949+0000","last_clean":"2024-03-08T06:00:42.206949+0000","last_became_active":"2024-03-08T06:00:36.180087+0000","last_became_peered":"2024-03-08T06:00:36.180087+0000","last_unstale":"2024-03-08T06:00:42.206949+0000","last_undegraded":"2024-03-08T06:00:42.206949+0000","last_fullsized":"2024-03-08T06:00:42.206949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:30:07.063230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751552+0000","last_change":"2024-03-08T06:00:35.179401+0000","last_active":"2024-03-08T06:00:42.751552+0000","last_peered":"2024-03-08T06:00:42.751552+0000","last_clean":"2024-03-08T06:00:42.751552+0000","last_became_active":"2024-03-08T06:00:35.179121+0000","last_became_peered":"2024-03-08T06:00:35.179121+0000","last_unstale":"2024-03-08T06:00:42.751552+0000","last_undegraded":"2024-03-08T06:00:42.751552+0000","last_fullsized":"2024-03-08T06:00:42.751552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:30:06.058910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.207092+0000","last_change":"2024-03-08T06:00:40.222278+0000","last_active":"2024-03-08T06:00:42.207092+0000","last_peered":"2024-03-08T06:00:42.207092+0000","last_clean":"2024-03-08T06:00:42.207092+0000","last_became_active":"2024-03-08T06:00:40.221874+0000","last_became_peered":"2024-03-08T06:00:40.221874+0000","last_unstale":"2024-03-08T06:00:42.207092+0000","last_undegraded":"2024-03-08T06:00:42.207092+0000","last_fullsized":"2024-03-08T06:00:42.207092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:33.029056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251991+0000","last_change":"2024-03-08T06:00:36.180994+0000","last_active":"2024-03-08T06:00:43.251991+0000","last_peered":"2024-03-08T06:00:43.251991+0000","last_clean":"2024-03-08T06:00:43.251991+0000","last_became_active":"2024-03-08T06:00:36.178891+0000","last_became_peered":"2024-03-08T06:00:36.178891+0000","last_unstale":"2024-03-08T06:00:43.251991+0000","last_undegraded":"2024-03-08T06:00:43.251991+0000","last_fullsized":"2024-03-08T06:00:43.251991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:40:16.053974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253772+0000","last_change":"2024-03-08T06:00:35.166077+0000","last_active":"2024-03-08T06:00:43.253772+0000","last_peered":"2024-03-08T06:00:43.253772+0000","last_clean":"2024-03-08T06:00:43.253772+0000","last_became_active":"2024-03-08T06:00:35.165721+0000","last_became_peered":"2024-03-08T06:00:35.165721+0000","last_unstale":"2024-03-08T06:00:43.253772+0000","last_undegraded":"2024-03-08T06:00:43.253772+0000","last_fullsized":"2024-03-08T06:00:43.253772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206253+0000","last_change":"2024-03-08T06:00:40.772000+0000","last_active":"2024-03-08T06:00:42.206253+0000","last_peered":"2024-03-08T06:00:42.206253+0000","last_clean":"2024-03-08T06:00:42.206253+0000","last_became_active":"2024-03-08T06:00:40.771449+0000","last_became_peered":"2024-03-08T06:00:40.771449+0000","last_unstale":"2024-03-08T06:00:42.206253+0000","last_undegraded":"2024-03-08T06:00:42.206253+0000","last_fullsized":"2024-03-08T06:00:42.206253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:44:48.992193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205931+0000","last_change":"2024-03-08T06:00:36.185234+0000","last_active":"2024-03-08T06:00:42.205931+0000","last_peered":"2024-03-08T06:00:42.205931+0000","last_clean":"2024-03-08T06:00:42.205931+0000","last_became_active":"2024-03-08T06:00:36.184020+0000","last_became_peered":"2024-03-08T06:00:36.184020+0000","last_unstale":"2024-03-08T06:00:42.205931+0000","last_undegraded":"2024-03-08T06:00:42.205931+0000","last_fullsized":"2024-03-08T06:00:42.205931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:39:59.986300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252725+0000","last_change":"2024-03-08T06:00:35.166222+0000","last_active":"2024-03-08T06:00:43.252725+0000","last_peered":"2024-03-08T06:00:43.252725+0000","last_clean":"2024-03-08T06:00:43.252725+0000","last_became_active":"2024-03-08T06:00:35.165815+0000","last_became_peered":"2024-03-08T06:00:35.165815+0000","last_unstale":"2024-03-08T06:00:43.252725+0000","last_undegraded":"2024-03-08T06:00:43.252725+0000","last_fullsized":"2024-03-08T06:00:43.252725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.204928+0000","last_change":"2024-03-08T06:00:40.771309+0000","last_active":"2024-03-08T06:00:42.204928+0000","last_peered":"2024-03-08T06:00:42.204928+0000","last_clean":"2024-03-08T06:00:42.204928+0000","last_became_active":"2024-03-08T06:00:40.770916+0000","last_became_peered":"2024-03-08T06:00:40.770916+0000","last_unstale":"2024-03-08T06:00:42.204928+0000","last_undegraded":"2024-03-08T06:00:42.204928+0000","last_fullsized":"2024-03-08T06:00:42.204928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:26:51.436673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252117+0000","last_change":"2024-03-08T06:00:36.177665+0000","last_active":"2024-03-08T06:00:43.252117+0000","last_peered":"2024-03-08T06:00:43.252117+0000","last_clean":"2024-03-08T06:00:43.252117+0000","last_became_active":"2024-03-08T06:00:36.177409+0000","last_became_peered":"2024-03-08T06:00:36.177409+0000","last_unstale":"2024-03-08T06:00:43.252117+0000","last_undegraded":"2024-03-08T06:00:43.252117+0000","last_fullsized":"2024-03-08T06:00:43.252117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752203+0000","last_change":"2024-03-08T06:00:35.179418+0000","last_active":"2024-03-08T06:00:42.752203+0000","last_peered":"2024-03-08T06:00:42.752203+0000","last_clean":"2024-03-08T06:00:42.752203+0000","last_became_active":"2024-03-08T06:00:35.179132+0000","last_became_peered":"2024-03-08T06:00:35.179132+0000","last_unstale":"2024-03-08T06:00:42.752203+0000","last_undegraded":"2024-03-08T06:00:42.752203+0000","last_fullsized":"2024-03-08T06:00:42.752203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:11.262668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251653+0000","last_change":"2024-03-08T06:00:40.771662+0000","last_active":"2024-03-08T06:00:43.251653+0000","last_peered":"2024-03-08T06:00:43.251653+0000","last_clean":"2024-03-08T06:00:43.251653+0000","last_became_active":"2024-03-08T06:00:40.771168+0000","last_became_peered":"2024-03-08T06:00:40.771168+0000","last_unstale":"2024-03-08T06:00:43.251653+0000","last_undegraded":"2024-03-08T06:00:43.251653+0000","last_fullsized":"2024-03-08T06:00:43.251653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:16:51.281060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252306+0000","last_change":"2024-03-08T06:00:36.193542+0000","last_active":"2024-03-08T06:00:43.252306+0000","last_peered":"2024-03-08T06:00:43.252306+0000","last_clean":"2024-03-08T06:00:43.252306+0000","last_became_active":"2024-03-08T06:00:36.191695+0000","last_became_peered":"2024-03-08T06:00:36.191695+0000","last_unstale":"2024-03-08T06:00:43.252306+0000","last_undegraded":"2024-03-08T06:00:43.252306+0000","last_fullsized":"2024-03-08T06:00:43.252306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:16:14.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252207+0000","last_change":"2024-03-08T06:00:35.177137+0000","last_active":"2024-03-08T06:00:43.252207+0000","last_peered":"2024-03-08T06:00:43.252207+0000","last_clean":"2024-03-08T06:00:43.252207+0000","last_became_active":"2024-03-08T06:00:35.176790+0000","last_became_peered":"2024-03-08T06:00:35.176790+0000","last_unstale":"2024-03-08T06:00:43.252207+0000","last_undegraded":"2024-03-08T06:00:43.252207+0000","last_fullsized":"2024-03-08T06:00:43.252207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:11.262668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252112+0000","last_change":"2024-03-08T06:00:40.220329+0000","last_active":"2024-03-08T06:00:43.252112+0000","last_peered":"2024-03-08T06:00:43.252112+0000","last_clean":"2024-03-08T06:00:43.252112+0000","last_became_active":"2024-03-08T06:00:40.219921+0000","last_became_peered":"2024-03-08T06:00:40.219921+0000","last_unstale":"2024-03-08T06:00:43.252112+0000","last_undegraded":"2024-03-08T06:00:43.252112+0000","last_fullsized":"2024-03-08T06:00:43.252112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:08:55.908866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.207142+0000","last_change":"2024-03-08T06:00:36.187190+0000","last_active":"2024-03-08T06:00:42.207142+0000","last_peered":"2024-03-08T06:00:42.207142+0000","last_clean":"2024-03-08T06:00:42.207142+0000","last_became_active":"2024-03-08T06:00:36.186916+0000","last_became_peered":"2024-03-08T06:00:36.186916+0000","last_unstale":"2024-03-08T06:00:42.207142+0000","last_undegraded":"2024-03-08T06:00:42.207142+0000","last_fullsized":"2024-03-08T06:00:42.207142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:39:00.058426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251977+0000","last_change":"2024-03-08T06:00:35.179558+0000","last_active":"2024-03-08T06:00:43.251977+0000","last_peered":"2024-03-08T06:00:43.251977+0000","last_clean":"2024-03-08T06:00:43.251977+0000","last_became_active":"2024-03-08T06:00:35.179266+0000","last_became_peered":"2024-03-08T06:00:35.179266+0000","last_unstale":"2024-03-08T06:00:43.251977+0000","last_undegraded":"2024-03-08T06:00:43.251977+0000","last_fullsized":"2024-03-08T06:00:43.251977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:38.366092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252067+0000","last_change":"2024-03-08T06:00:40.224570+0000","last_active":"2024-03-08T06:00:43.252067+0000","last_peered":"2024-03-08T06:00:43.252067+0000","last_clean":"2024-03-08T06:00:43.252067+0000","last_became_active":"2024-03-08T06:00:40.222949+0000","last_became_peered":"2024-03-08T06:00:40.222949+0000","last_unstale":"2024-03-08T06:00:43.252067+0000","last_undegraded":"2024-03-08T06:00:43.252067+0000","last_fullsized":"2024-03-08T06:00:43.252067+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:16:51.281060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251279+0000","last_change":"2024-03-08T06:00:36.178768+0000","last_active":"2024-03-08T06:00:43.251279+0000","last_peered":"2024-03-08T06:00:43.251279+0000","last_clean":"2024-03-08T06:00:43.251279+0000","last_became_active":"2024-03-08T06:00:36.177890+0000","last_became_peered":"2024-03-08T06:00:36.177890+0000","last_unstale":"2024-03-08T06:00:43.251279+0000","last_undegraded":"2024-03-08T06:00:43.251279+0000","last_fullsized":"2024-03-08T06:00:43.251279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:01:37.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251961+0000","last_change":"2024-03-08T06:00:35.177031+0000","last_active":"2024-03-08T06:00:43.251961+0000","last_peered":"2024-03-08T06:00:43.251961+0000","last_clean":"2024-03-08T06:00:43.251961+0000","last_became_active":"2024-03-08T06:00:35.176687+0000","last_became_peered":"2024-03-08T06:00:35.176687+0000","last_unstale":"2024-03-08T06:00:43.251961+0000","last_undegraded":"2024-03-08T06:00:43.251961+0000","last_fullsized":"2024-03-08T06:00:43.251961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253204+0000","last_change":"2024-03-08T06:00:40.219392+0000","last_active":"2024-03-08T06:00:43.253204+0000","last_peered":"2024-03-08T06:00:43.253204+0000","last_clean":"2024-03-08T06:00:43.253204+0000","last_became_active":"2024-03-08T06:00:40.219040+0000","last_became_peered":"2024-03-08T06:00:40.219040+0000","last_unstale":"2024-03-08T06:00:43.253204+0000","last_undegraded":"2024-03-08T06:00:43.253204+0000","last_fullsized":"2024-03-08T06:00:43.253204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:33.029056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251221+0000","last_change":"2024-03-08T06:00:36.185525+0000","last_active":"2024-03-08T06:00:43.251221+0000","last_peered":"2024-03-08T06:00:43.251221+0000","last_clean":"2024-03-08T06:00:43.251221+0000","last_became_active":"2024-03-08T06:00:36.185111+0000","last_became_peered":"2024-03-08T06:00:36.185111+0000","last_unstale":"2024-03-08T06:00:43.251221+0000","last_undegraded":"2024-03-08T06:00:43.251221+0000","last_fullsized":"2024-03-08T06:00:43.251221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:52.669024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751556+0000","last_change":"2024-03-08T06:00:35.166949+0000","last_active":"2024-03-08T06:00:42.751556+0000","last_peered":"2024-03-08T06:00:42.751556+0000","last_clean":"2024-03-08T06:00:42.751556+0000","last_became_active":"2024-03-08T06:00:35.166680+0000","last_became_peered":"2024-03-08T06:00:35.166680+0000","last_unstale":"2024-03-08T06:00:42.751556+0000","last_undegraded":"2024-03-08T06:00:42.751556+0000","last_fullsized":"2024-03-08T06:00:42.751556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:58:33.115927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251873+0000","last_change":"2024-03-08T06:00:40.771268+0000","last_active":"2024-03-08T06:00:43.251873+0000","last_peered":"2024-03-08T06:00:43.251873+0000","last_clean":"2024-03-08T06:00:43.251873+0000","last_became_active":"2024-03-08T06:00:40.770846+0000","last_became_peered":"2024-03-08T06:00:40.770846+0000","last_unstale":"2024-03-08T06:00:43.251873+0000","last_undegraded":"2024-03-08T06:00:43.251873+0000","last_fullsized":"2024-03-08T06:00:43.251873+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:34:22.301392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205219+0000","last_change":"2024-03-08T06:00:36.200350+0000","last_active":"2024-03-08T06:00:42.205219+0000","last_peered":"2024-03-08T06:00:42.205219+0000","last_clean":"2024-03-08T06:00:42.205219+0000","last_became_active":"2024-03-08T06:00:36.200036+0000","last_became_peered":"2024-03-08T06:00:36.200036+0000","last_unstale":"2024-03-08T06:00:42.205219+0000","last_undegraded":"2024-03-08T06:00:42.205219+0000","last_fullsized":"2024-03-08T06:00:42.205219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:10:17.180499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251960+0000","last_change":"2024-03-08T06:00:35.176857+0000","last_active":"2024-03-08T06:00:43.251960+0000","last_peered":"2024-03-08T06:00:43.251960+0000","last_clean":"2024-03-08T06:00:43.251960+0000","last_became_active":"2024-03-08T06:00:35.176593+0000","last_became_peered":"2024-03-08T06:00:35.176593+0000","last_unstale":"2024-03-08T06:00:43.251960+0000","last_undegraded":"2024-03-08T06:00:43.251960+0000","last_fullsized":"2024-03-08T06:00:43.251960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:41.059111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751648+0000","last_change":"2024-03-08T06:00:26.178419+0000","last_active":"2024-03-08T06:00:42.751648+0000","last_peered":"2024-03-08T06:00:42.751648+0000","last_clean":"2024-03-08T06:00:42.751648+0000","last_became_active":"2024-03-08T06:00:26.175826+0000","last_became_peered":"2024-03-08T06:00:26.175826+0000","last_unstale":"2024-03-08T06:00:42.751648+0000","last_undegraded":"2024-03-08T06:00:42.751648+0000","last_fullsized":"2024-03-08T06:00:42.751648+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_clean_scrub_stamp":"2024-03-08T06:00:24.029492+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:36:00.682517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254739+0000","last_change":"2024-03-08T06:00:40.773106+0000","last_active":"2024-03-08T06:00:43.254739+0000","last_peered":"2024-03-08T06:00:43.254739+0000","last_clean":"2024-03-08T06:00:43.254739+0000","last_became_active":"2024-03-08T06:00:40.772665+0000","last_became_peered":"2024-03-08T06:00:40.772665+0000","last_unstale":"2024-03-08T06:00:43.254739+0000","last_undegraded":"2024-03-08T06:00:43.254739+0000","last_fullsized":"2024-03-08T06:00:43.254739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:02:19.499764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752004+0000","last_change":"2024-03-08T06:00:36.204915+0000","last_active":"2024-03-08T06:00:42.752004+0000","last_peered":"2024-03-08T06:00:42.752004+0000","last_clean":"2024-03-08T06:00:42.752004+0000","last_became_active":"2024-03-08T06:00:36.204663+0000","last_became_peered":"2024-03-08T06:00:36.204663+0000","last_unstale":"2024-03-08T06:00:42.752004+0000","last_undegraded":"2024-03-08T06:00:42.752004+0000","last_fullsized":"2024-03-08T06:00:42.752004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:05.458469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252634+0000","last_change":"2024-03-08T06:00:35.173339+0000","last_active":"2024-03-08T06:00:43.252634+0000","last_peered":"2024-03-08T06:00:43.252634+0000","last_clean":"2024-03-08T06:00:43.252634+0000","last_became_active":"2024-03-08T06:00:35.173024+0000","last_became_peered":"2024-03-08T06:00:35.173024+0000","last_unstale":"2024-03-08T06:00:43.252634+0000","last_undegraded":"2024-03-08T06:00:43.252634+0000","last_fullsized":"2024-03-08T06:00:43.252634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:41.059111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254629+0000","last_change":"2024-03-08T06:00:40.771194+0000","last_active":"2024-03-08T06:00:43.254629+0000","last_peered":"2024-03-08T06:00:43.254629+0000","last_clean":"2024-03-08T06:00:43.254629+0000","last_became_active":"2024-03-08T06:00:40.770762+0000","last_became_peered":"2024-03-08T06:00:40.770762+0000","last_unstale":"2024-03-08T06:00:43.254629+0000","last_undegraded":"2024-03-08T06:00:43.254629+0000","last_fullsized":"2024-03-08T06:00:43.254629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:21:03.123682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252111+0000","last_change":"2024-03-08T06:00:36.186111+0000","last_active":"2024-03-08T06:00:43.252111+0000","last_peered":"2024-03-08T06:00:43.252111+0000","last_clean":"2024-03-08T06:00:43.252111+0000","last_became_active":"2024-03-08T06:00:36.185865+0000","last_became_peered":"2024-03-08T06:00:36.185865+0000","last_unstale":"2024-03-08T06:00:43.252111+0000","last_undegraded":"2024-03-08T06:00:43.252111+0000","last_fullsized":"2024-03-08T06:00:43.252111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:19:26.058130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251711+0000","last_change":"2024-03-08T06:00:35.166769+0000","last_active":"2024-03-08T06:00:43.251711+0000","last_peered":"2024-03-08T06:00:43.251711+0000","last_clean":"2024-03-08T06:00:43.251711+0000","last_became_active":"2024-03-08T06:00:35.166396+0000","last_became_peered":"2024-03-08T06:00:35.166396+0000","last_unstale":"2024-03-08T06:00:43.251711+0000","last_undegraded":"2024-03-08T06:00:43.251711+0000","last_fullsized":"2024-03-08T06:00:43.251711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:30:06.058910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251907+0000","last_change":"2024-03-08T06:00:40.219431+0000","last_active":"2024-03-08T06:00:43.251907+0000","last_peered":"2024-03-08T06:00:43.251907+0000","last_clean":"2024-03-08T06:00:43.251907+0000","last_became_active":"2024-03-08T06:00:40.219091+0000","last_became_peered":"2024-03-08T06:00:40.219091+0000","last_unstale":"2024-03-08T06:00:43.251907+0000","last_undegraded":"2024-03-08T06:00:43.251907+0000","last_fullsized":"2024-03-08T06:00:43.251907+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:24:20.856936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751480+0000","last_change":"2024-03-08T06:00:36.202917+0000","last_active":"2024-03-08T06:00:42.751480+0000","last_peered":"2024-03-08T06:00:42.751480+0000","last_clean":"2024-03-08T06:00:42.751480+0000","last_became_active":"2024-03-08T06:00:36.202100+0000","last_became_peered":"2024-03-08T06:00:36.202100+0000","last_unstale":"2024-03-08T06:00:42.751480+0000","last_undegraded":"2024-03-08T06:00:42.751480+0000","last_fullsized":"2024-03-08T06:00:42.751480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:31.645981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252205+0000","last_change":"2024-03-08T06:00:35.171650+0000","last_active":"2024-03-08T06:00:43.252205+0000","last_peered":"2024-03-08T06:00:43.252205+0000","last_clean":"2024-03-08T06:00:43.252205+0000","last_became_active":"2024-03-08T06:00:35.171310+0000","last_became_peered":"2024-03-08T06:00:35.171310+0000","last_unstale":"2024-03-08T06:00:43.252205+0000","last_undegraded":"2024-03-08T06:00:43.252205+0000","last_fullsized":"2024-03-08T06:00:43.252205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251524+0000","last_change":"2024-03-08T06:00:40.774574+0000","last_active":"2024-03-08T06:00:43.251524+0000","last_peered":"2024-03-08T06:00:43.251524+0000","last_clean":"2024-03-08T06:00:43.251524+0000","last_became_active":"2024-03-08T06:00:40.774172+0000","last_became_peered":"2024-03-08T06:00:40.774172+0000","last_unstale":"2024-03-08T06:00:43.251524+0000","last_undegraded":"2024-03-08T06:00:43.251524+0000","last_fullsized":"2024-03-08T06:00:43.251524+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:48:26.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752182+0000","last_change":"2024-03-08T06:00:36.195883+0000","last_active":"2024-03-08T06:00:42.752182+0000","last_peered":"2024-03-08T06:00:42.752182+0000","last_clean":"2024-03-08T06:00:42.752182+0000","last_became_active":"2024-03-08T06:00:36.195641+0000","last_became_peered":"2024-03-08T06:00:36.195641+0000","last_unstale":"2024-03-08T06:00:42.752182+0000","last_undegraded":"2024-03-08T06:00:42.752182+0000","last_fullsized":"2024-03-08T06:00:42.752182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:02:15.458747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206956+0000","last_change":"2024-03-08T06:00:35.165484+0000","last_active":"2024-03-08T06:00:42.206956+0000","last_peered":"2024-03-08T06:00:42.206956+0000","last_clean":"2024-03-08T06:00:42.206956+0000","last_became_active":"2024-03-08T06:00:35.165147+0000","last_became_peered":"2024-03-08T06:00:35.165147+0000","last_unstale":"2024-03-08T06:00:42.206956+0000","last_undegraded":"2024-03-08T06:00:42.206956+0000","last_fullsized":"2024-03-08T06:00:42.206956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:07:54.742119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206806+0000","last_change":"2024-03-08T06:00:40.221435+0000","last_active":"2024-03-08T06:00:42.206806+0000","last_peered":"2024-03-08T06:00:42.206806+0000","last_clean":"2024-03-08T06:00:42.206806+0000","last_became_active":"2024-03-08T06:00:40.220886+0000","last_became_peered":"2024-03-08T06:00:40.220886+0000","last_unstale":"2024-03-08T06:00:42.206806+0000","last_undegraded":"2024-03-08T06:00:42.206806+0000","last_fullsized":"2024-03-08T06:00:42.206806+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:24:20.856936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751752+0000","last_change":"2024-03-08T06:00:36.202146+0000","last_active":"2024-03-08T06:00:42.751752+0000","last_peered":"2024-03-08T06:00:42.751752+0000","last_clean":"2024-03-08T06:00:42.751752+0000","last_became_active":"2024-03-08T06:00:36.201880+0000","last_became_peered":"2024-03-08T06:00:36.201880+0000","last_unstale":"2024-03-08T06:00:42.751752+0000","last_undegraded":"2024-03-08T06:00:42.751752+0000","last_fullsized":"2024-03-08T06:00:42.751752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:45:22.429641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252369+0000","last_change":"2024-03-08T06:00:35.168305+0000","last_active":"2024-03-08T06:00:43.252369+0000","last_peered":"2024-03-08T06:00:43.252369+0000","last_clean":"2024-03-08T06:00:43.252369+0000","last_became_active":"2024-03-08T06:00:35.168007+0000","last_became_peered":"2024-03-08T06:00:35.168007+0000","last_unstale":"2024-03-08T06:00:43.252369+0000","last_undegraded":"2024-03-08T06:00:43.252369+0000","last_fullsized":"2024-03-08T06:00:43.252369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:57:55.324372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251733+0000","last_change":"2024-03-08T06:00:40.217477+0000","last_active":"2024-03-08T06:00:43.251733+0000","last_peered":"2024-03-08T06:00:43.251733+0000","last_clean":"2024-03-08T06:00:43.251733+0000","last_became_active":"2024-03-08T06:00:40.217140+0000","last_became_peered":"2024-03-08T06:00:40.217140+0000","last_unstale":"2024-03-08T06:00:43.251733+0000","last_undegraded":"2024-03-08T06:00:43.251733+0000","last_fullsized":"2024-03-08T06:00:43.251733+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:08:55.908866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254784+0000","last_change":"2024-03-08T06:00:36.188881+0000","last_active":"2024-03-08T06:00:43.254784+0000","last_peered":"2024-03-08T06:00:43.254784+0000","last_clean":"2024-03-08T06:00:43.254784+0000","last_became_active":"2024-03-08T06:00:36.188323+0000","last_became_peered":"2024-03-08T06:00:36.188323+0000","last_unstale":"2024-03-08T06:00:43.254784+0000","last_undegraded":"2024-03-08T06:00:43.254784+0000","last_fullsized":"2024-03-08T06:00:43.254784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:11:34.950063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253637+0000","last_change":"2024-03-08T06:00:35.163917+0000","last_active":"2024-03-08T06:00:43.253637+0000","last_peered":"2024-03-08T06:00:43.253637+0000","last_clean":"2024-03-08T06:00:43.253637+0000","last_became_active":"2024-03-08T06:00:35.163650+0000","last_became_peered":"2024-03-08T06:00:35.163650+0000","last_unstale":"2024-03-08T06:00:43.253637+0000","last_undegraded":"2024-03-08T06:00:43.253637+0000","last_fullsized":"2024-03-08T06:00:43.253637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:11.262668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205799+0000","last_change":"2024-03-08T06:00:40.773617+0000","last_active":"2024-03-08T06:00:42.205799+0000","last_peered":"2024-03-08T06:00:42.205799+0000","last_clean":"2024-03-08T06:00:42.205799+0000","last_became_active":"2024-03-08T06:00:40.773144+0000","last_became_peered":"2024-03-08T06:00:40.773144+0000","last_unstale":"2024-03-08T06:00:42.205799+0000","last_undegraded":"2024-03-08T06:00:42.205799+0000","last_fullsized":"2024-03-08T06:00:42.205799+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:32:08.762183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751941+0000","last_change":"2024-03-08T06:00:36.202751+0000","last_active":"2024-03-08T06:00:42.751941+0000","last_peered":"2024-03-08T06:00:42.751941+0000","last_clean":"2024-03-08T06:00:42.751941+0000","last_became_active":"2024-03-08T06:00:36.202228+0000","last_became_peered":"2024-03-08T06:00:36.202228+0000","last_unstale":"2024-03-08T06:00:42.751941+0000","last_undegraded":"2024-03-08T06:00:42.751941+0000","last_fullsized":"2024-03-08T06:00:42.751941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:16:14.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251319+0000","last_change":"2024-03-08T06:00:35.176896+0000","last_active":"2024-03-08T06:00:43.251319+0000","last_peered":"2024-03-08T06:00:43.251319+0000","last_clean":"2024-03-08T06:00:43.251319+0000","last_became_active":"2024-03-08T06:00:35.176665+0000","last_became_peered":"2024-03-08T06:00:35.176665+0000","last_unstale":"2024-03-08T06:00:43.251319+0000","last_undegraded":"2024-03-08T06:00:43.251319+0000","last_fullsized":"2024-03-08T06:00:43.251319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:11.262668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205674+0000","last_change":"2024-03-08T06:00:40.771254+0000","last_active":"2024-03-08T06:00:42.205674+0000","last_peered":"2024-03-08T06:00:42.205674+0000","last_clean":"2024-03-08T06:00:42.205674+0000","last_became_active":"2024-03-08T06:00:40.770828+0000","last_became_peered":"2024-03-08T06:00:40.770828+0000","last_unstale":"2024-03-08T06:00:42.205674+0000","last_undegraded":"2024-03-08T06:00:42.205674+0000","last_fullsized":"2024-03-08T06:00:42.205674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:06:36.831014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251667+0000","last_change":"2024-03-08T06:00:36.192057+0000","last_active":"2024-03-08T06:00:43.251667+0000","last_peered":"2024-03-08T06:00:43.251667+0000","last_clean":"2024-03-08T06:00:43.251667+0000","last_became_active":"2024-03-08T06:00:36.191786+0000","last_became_peered":"2024-03-08T06:00:36.191786+0000","last_unstale":"2024-03-08T06:00:43.251667+0000","last_undegraded":"2024-03-08T06:00:43.251667+0000","last_fullsized":"2024-03-08T06:00:43.251667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:10:17.180499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251277+0000","last_change":"2024-03-08T06:00:35.164201+0000","last_active":"2024-03-08T06:00:43.251277+0000","last_peered":"2024-03-08T06:00:43.251277+0000","last_clean":"2024-03-08T06:00:43.251277+0000","last_became_active":"2024-03-08T06:00:35.163882+0000","last_became_peered":"2024-03-08T06:00:35.163882+0000","last_unstale":"2024-03-08T06:00:43.251277+0000","last_undegraded":"2024-03-08T06:00:43.251277+0000","last_fullsized":"2024-03-08T06:00:43.251277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:43:03.634887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252307+0000","last_change":"2024-03-08T06:00:40.773272+0000","last_active":"2024-03-08T06:00:43.252307+0000","last_peered":"2024-03-08T06:00:43.252307+0000","last_clean":"2024-03-08T06:00:43.252307+0000","last_became_active":"2024-03-08T06:00:40.772833+0000","last_became_peered":"2024-03-08T06:00:40.772833+0000","last_unstale":"2024-03-08T06:00:43.252307+0000","last_undegraded":"2024-03-08T06:00:43.252307+0000","last_fullsized":"2024-03-08T06:00:43.252307+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:53:52.773407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751887+0000","last_change":"2024-03-08T06:00:36.191731+0000","last_active":"2024-03-08T06:00:42.751887+0000","last_peered":"2024-03-08T06:00:42.751887+0000","last_clean":"2024-03-08T06:00:42.751887+0000","last_became_active":"2024-03-08T06:00:36.191469+0000","last_became_peered":"2024-03-08T06:00:36.191469+0000","last_unstale":"2024-03-08T06:00:42.751887+0000","last_undegraded":"2024-03-08T06:00:42.751887+0000","last_fullsized":"2024-03-08T06:00:42.751887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:59.082666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:45.428919+0000","last_change":"2024-03-08T06:00:35.171245+0000","last_active":"2024-03-08T06:00:45.428919+0000","last_peered":"2024-03-08T06:00:45.428919+0000","last_clean":"2024-03-08T06:00:45.428919+0000","last_became_active":"2024-03-08T06:00:35.170999+0000","last_became_peered":"2024-03-08T06:00:35.170999+0000","last_unstale":"2024-03-08T06:00:45.428919+0000","last_undegraded":"2024-03-08T06:00:45.428919+0000","last_fullsized":"2024-03-08T06:00:45.428919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252315+0000","last_change":"2024-03-08T06:00:40.222311+0000","last_active":"2024-03-08T06:00:43.252315+0000","last_peered":"2024-03-08T06:00:43.252315+0000","last_clean":"2024-03-08T06:00:43.252315+0000","last_became_active":"2024-03-08T06:00:40.221923+0000","last_became_peered":"2024-03-08T06:00:40.221923+0000","last_unstale":"2024-03-08T06:00:43.252315+0000","last_undegraded":"2024-03-08T06:00:43.252315+0000","last_fullsized":"2024-03-08T06:00:43.252315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:34:22.301392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206554+0000","last_change":"2024-03-08T06:00:36.190623+0000","last_active":"2024-03-08T06:00:42.206554+0000","last_peered":"2024-03-08T06:00:42.206554+0000","last_clean":"2024-03-08T06:00:42.206554+0000","last_became_active":"2024-03-08T06:00:36.190343+0000","last_became_peered":"2024-03-08T06:00:36.190343+0000","last_unstale":"2024-03-08T06:00:42.206554+0000","last_undegraded":"2024-03-08T06:00:42.206554+0000","last_fullsized":"2024-03-08T06:00:42.206554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:39.370412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252587+0000","last_change":"2024-03-08T06:00:35.169188+0000","last_active":"2024-03-08T06:00:43.252587+0000","last_peered":"2024-03-08T06:00:43.252587+0000","last_clean":"2024-03-08T06:00:43.252587+0000","last_became_active":"2024-03-08T06:00:35.168945+0000","last_became_peered":"2024-03-08T06:00:35.168945+0000","last_unstale":"2024-03-08T06:00:43.252587+0000","last_undegraded":"2024-03-08T06:00:43.252587+0000","last_fullsized":"2024-03-08T06:00:43.252587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:58:33.115927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206843+0000","last_change":"2024-03-08T06:00:40.219289+0000","last_active":"2024-03-08T06:00:42.206843+0000","last_peered":"2024-03-08T06:00:42.206843+0000","last_clean":"2024-03-08T06:00:42.206843+0000","last_became_active":"2024-03-08T06:00:40.218831+0000","last_became_peered":"2024-03-08T06:00:40.218831+0000","last_unstale":"2024-03-08T06:00:42.206843+0000","last_undegraded":"2024-03-08T06:00:42.206843+0000","last_fullsized":"2024-03-08T06:00:42.206843+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:08:55.908866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251809+0000","last_change":"2024-03-08T06:00:36.191492+0000","last_active":"2024-03-08T06:00:43.251809+0000","last_peered":"2024-03-08T06:00:43.251809+0000","last_clean":"2024-03-08T06:00:43.251809+0000","last_became_active":"2024-03-08T06:00:36.190895+0000","last_became_peered":"2024-03-08T06:00:36.190895+0000","last_unstale":"2024-03-08T06:00:43.251809+0000","last_undegraded":"2024-03-08T06:00:43.251809+0000","last_fullsized":"2024-03-08T06:00:43.251809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252270+0000","last_change":"2024-03-08T06:00:35.174075+0000","last_active":"2024-03-08T06:00:43.252270+0000","last_peered":"2024-03-08T06:00:43.252270+0000","last_clean":"2024-03-08T06:00:43.252270+0000","last_became_active":"2024-03-08T06:00:35.173803+0000","last_became_peered":"2024-03-08T06:00:35.173803+0000","last_unstale":"2024-03-08T06:00:43.252270+0000","last_undegraded":"2024-03-08T06:00:43.252270+0000","last_fullsized":"2024-03-08T06:00:43.252270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205087+0000","last_change":"2024-03-08T06:00:40.222466+0000","last_active":"2024-03-08T06:00:42.205087+0000","last_peered":"2024-03-08T06:00:42.205087+0000","last_clean":"2024-03-08T06:00:42.205087+0000","last_became_active":"2024-03-08T06:00:40.222083+0000","last_became_peered":"2024-03-08T06:00:40.222083+0000","last_unstale":"2024-03-08T06:00:42.205087+0000","last_undegraded":"2024-03-08T06:00:42.205087+0000","last_fullsized":"2024-03-08T06:00:42.205087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:21:08.958898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252185+0000","last_change":"2024-03-08T06:00:36.191479+0000","last_active":"2024-03-08T06:00:43.252185+0000","last_peered":"2024-03-08T06:00:43.252185+0000","last_clean":"2024-03-08T06:00:43.252185+0000","last_became_active":"2024-03-08T06:00:36.189667+0000","last_became_peered":"2024-03-08T06:00:36.189667+0000","last_unstale":"2024-03-08T06:00:43.252185+0000","last_undegraded":"2024-03-08T06:00:43.252185+0000","last_fullsized":"2024-03-08T06:00:43.252185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:59.082666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253827+0000","last_change":"2024-03-08T06:00:35.173447+0000","last_active":"2024-03-08T06:00:43.253827+0000","last_peered":"2024-03-08T06:00:43.253827+0000","last_clean":"2024-03-08T06:00:43.253827+0000","last_became_active":"2024-03-08T06:00:35.173050+0000","last_became_peered":"2024-03-08T06:00:35.173050+0000","last_unstale":"2024-03-08T06:00:43.253827+0000","last_undegraded":"2024-03-08T06:00:43.253827+0000","last_fullsized":"2024-03-08T06:00:43.253827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:41.059111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205049+0000","last_change":"2024-03-08T06:00:40.772940+0000","last_active":"2024-03-08T06:00:42.205049+0000","last_peered":"2024-03-08T06:00:42.205049+0000","last_clean":"2024-03-08T06:00:42.205049+0000","last_became_active":"2024-03-08T06:00:40.772593+0000","last_became_peered":"2024-03-08T06:00:40.772593+0000","last_unstale":"2024-03-08T06:00:42.205049+0000","last_undegraded":"2024-03-08T06:00:42.205049+0000","last_fullsized":"2024-03-08T06:00:42.205049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:58:58.974013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252132+0000","last_change":"2024-03-08T06:00:40.773346+0000","last_active":"2024-03-08T06:00:43.252132+0000","last_peered":"2024-03-08T06:00:43.252132+0000","last_clean":"2024-03-08T06:00:43.252132+0000","last_became_active":"2024-03-08T06:00:40.772977+0000","last_became_peered":"2024-03-08T06:00:40.772977+0000","last_unstale":"2024-03-08T06:00:43.252132+0000","last_undegraded":"2024-03-08T06:00:43.252132+0000","last_fullsized":"2024-03-08T06:00:43.252132+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:33.029056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251192+0000","last_change":"2024-03-08T06:00:36.182265+0000","last_active":"2024-03-08T06:00:43.251192+0000","last_peered":"2024-03-08T06:00:43.251192+0000","last_clean":"2024-03-08T06:00:43.251192+0000","last_became_active":"2024-03-08T06:00:36.181732+0000","last_became_peered":"2024-03-08T06:00:36.181732+0000","last_unstale":"2024-03-08T06:00:43.251192+0000","last_undegraded":"2024-03-08T06:00:43.251192+0000","last_fullsized":"2024-03-08T06:00:43.251192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:40:16.053974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251308+0000","last_change":"2024-03-08T06:00:35.173204+0000","last_active":"2024-03-08T06:00:43.251308+0000","last_peered":"2024-03-08T06:00:43.251308+0000","last_clean":"2024-03-08T06:00:43.251308+0000","last_became_active":"2024-03-08T06:00:35.172937+0000","last_became_peered":"2024-03-08T06:00:35.172937+0000","last_unstale":"2024-03-08T06:00:43.251308+0000","last_undegraded":"2024-03-08T06:00:43.251308+0000","last_fullsized":"2024-03-08T06:00:43.251308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254582+0000","last_change":"2024-03-08T06:00:40.223795+0000","last_active":"2024-03-08T06:00:43.254582+0000","last_peered":"2024-03-08T06:00:43.254582+0000","last_clean":"2024-03-08T06:00:43.254582+0000","last_became_active":"2024-03-08T06:00:40.223283+0000","last_became_peered":"2024-03-08T06:00:40.223283+0000","last_unstale":"2024-03-08T06:00:43.254582+0000","last_undegraded":"2024-03-08T06:00:43.254582+0000","last_fullsized":"2024-03-08T06:00:43.254582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:09.499486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205033+0000","last_change":"2024-03-08T06:00:36.185555+0000","last_active":"2024-03-08T06:00:42.205033+0000","last_peered":"2024-03-08T06:00:42.205033+0000","last_clean":"2024-03-08T06:00:42.205033+0000","last_became_active":"2024-03-08T06:00:36.185294+0000","last_became_peered":"2024-03-08T06:00:36.185294+0000","last_unstale":"2024-03-08T06:00:42.205033+0000","last_undegraded":"2024-03-08T06:00:42.205033+0000","last_fullsized":"2024-03-08T06:00:42.205033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:13.003781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205103+0000","last_change":"2024-03-08T06:00:35.171146+0000","last_active":"2024-03-08T06:00:42.205103+0000","last_peered":"2024-03-08T06:00:42.205103+0000","last_clean":"2024-03-08T06:00:42.205103+0000","last_became_active":"2024-03-08T06:00:35.170885+0000","last_became_peered":"2024-03-08T06:00:35.170885+0000","last_unstale":"2024-03-08T06:00:42.205103+0000","last_undegraded":"2024-03-08T06:00:42.205103+0000","last_fullsized":"2024-03-08T06:00:42.205103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:59:03.457310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253633+0000","last_change":"2024-03-08T06:00:35.163289+0000","last_active":"2024-03-08T06:00:43.253633+0000","last_peered":"2024-03-08T06:00:43.253633+0000","last_clean":"2024-03-08T06:00:43.253633+0000","last_became_active":"2024-03-08T06:00:35.163005+0000","last_became_peered":"2024-03-08T06:00:35.163005+0000","last_unstale":"2024-03-08T06:00:43.253633+0000","last_undegraded":"2024-03-08T06:00:43.253633+0000","last_fullsized":"2024-03-08T06:00:43.253633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:38.366092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252056+0000","last_change":"2024-03-08T06:00:36.194840+0000","last_active":"2024-03-08T06:00:43.252056+0000","last_peered":"2024-03-08T06:00:43.252056+0000","last_clean":"2024-03-08T06:00:43.252056+0000","last_became_active":"2024-03-08T06:00:36.194547+0000","last_became_peered":"2024-03-08T06:00:36.194547+0000","last_unstale":"2024-03-08T06:00:43.252056+0000","last_undegraded":"2024-03-08T06:00:43.252056+0000","last_fullsized":"2024-03-08T06:00:43.252056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:11:34.950063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253014+0000","last_change":"2024-03-08T06:00:40.219835+0000","last_active":"2024-03-08T06:00:43.253014+0000","last_peered":"2024-03-08T06:00:43.253014+0000","last_clean":"2024-03-08T06:00:43.253014+0000","last_became_active":"2024-03-08T06:00:40.219483+0000","last_became_peered":"2024-03-08T06:00:40.219483+0000","last_unstale":"2024-03-08T06:00:43.253014+0000","last_undegraded":"2024-03-08T06:00:43.253014+0000","last_fullsized":"2024-03-08T06:00:43.253014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:41:06.159304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205357+0000","last_change":"2024-03-08T06:00:35.171304+0000","last_active":"2024-03-08T06:00:42.205357+0000","last_peered":"2024-03-08T06:00:42.205357+0000","last_clean":"2024-03-08T06:00:42.205357+0000","last_became_active":"2024-03-08T06:00:35.170979+0000","last_became_peered":"2024-03-08T06:00:35.170979+0000","last_unstale":"2024-03-08T06:00:42.205357+0000","last_undegraded":"2024-03-08T06:00:42.205357+0000","last_fullsized":"2024-03-08T06:00:42.205357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:41.059111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206410+0000","last_change":"2024-03-08T06:00:36.179352+0000","last_active":"2024-03-08T06:00:42.206410+0000","last_peered":"2024-03-08T06:00:42.206410+0000","last_clean":"2024-03-08T06:00:42.206410+0000","last_became_active":"2024-03-08T06:00:36.179123+0000","last_became_peered":"2024-03-08T06:00:36.179123+0000","last_unstale":"2024-03-08T06:00:42.206410+0000","last_undegraded":"2024-03-08T06:00:42.206410+0000","last_fullsized":"2024-03-08T06:00:42.206410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:12.266989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205576+0000","last_change":"2024-03-08T06:00:40.220569+0000","last_active":"2024-03-08T06:00:42.205576+0000","last_peered":"2024-03-08T06:00:42.205576+0000","last_clean":"2024-03-08T06:00:42.205576+0000","last_became_active":"2024-03-08T06:00:40.220153+0000","last_became_peered":"2024-03-08T06:00:40.220153+0000","last_unstale":"2024-03-08T06:00:42.205576+0000","last_undegraded":"2024-03-08T06:00:42.205576+0000","last_fullsized":"2024-03-08T06:00:42.205576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:44:28.955585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254222+0000","last_change":"2024-03-08T06:00:36.189112+0000","last_active":"2024-03-08T06:00:43.254222+0000","last_peered":"2024-03-08T06:00:43.254222+0000","last_clean":"2024-03-08T06:00:43.254222+0000","last_became_active":"2024-03-08T06:00:36.188650+0000","last_became_peered":"2024-03-08T06:00:36.188650+0000","last_unstale":"2024-03-08T06:00:43.254222+0000","last_undegraded":"2024-03-08T06:00:43.254222+0000","last_fullsized":"2024-03-08T06:00:43.254222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:13.003781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252434+0000","last_change":"2024-03-08T06:00:36.188109+0000","last_active":"2024-03-08T06:00:43.252434+0000","last_peered":"2024-03-08T06:00:43.252434+0000","last_clean":"2024-03-08T06:00:43.252434+0000","last_became_active":"2024-03-08T06:00:36.187798+0000","last_became_peered":"2024-03-08T06:00:36.187798+0000","last_unstale":"2024-03-08T06:00:43.252434+0000","last_undegraded":"2024-03-08T06:00:43.252434+0000","last_fullsized":"2024-03-08T06:00:43.252434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:30:07.063230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251880+0000","last_change":"2024-03-08T06:00:36.193357+0000","last_active":"2024-03-08T06:00:43.251880+0000","last_peered":"2024-03-08T06:00:43.251880+0000","last_clean":"2024-03-08T06:00:43.251880+0000","last_became_active":"2024-03-08T06:00:36.191586+0000","last_became_peered":"2024-03-08T06:00:36.191586+0000","last_unstale":"2024-03-08T06:00:43.251880+0000","last_undegraded":"2024-03-08T06:00:43.251880+0000","last_fullsized":"2024-03-08T06:00:43.251880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:44:44.951176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251957+0000","last_change":"2024-03-08T06:00:36.186266+0000","last_active":"2024-03-08T06:00:43.251957+0000","last_peered":"2024-03-08T06:00:43.251957+0000","last_clean":"2024-03-08T06:00:43.251957+0000","last_became_active":"2024-03-08T06:00:36.185663+0000","last_became_peered":"2024-03-08T06:00:36.185663+0000","last_unstale":"2024-03-08T06:00:43.251957+0000","last_undegraded":"2024-03-08T06:00:43.251957+0000","last_fullsized":"2024-03-08T06:00:43.251957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:53:20.034194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205842+0000","last_change":"2024-03-08T06:00:36.192645+0000","last_active":"2024-03-08T06:00:42.205842+0000","last_peered":"2024-03-08T06:00:42.205842+0000","last_clean":"2024-03-08T06:00:42.205842+0000","last_became_active":"2024-03-08T06:00:36.192391+0000","last_became_peered":"2024-03-08T06:00:36.192391+0000","last_unstale":"2024-03-08T06:00:42.205842+0000","last_undegraded":"2024-03-08T06:00:42.205842+0000","last_fullsized":"2024-03-08T06:00:42.205842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:45:22.429641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252413+0000","last_change":"2024-03-08T06:00:36.181871+0000","last_active":"2024-03-08T06:00:43.252413+0000","last_peered":"2024-03-08T06:00:43.252413+0000","last_clean":"2024-03-08T06:00:43.252413+0000","last_became_active":"2024-03-08T06:00:36.181557+0000","last_became_peered":"2024-03-08T06:00:36.181557+0000","last_unstale":"2024-03-08T06:00:43.252413+0000","last_undegraded":"2024-03-08T06:00:43.252413+0000","last_fullsized":"2024-03-08T06:00:43.252413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:10:17.180499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254922+0000","last_change":"2024-03-08T06:00:36.185046+0000","last_active":"2024-03-08T06:00:43.254922+0000","last_peered":"2024-03-08T06:00:43.254922+0000","last_clean":"2024-03-08T06:00:43.254922+0000","last_became_active":"2024-03-08T06:00:36.184066+0000","last_became_peered":"2024-03-08T06:00:36.184066+0000","last_unstale":"2024-03-08T06:00:43.254922+0000","last_undegraded":"2024-03-08T06:00:43.254922+0000","last_fullsized":"2024-03-08T06:00:43.254922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205846+0000","last_change":"2024-03-08T06:00:36.186595+0000","last_active":"2024-03-08T06:00:42.205846+0000","last_peered":"2024-03-08T06:00:42.205846+0000","last_clean":"2024-03-08T06:00:42.205846+0000","last_became_active":"2024-03-08T06:00:36.186261+0000","last_became_peered":"2024-03-08T06:00:36.186261+0000","last_unstale":"2024-03-08T06:00:42.205846+0000","last_undegraded":"2024-03-08T06:00:42.205846+0000","last_fullsized":"2024-03-08T06:00:42.205846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:16:14.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251609+0000","last_change":"2024-03-08T06:00:36.186618+0000","last_active":"2024-03-08T06:00:43.251609+0000","last_peered":"2024-03-08T06:00:43.251609+0000","last_clean":"2024-03-08T06:00:43.251609+0000","last_became_active":"2024-03-08T06:00:36.186068+0000","last_became_peered":"2024-03-08T06:00:36.186068+0000","last_unstale":"2024-03-08T06:00:43.251609+0000","last_undegraded":"2024-03-08T06:00:43.251609+0000","last_fullsized":"2024-03-08T06:00:43.251609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:39:59.986300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205158+0000","last_change":"2024-03-08T06:00:36.186759+0000","last_active":"2024-03-08T06:00:42.205158+0000","last_peered":"2024-03-08T06:00:42.205158+0000","last_clean":"2024-03-08T06:00:42.205158+0000","last_became_active":"2024-03-08T06:00:36.186338+0000","last_became_peered":"2024-03-08T06:00:36.186338+0000","last_unstale":"2024-03-08T06:00:42.205158+0000","last_undegraded":"2024-03-08T06:00:42.205158+0000","last_fullsized":"2024-03-08T06:00:42.205158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:40:16.053974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251835+0000","last_change":"2024-03-08T06:00:36.197002+0000","last_active":"2024-03-08T06:00:43.251835+0000","last_peered":"2024-03-08T06:00:43.251835+0000","last_clean":"2024-03-08T06:00:43.251835+0000","last_became_active":"2024-03-08T06:00:36.196750+0000","last_became_peered":"2024-03-08T06:00:36.196750+0000","last_unstale":"2024-03-08T06:00:43.251835+0000","last_undegraded":"2024-03-08T06:00:43.251835+0000","last_fullsized":"2024-03-08T06:00:43.251835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:45:22.429641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251708+0000","last_change":"2024-03-08T06:00:36.184956+0000","last_active":"2024-03-08T06:00:43.251708+0000","last_peered":"2024-03-08T06:00:43.251708+0000","last_clean":"2024-03-08T06:00:43.251708+0000","last_became_active":"2024-03-08T06:00:36.184694+0000","last_became_peered":"2024-03-08T06:00:36.184694+0000","last_unstale":"2024-03-08T06:00:43.251708+0000","last_undegraded":"2024-03-08T06:00:43.251708+0000","last_fullsized":"2024-03-08T06:00:43.251708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:41:02.118287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254526+0000","last_change":"2024-03-08T06:00:36.187667+0000","last_active":"2024-03-08T06:00:43.254526+0000","last_peered":"2024-03-08T06:00:43.254526+0000","last_clean":"2024-03-08T06:00:43.254526+0000","last_became_active":"2024-03-08T06:00:36.187407+0000","last_became_peered":"2024-03-08T06:00:36.187407+0000","last_unstale":"2024-03-08T06:00:43.254526+0000","last_undegraded":"2024-03-08T06:00:43.254526+0000","last_fullsized":"2024-03-08T06:00:43.254526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:10:17.180499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752121+0000","last_change":"2024-03-08T06:00:36.198182+0000","last_active":"2024-03-08T06:00:42.752121+0000","last_peered":"2024-03-08T06:00:42.752121+0000","last_clean":"2024-03-08T06:00:42.752121+0000","last_became_active":"2024-03-08T06:00:36.197894+0000","last_became_peered":"2024-03-08T06:00:36.197894+0000","last_unstale":"2024-03-08T06:00:42.752121+0000","last_undegraded":"2024-03-08T06:00:42.752121+0000","last_fullsized":"2024-03-08T06:00:42.752121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:02.331109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253403+0000","last_change":"2024-03-08T06:00:36.180664+0000","last_active":"2024-03-08T06:00:43.253403+0000","last_peered":"2024-03-08T06:00:43.253403+0000","last_clean":"2024-03-08T06:00:43.253403+0000","last_became_active":"2024-03-08T06:00:36.180389+0000","last_became_peered":"2024-03-08T06:00:36.180389+0000","last_unstale":"2024-03-08T06:00:43.253403+0000","last_undegraded":"2024-03-08T06:00:43.253403+0000","last_fullsized":"2024-03-08T06:00:43.253403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:58:34.120247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252430+0000","last_change":"2024-03-08T06:00:36.189412+0000","last_active":"2024-03-08T06:00:43.252430+0000","last_peered":"2024-03-08T06:00:43.252430+0000","last_clean":"2024-03-08T06:00:43.252430+0000","last_became_active":"2024-03-08T06:00:36.187130+0000","last_became_peered":"2024-03-08T06:00:36.187130+0000","last_unstale":"2024-03-08T06:00:43.252430+0000","last_undegraded":"2024-03-08T06:00:43.252430+0000","last_fullsized":"2024-03-08T06:00:43.252430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:05.458469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251511+0000","last_change":"2024-03-08T06:00:36.186949+0000","last_active":"2024-03-08T06:00:43.251511+0000","last_peered":"2024-03-08T06:00:43.251511+0000","last_clean":"2024-03-08T06:00:43.251511+0000","last_became_active":"2024-03-08T06:00:36.186386+0000","last_became_peered":"2024-03-08T06:00:36.186386+0000","last_unstale":"2024-03-08T06:00:43.251511+0000","last_undegraded":"2024-03-08T06:00:43.251511+0000","last_fullsized":"2024-03-08T06:00:43.251511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251280+0000","last_change":"2024-03-08T06:00:36.178773+0000","last_active":"2024-03-08T06:00:43.251280+0000","last_peered":"2024-03-08T06:00:43.251280+0000","last_clean":"2024-03-08T06:00:43.251280+0000","last_became_active":"2024-03-08T06:00:36.178515+0000","last_became_peered":"2024-03-08T06:00:36.178515+0000","last_unstale":"2024-03-08T06:00:43.251280+0000","last_undegraded":"2024-03-08T06:00:43.251280+0000","last_fullsized":"2024-03-08T06:00:43.251280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:11:34.950063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251635+0000","last_change":"2024-03-08T06:00:36.179334+0000","last_active":"2024-03-08T06:00:43.251635+0000","last_peered":"2024-03-08T06:00:43.251635+0000","last_clean":"2024-03-08T06:00:43.251635+0000","last_became_active":"2024-03-08T06:00:36.179056+0000","last_became_peered":"2024-03-08T06:00:36.179056+0000","last_unstale":"2024-03-08T06:00:43.251635+0000","last_undegraded":"2024-03-08T06:00:43.251635+0000","last_fullsized":"2024-03-08T06:00:43.251635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:13.003781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205268+0000","last_change":"2024-03-08T06:00:36.182864+0000","last_active":"2024-03-08T06:00:42.205268+0000","last_peered":"2024-03-08T06:00:42.205268+0000","last_clean":"2024-03-08T06:00:42.205268+0000","last_became_active":"2024-03-08T06:00:36.182290+0000","last_became_peered":"2024-03-08T06:00:36.182290+0000","last_unstale":"2024-03-08T06:00:42.205268+0000","last_undegraded":"2024-03-08T06:00:42.205268+0000","last_fullsized":"2024-03-08T06:00:42.205268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206446+0000","last_change":"2024-03-08T06:00:36.184044+0000","last_active":"2024-03-08T06:00:42.206446+0000","last_peered":"2024-03-08T06:00:42.206446+0000","last_clean":"2024-03-08T06:00:42.206446+0000","last_became_active":"2024-03-08T06:00:36.183745+0000","last_became_peered":"2024-03-08T06:00:36.183745+0000","last_unstale":"2024-03-08T06:00:42.206446+0000","last_undegraded":"2024-03-08T06:00:42.206446+0000","last_fullsized":"2024-03-08T06:00:42.206446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:58:34.120247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254652+0000","last_change":"2024-03-08T06:00:36.200379+0000","last_active":"2024-03-08T06:00:43.254652+0000","last_peered":"2024-03-08T06:00:43.254652+0000","last_clean":"2024-03-08T06:00:43.254652+0000","last_became_active":"2024-03-08T06:00:36.200023+0000","last_became_peered":"2024-03-08T06:00:36.200023+0000","last_unstale":"2024-03-08T06:00:43.254652+0000","last_undegraded":"2024-03-08T06:00:43.254652+0000","last_fullsized":"2024-03-08T06:00:43.254652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:40:16.053974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254844+0000","last_change":"2024-03-08T06:00:36.200329+0000","last_active":"2024-03-08T06:00:43.254844+0000","last_peered":"2024-03-08T06:00:43.254844+0000","last_clean":"2024-03-08T06:00:43.254844+0000","last_became_active":"2024-03-08T06:00:36.200069+0000","last_became_peered":"2024-03-08T06:00:36.200069+0000","last_unstale":"2024-03-08T06:00:43.254844+0000","last_undegraded":"2024-03-08T06:00:43.254844+0000","last_fullsized":"2024-03-08T06:00:43.254844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751472+0000","last_change":"2024-03-08T06:00:36.205202+0000","last_active":"2024-03-08T06:00:42.751472+0000","last_peered":"2024-03-08T06:00:42.751472+0000","last_clean":"2024-03-08T06:00:42.751472+0000","last_became_active":"2024-03-08T06:00:36.204866+0000","last_became_peered":"2024-03-08T06:00:36.204866+0000","last_unstale":"2024-03-08T06:00:42.751472+0000","last_undegraded":"2024-03-08T06:00:42.751472+0000","last_fullsized":"2024-03-08T06:00:42.751472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:44:44.951176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.204931+0000","last_change":"2024-03-08T06:00:36.188976+0000","last_active":"2024-03-08T06:00:42.204931+0000","last_peered":"2024-03-08T06:00:42.204931+0000","last_clean":"2024-03-08T06:00:42.204931+0000","last_became_active":"2024-03-08T06:00:36.188717+0000","last_became_peered":"2024-03-08T06:00:36.188717+0000","last_unstale":"2024-03-08T06:00:42.204931+0000","last_undegraded":"2024-03-08T06:00:42.204931+0000","last_fullsized":"2024-03-08T06:00:42.204931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:11:34.950063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751727+0000","last_change":"2024-03-08T06:00:36.197547+0000","last_active":"2024-03-08T06:00:42.751727+0000","last_peered":"2024-03-08T06:00:42.751727+0000","last_clean":"2024-03-08T06:00:42.751727+0000","last_became_active":"2024-03-08T06:00:36.197295+0000","last_became_peered":"2024-03-08T06:00:36.197295+0000","last_unstale":"2024-03-08T06:00:42.751727+0000","last_undegraded":"2024-03-08T06:00:42.751727+0000","last_fullsized":"2024-03-08T06:00:42.751727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:39:59.986300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251773+0000","last_change":"2024-03-08T06:00:36.191642+0000","last_active":"2024-03-08T06:00:43.251773+0000","last_peered":"2024-03-08T06:00:43.251773+0000","last_clean":"2024-03-08T06:00:43.251773+0000","last_became_active":"2024-03-08T06:00:36.191242+0000","last_became_peered":"2024-03-08T06:00:36.191242+0000","last_unstale":"2024-03-08T06:00:43.251773+0000","last_undegraded":"2024-03-08T06:00:43.251773+0000","last_fullsized":"2024-03-08T06:00:43.251773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751820+0000","last_change":"2024-03-08T06:00:36.197506+0000","last_active":"2024-03-08T06:00:42.751820+0000","last_peered":"2024-03-08T06:00:42.751820+0000","last_clean":"2024-03-08T06:00:42.751820+0000","last_became_active":"2024-03-08T06:00:36.197208+0000","last_became_peered":"2024-03-08T06:00:36.197208+0000","last_unstale":"2024-03-08T06:00:42.751820+0000","last_undegraded":"2024-03-08T06:00:42.751820+0000","last_fullsized":"2024-03-08T06:00:42.751820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:59:39.518589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252005+0000","last_change":"2024-03-08T06:00:36.186669+0000","last_active":"2024-03-08T06:00:43.252005+0000","last_peered":"2024-03-08T06:00:43.252005+0000","last_clean":"2024-03-08T06:00:43.252005+0000","last_became_active":"2024-03-08T06:00:36.186417+0000","last_became_peered":"2024-03-08T06:00:36.186417+0000","last_unstale":"2024-03-08T06:00:43.252005+0000","last_undegraded":"2024-03-08T06:00:43.252005+0000","last_fullsized":"2024-03-08T06:00:43.252005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:59:39.518589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251432+0000","last_change":"2024-03-08T06:00:36.186124+0000","last_active":"2024-03-08T06:00:43.251432+0000","last_peered":"2024-03-08T06:00:43.251432+0000","last_clean":"2024-03-08T06:00:43.251432+0000","last_became_active":"2024-03-08T06:00:36.185808+0000","last_became_peered":"2024-03-08T06:00:36.185808+0000","last_unstale":"2024-03-08T06:00:43.251432+0000","last_undegraded":"2024-03-08T06:00:43.251432+0000","last_fullsized":"2024-03-08T06:00:43.251432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:01:37.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252630+0000","last_change":"2024-03-08T06:00:36.183150+0000","last_active":"2024-03-08T06:00:43.252630+0000","last_peered":"2024-03-08T06:00:43.252630+0000","last_clean":"2024-03-08T06:00:43.252630+0000","last_became_active":"2024-03-08T06:00:36.182716+0000","last_became_peered":"2024-03-08T06:00:36.182716+0000","last_unstale":"2024-03-08T06:00:43.252630+0000","last_undegraded":"2024-03-08T06:00:43.252630+0000","last_fullsized":"2024-03-08T06:00:43.252630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752346+0000","last_change":"2024-03-08T06:00:36.198478+0000","last_active":"2024-03-08T06:00:42.752346+0000","last_peered":"2024-03-08T06:00:42.752346+0000","last_clean":"2024-03-08T06:00:42.752346+0000","last_became_active":"2024-03-08T06:00:36.198218+0000","last_became_peered":"2024-03-08T06:00:36.198218+0000","last_unstale":"2024-03-08T06:00:42.752346+0000","last_undegraded":"2024-03-08T06:00:42.752346+0000","last_fullsized":"2024-03-08T06:00:42.752346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:19:26.058130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":876544,"data_stored":703400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":116292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":112540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":112576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":116292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":112450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":112610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":703456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:00:59.217 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:00:59.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.350+0000 7f7d16cc3640 1 Processor -- start 2024-03-08T06:00:59.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.350+0000 7f7d16cc3640 1 -- start start 2024-03-08T06:00:59.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.351+0000 7f7d16cc3640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7d100ceee0 0x7f7d1015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.351+0000 7f7d16cc3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7d100cdc50 0x7f7d100cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.351+0000 7f7d16cc3640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7d100ce4f0 0x7f7d1015e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.351+0000 7f7d16cc3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1015bea0 con 0x7f7d100cdc50 2024-03-08T06:00:59.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.351+0000 7f7d16cc3640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d100cf800 con 0x7f7d100ce4f0 2024-03-08T06:00:59.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.351+0000 7f7d16cc3640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1015b570 con 0x7f7d100ceee0 2024-03-08T06:00:59.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.351+0000 7f7d0ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7d100cdc50 0x7f7d100cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.351+0000 7f7d0ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7d100cdc50 0x7f7d100cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51228/0 (socket says 172.21.15.76:51228) 2024-03-08T06:00:59.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.351+0000 7f7d0ffff640 1 -- 172.21.15.76:0/3219683013 learned_addr learned my addr 172.21.15.76:0/3219683013 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:59.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.351+0000 7f7d15260640 1 --2- 172.21.15.76:0/3219683013 >> v2:172.21.15.81:3300/0 conn(0x7f7d100ce4f0 0x7f7d1015e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.352+0000 7f7d14a5f640 1 --2- 172.21.15.76:0/3219683013 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7d100ceee0 0x7f7d1015aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.352+0000 7f7d0ffff640 1 -- 172.21.15.76:0/3219683013 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7d100ceee0 msgr2=0x7f7d1015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:59.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.352+0000 7f7d0ffff640 1 --2- 172.21.15.76:0/3219683013 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7d100ceee0 0x7f7d1015aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.352+0000 7f7d0ffff640 1 -- 172.21.15.76:0/3219683013 >> v2:172.21.15.81:3300/0 conn(0x7f7d100ce4f0 msgr2=0x7f7d1015e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:00:59.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.352+0000 7f7d0ffff640 1 --2- 172.21.15.76:0/3219683013 >> v2:172.21.15.81:3300/0 conn(0x7f7d100ce4f0 0x7f7d1015e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.352+0000 7f7d0ffff640 1 -- 172.21.15.76:0/3219683013 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1015b850 con 0x7f7d100cdc50 2024-03-08T06:00:59.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.352+0000 7f7d15260640 1 --2- 172.21.15.76:0/3219683013 >> v2:172.21.15.81:3300/0 conn(0x7f7d100ce4f0 0x7f7d1015e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:00:59.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.352+0000 7f7d0ffff640 1 --2- 172.21.15.76:0/3219683013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d100cdc50 0x7f7d100cf2c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7d10072780 tx=0x7f7d0402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fd7c4ca5e4c34d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:59.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.352+0000 7f7d0f7fe640 1 -- 172.21.15.76:0/3219683013 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0403b070 con 0x7f7d100cdc50 2024-03-08T06:00:59.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.353+0000 7f7d0f7fe640 1 -- 172.21.15.76:0/3219683013 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d04036440 con 0x7f7d100cdc50 2024-03-08T06:00:59.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.353+0000 7f7d0f7fe640 1 -- 172.21.15.76:0/3219683013 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04036810 con 0x7f7d100cdc50 2024-03-08T06:00:59.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.353+0000 7f7d16cc3640 1 -- 172.21.15.76:0/3219683013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d100cdc50 msgr2=0x7f7d100cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.353+0000 7f7d16cc3640 1 --2- 172.21.15.76:0/3219683013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d100cdc50 0x7f7d100cf2c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7d10072780 tx=0x7f7d0402e0e0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.353+0000 7f7d16cc3640 1 -- 172.21.15.76:0/3219683013 shutdown_connections 2024-03-08T06:00:59.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.353+0000 7f7d16cc3640 1 --2- 172.21.15.76:0/3219683013 >> v2:172.21.15.81:3300/0 conn(0x7f7d100ce4f0 0x7f7d1015e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.353+0000 7f7d16cc3640 1 --2- 172.21.15.76:0/3219683013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d100cdc50 0x7f7d100cf2c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.353+0000 7f7d16cc3640 1 --2- 172.21.15.76:0/3219683013 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7d100ceee0 0x7f7d1015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.353+0000 7f7d16cc3640 1 -- 172.21.15.76:0/3219683013 >> 172.21.15.76:0/3219683013 conn(0x7f7d10099a60 msgr2=0x7f7d100b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.353+0000 7f7d16cc3640 1 -- 172.21.15.76:0/3219683013 shutdown_connections 2024-03-08T06:00:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.353+0000 7f7d16cc3640 1 -- 172.21.15.76:0/3219683013 wait complete. 2024-03-08T06:00:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.354+0000 7f7d16cc3640 1 Processor -- start 2024-03-08T06:00:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.354+0000 7f7d16cc3640 1 -- start start 2024-03-08T06:00:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.355+0000 7f7d16cc3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d100cdc50 0x7f7d100c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.355+0000 7f7d16cc3640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7d100ce4f0 0x7f7d10152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.355+0000 7f7d16cc3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d100ceee0 0x7f7d1006f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.355+0000 7f7d16cc3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d100ceb00 con 0x7f7d100ceee0 2024-03-08T06:00:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.355+0000 7f7d16cc3640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d100ce980 con 0x7f7d100cdc50 2024-03-08T06:00:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.355+0000 7f7d16cc3640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1015ecd0 con 0x7f7d100ce4f0 2024-03-08T06:00:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.355+0000 7f7d15260640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d100ceee0 0x7f7d1006f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.355+0000 7f7d15260640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d100ceee0 0x7f7d1006f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51238/0 (socket says 172.21.15.76:51238) 2024-03-08T06:00:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.355+0000 7f7d15260640 1 -- 172.21.15.76:0/149594837 learned_addr learned my addr 172.21.15.76:0/149594837 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.355+0000 7f7d0ffff640 1 --2- 172.21.15.76:0/149594837 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7d100ce4f0 0x7f7d10152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.355+0000 7f7d14a5f640 1 --2- 172.21.15.76:0/149594837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d100cdc50 0x7f7d100c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.356+0000 7f7d15260640 1 -- 172.21.15.76:0/149594837 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7d100ce4f0 msgr2=0x7f7d10152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.356+0000 7f7d15260640 1 --2- 172.21.15.76:0/149594837 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7d100ce4f0 0x7f7d10152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.356+0000 7f7d15260640 1 -- 172.21.15.76:0/149594837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d100cdc50 msgr2=0x7f7d100c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.356+0000 7f7d15260640 1 --2- 172.21.15.76:0/149594837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d100cdc50 0x7f7d100c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.356+0000 7f7d15260640 1 -- 172.21.15.76:0/149594837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d100b38f0 con 0x7f7d100ceee0 2024-03-08T06:00:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.356+0000 7f7d15260640 1 --2- 172.21.15.76:0/149594837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d100ceee0 0x7f7d1006f210 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7d08007e10 tx=0x7f7d0800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.356+0000 7f7d0d7fa640 1 -- 172.21.15.76:0/149594837 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0801a070 con 0x7f7d100ceee0 2024-03-08T06:00:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.356+0000 7f7d0d7fa640 1 -- 172.21.15.76:0/149594837 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d08003b90 con 0x7f7d100ceee0 2024-03-08T06:00:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.356+0000 7f7d16cc3640 1 -- 172.21.15.76:0/149594837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d100b3bd0 con 0x7f7d100ceee0 2024-03-08T06:00:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.356+0000 7f7d0d7fa640 1 -- 172.21.15.76:0/149594837 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08004030 con 0x7f7d100ceee0 2024-03-08T06:00:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.356+0000 7f7d16cc3640 1 -- 172.21.15.76:0/149594837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d100b4190 con 0x7f7d100ceee0 2024-03-08T06:00:59.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.357+0000 7f7d16cc3640 1 -- 172.21.15.76:0/149594837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1009a2c0 con 0x7f7d100ceee0 2024-03-08T06:00:59.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.357+0000 7f7d0d7fa640 1 -- 172.21.15.76:0/149594837 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d0801d070 con 0x7f7d100ceee0 2024-03-08T06:00:59.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.358+0000 7f7d0d7fa640 1 --2- 172.21.15.76:0/149594837 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7cfc06b4d0 0x7f7cfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.358+0000 7f7d0d7fa640 1 -- 172.21.15.76:0/149594837 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7d0808a700 con 0x7f7d100ceee0 2024-03-08T06:00:59.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.358+0000 7f7d14a5f640 1 --2- 172.21.15.76:0/149594837 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7cfc06b4d0 0x7f7cfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.358+0000 7f7d14a5f640 1 --2- 172.21.15.76:0/149594837 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7cfc06b4d0 0x7f7cfc06d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7d000079a0 tx=0x7f7d00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:59.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.362+0000 7f7d0d7fa640 1 -- 172.21.15.76:0/149594837 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d1009a2c0 con 0x7f7d100ceee0 2024-03-08T06:00:59.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.528+0000 7f7d16cc3640 1 -- 172.21.15.76:0/149594837 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d1004f030 con 0x7f7cfc06b4d0 2024-03-08T06:00:59.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.536+0000 7f7d0d7fa640 1 -- 172.21.15.76:0/149594837 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f7d1004f030 con 0x7f7cfc06b4d0 2024-03-08T06:00:59.540 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:59.542 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-08T06:00:59.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.539+0000 7f7d16cc3640 1 -- 172.21.15.76:0/149594837 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7cfc06b4d0 msgr2=0x7f7cfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.539+0000 7f7d16cc3640 1 --2- 172.21.15.76:0/149594837 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7cfc06b4d0 0x7f7cfc06d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7d000079a0 tx=0x7f7d00008040 comp rx=0 tx=0).stop 2024-03-08T06:00:59.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.539+0000 7f7d16cc3640 1 -- 172.21.15.76:0/149594837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d100ceee0 msgr2=0x7f7d1006f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.539+0000 7f7d16cc3640 1 --2- 172.21.15.76:0/149594837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d100ceee0 0x7f7d1006f210 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7d08007e10 tx=0x7f7d0800d4a0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.540+0000 7f7d16cc3640 1 -- 172.21.15.76:0/149594837 shutdown_connections 2024-03-08T06:00:59.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.540+0000 7f7d16cc3640 1 --2- 172.21.15.76:0/149594837 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7cfc06b4d0 0x7f7cfc06d990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.540+0000 7f7d16cc3640 1 --2- 172.21.15.76:0/149594837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d100ceee0 0x7f7d1006f210 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.540+0000 7f7d16cc3640 1 --2- 172.21.15.76:0/149594837 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7d100ce4f0 0x7f7d10152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.540+0000 7f7d16cc3640 1 --2- 172.21.15.76:0/149594837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d100cdc50 0x7f7d100c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.540+0000 7f7d16cc3640 1 -- 172.21.15.76:0/149594837 >> 172.21.15.76:0/149594837 conn(0x7f7d10099a60 msgr2=0x7f7d1015a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:59.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.540+0000 7f7d16cc3640 1 -- 172.21.15.76:0/149594837 shutdown_connections 2024-03-08T06:00:59.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.540+0000 7f7d16cc3640 1 -- 172.21.15.76:0/149594837 wait complete. 2024-03-08T06:00:59.551 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-03-08T06:00:57.925087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219560,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755160,"statfs":{"total":773094113280,"available":772869283840,"internally_reserved":0,"allocated":3481600,"data_stored":2089616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003253"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.250953+0000","last_change":"2024-03-08T06:00:35.165081+0000","last_active":"2024-03-08T06:00:43.250953+0000","last_peered":"2024-03-08T06:00:43.250953+0000","last_clean":"2024-03-08T06:00:43.250953+0000","last_became_active":"2024-03-08T06:00:35.164818+0000","last_became_peered":"2024-03-08T06:00:35.164818+0000","last_unstale":"2024-03-08T06:00:43.250953+0000","last_undegraded":"2024-03-08T06:00:43.250953+0000","last_fullsized":"2024-03-08T06:00:43.250953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:59:03.457310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205755+0000","last_change":"2024-03-08T06:00:36.182716+0000","last_active":"2024-03-08T06:00:42.205755+0000","last_peered":"2024-03-08T06:00:42.205755+0000","last_clean":"2024-03-08T06:00:42.205755+0000","last_became_active":"2024-03-08T06:00:36.182377+0000","last_became_peered":"2024-03-08T06:00:36.182377+0000","last_unstale":"2024-03-08T06:00:42.205755+0000","last_undegraded":"2024-03-08T06:00:42.205755+0000","last_fullsized":"2024-03-08T06:00:42.205755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:59:39.518589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.204982+0000","last_change":"2024-03-08T06:00:40.773175+0000","last_active":"2024-03-08T06:00:42.204982+0000","last_peered":"2024-03-08T06:00:42.204982+0000","last_clean":"2024-03-08T06:00:42.204982+0000","last_became_active":"2024-03-08T06:00:40.772777+0000","last_became_peered":"2024-03-08T06:00:40.772777+0000","last_unstale":"2024-03-08T06:00:42.204982+0000","last_undegraded":"2024-03-08T06:00:42.204982+0000","last_fullsized":"2024-03-08T06:00:42.204982+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:51:44.068190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205870+0000","last_change":"2024-03-08T06:00:35.177115+0000","last_active":"2024-03-08T06:00:42.205870+0000","last_peered":"2024-03-08T06:00:42.205870+0000","last_clean":"2024-03-08T06:00:42.205870+0000","last_became_active":"2024-03-08T06:00:35.176827+0000","last_became_peered":"2024-03-08T06:00:35.176827+0000","last_unstale":"2024-03-08T06:00:42.205870+0000","last_undegraded":"2024-03-08T06:00:42.205870+0000","last_fullsized":"2024-03-08T06:00:42.205870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:57:55.324372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251714+0000","last_change":"2024-03-08T06:00:36.191516+0000","last_active":"2024-03-08T06:00:43.251714+0000","last_peered":"2024-03-08T06:00:43.251714+0000","last_clean":"2024-03-08T06:00:43.251714+0000","last_became_active":"2024-03-08T06:00:36.191258+0000","last_became_peered":"2024-03-08T06:00:36.191258+0000","last_unstale":"2024-03-08T06:00:43.251714+0000","last_undegraded":"2024-03-08T06:00:43.251714+0000","last_fullsized":"2024-03-08T06:00:43.251714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206166+0000","last_change":"2024-03-08T06:00:40.219312+0000","last_active":"2024-03-08T06:00:42.206166+0000","last_peered":"2024-03-08T06:00:42.206166+0000","last_clean":"2024-03-08T06:00:42.206166+0000","last_became_active":"2024-03-08T06:00:40.218891+0000","last_became_peered":"2024-03-08T06:00:40.218891+0000","last_unstale":"2024-03-08T06:00:42.206166+0000","last_undegraded":"2024-03-08T06:00:42.206166+0000","last_fullsized":"2024-03-08T06:00:42.206166+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:19:30.099147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752093+0000","last_change":"2024-03-08T06:00:35.163380+0000","last_active":"2024-03-08T06:00:42.752093+0000","last_peered":"2024-03-08T06:00:42.752093+0000","last_clean":"2024-03-08T06:00:42.752093+0000","last_became_active":"2024-03-08T06:00:35.163123+0000","last_became_peered":"2024-03-08T06:00:35.163123+0000","last_unstale":"2024-03-08T06:00:42.752093+0000","last_undegraded":"2024-03-08T06:00:42.752093+0000","last_fullsized":"2024-03-08T06:00:42.752093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252109+0000","last_change":"2024-03-08T06:00:36.194135+0000","last_active":"2024-03-08T06:00:43.252109+0000","last_peered":"2024-03-08T06:00:43.252109+0000","last_clean":"2024-03-08T06:00:43.252109+0000","last_became_active":"2024-03-08T06:00:36.193870+0000","last_became_peered":"2024-03-08T06:00:36.193870+0000","last_unstale":"2024-03-08T06:00:43.252109+0000","last_undegraded":"2024-03-08T06:00:43.252109+0000","last_fullsized":"2024-03-08T06:00:43.252109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:02:15.458747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206553+0000","last_change":"2024-03-08T06:00:40.221366+0000","last_active":"2024-03-08T06:00:42.206553+0000","last_peered":"2024-03-08T06:00:42.206553+0000","last_clean":"2024-03-08T06:00:42.206553+0000","last_became_active":"2024-03-08T06:00:40.220967+0000","last_became_peered":"2024-03-08T06:00:40.220967+0000","last_unstale":"2024-03-08T06:00:42.206553+0000","last_undegraded":"2024-03-08T06:00:42.206553+0000","last_fullsized":"2024-03-08T06:00:42.206553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:16:51.281060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751868+0000","last_change":"2024-03-08T06:00:35.173354+0000","last_active":"2024-03-08T06:00:42.751868+0000","last_peered":"2024-03-08T06:00:42.751868+0000","last_clean":"2024-03-08T06:00:42.751868+0000","last_became_active":"2024-03-08T06:00:35.173105+0000","last_became_peered":"2024-03-08T06:00:35.173105+0000","last_unstale":"2024-03-08T06:00:42.751868+0000","last_undegraded":"2024-03-08T06:00:42.751868+0000","last_fullsized":"2024-03-08T06:00:42.751868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252178+0000","last_change":"2024-03-08T06:00:36.193424+0000","last_active":"2024-03-08T06:00:43.252178+0000","last_peered":"2024-03-08T06:00:43.252178+0000","last_clean":"2024-03-08T06:00:43.252178+0000","last_became_active":"2024-03-08T06:00:36.191212+0000","last_became_peered":"2024-03-08T06:00:36.191212+0000","last_unstale":"2024-03-08T06:00:43.252178+0000","last_undegraded":"2024-03-08T06:00:43.252178+0000","last_fullsized":"2024-03-08T06:00:43.252178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:02.331109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206122+0000","last_change":"2024-03-08T06:00:40.221344+0000","last_active":"2024-03-08T06:00:42.206122+0000","last_peered":"2024-03-08T06:00:42.206122+0000","last_clean":"2024-03-08T06:00:42.206122+0000","last_became_active":"2024-03-08T06:00:40.220925+0000","last_became_peered":"2024-03-08T06:00:40.220925+0000","last_unstale":"2024-03-08T06:00:42.206122+0000","last_undegraded":"2024-03-08T06:00:42.206122+0000","last_fullsized":"2024-03-08T06:00:42.206122+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:06.372126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254602+0000","last_change":"2024-03-08T06:00:36.186043+0000","last_active":"2024-03-08T06:00:43.254602+0000","last_peered":"2024-03-08T06:00:43.254602+0000","last_clean":"2024-03-08T06:00:43.254602+0000","last_became_active":"2024-03-08T06:00:36.185793+0000","last_became_peered":"2024-03-08T06:00:36.185793+0000","last_unstale":"2024-03-08T06:00:43.254602+0000","last_undegraded":"2024-03-08T06:00:43.254602+0000","last_fullsized":"2024-03-08T06:00:43.254602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:01:37.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751980+0000","last_change":"2024-03-08T06:00:35.163406+0000","last_active":"2024-03-08T06:00:42.751980+0000","last_peered":"2024-03-08T06:00:42.751980+0000","last_clean":"2024-03-08T06:00:42.751980+0000","last_became_active":"2024-03-08T06:00:35.163138+0000","last_became_peered":"2024-03-08T06:00:35.163138+0000","last_unstale":"2024-03-08T06:00:42.751980+0000","last_undegraded":"2024-03-08T06:00:42.751980+0000","last_fullsized":"2024-03-08T06:00:42.751980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:38.366092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206808+0000","last_change":"2024-03-08T06:00:40.219654+0000","last_active":"2024-03-08T06:00:42.206808+0000","last_peered":"2024-03-08T06:00:42.206808+0000","last_clean":"2024-03-08T06:00:42.206808+0000","last_became_active":"2024-03-08T06:00:40.219225+0000","last_became_peered":"2024-03-08T06:00:40.219225+0000","last_unstale":"2024-03-08T06:00:42.206808+0000","last_undegraded":"2024-03-08T06:00:42.206808+0000","last_fullsized":"2024-03-08T06:00:42.206808+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:34:22.301392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205304+0000","last_change":"2024-03-08T06:00:36.184886+0000","last_active":"2024-03-08T06:00:42.205304+0000","last_peered":"2024-03-08T06:00:42.205304+0000","last_clean":"2024-03-08T06:00:42.205304+0000","last_became_active":"2024-03-08T06:00:36.184582+0000","last_became_peered":"2024-03-08T06:00:36.184582+0000","last_unstale":"2024-03-08T06:00:42.205304+0000","last_undegraded":"2024-03-08T06:00:42.205304+0000","last_fullsized":"2024-03-08T06:00:42.205304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251830+0000","last_change":"2024-03-08T06:00:35.171210+0000","last_active":"2024-03-08T06:00:43.251830+0000","last_peered":"2024-03-08T06:00:43.251830+0000","last_clean":"2024-03-08T06:00:43.251830+0000","last_became_active":"2024-03-08T06:00:35.170784+0000","last_became_peered":"2024-03-08T06:00:35.170784+0000","last_unstale":"2024-03-08T06:00:43.251830+0000","last_undegraded":"2024-03-08T06:00:43.251830+0000","last_fullsized":"2024-03-08T06:00:43.251830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206300+0000","last_change":"2024-03-08T06:00:40.221460+0000","last_active":"2024-03-08T06:00:42.206300+0000","last_peered":"2024-03-08T06:00:42.206300+0000","last_clean":"2024-03-08T06:00:42.206300+0000","last_became_active":"2024-03-08T06:00:40.221046+0000","last_became_peered":"2024-03-08T06:00:40.221046+0000","last_unstale":"2024-03-08T06:00:42.206300+0000","last_undegraded":"2024-03-08T06:00:42.206300+0000","last_fullsized":"2024-03-08T06:00:42.206300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:35.686997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205125+0000","last_change":"2024-03-08T06:00:36.181765+0000","last_active":"2024-03-08T06:00:42.205125+0000","last_peered":"2024-03-08T06:00:42.205125+0000","last_clean":"2024-03-08T06:00:42.205125+0000","last_became_active":"2024-03-08T06:00:36.181503+0000","last_became_peered":"2024-03-08T06:00:36.181503+0000","last_unstale":"2024-03-08T06:00:42.205125+0000","last_undegraded":"2024-03-08T06:00:42.205125+0000","last_fullsized":"2024-03-08T06:00:42.205125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:01:37.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.250999+0000","last_change":"2024-03-08T06:00:35.164505+0000","last_active":"2024-03-08T06:00:43.250999+0000","last_peered":"2024-03-08T06:00:43.250999+0000","last_clean":"2024-03-08T06:00:43.250999+0000","last_became_active":"2024-03-08T06:00:35.164203+0000","last_became_peered":"2024-03-08T06:00:35.164203+0000","last_unstale":"2024-03-08T06:00:43.250999+0000","last_undegraded":"2024-03-08T06:00:43.250999+0000","last_fullsized":"2024-03-08T06:00:43.250999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:38.366092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253073+0000","last_change":"2024-03-08T06:00:40.772590+0000","last_active":"2024-03-08T06:00:43.253073+0000","last_peered":"2024-03-08T06:00:43.253073+0000","last_clean":"2024-03-08T06:00:43.253073+0000","last_became_active":"2024-03-08T06:00:40.772198+0000","last_became_peered":"2024-03-08T06:00:40.772198+0000","last_unstale":"2024-03-08T06:00:43.253073+0000","last_undegraded":"2024-03-08T06:00:43.253073+0000","last_fullsized":"2024-03-08T06:00:43.253073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:24:20.856936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206949+0000","last_change":"2024-03-08T06:00:36.180323+0000","last_active":"2024-03-08T06:00:42.206949+0000","last_peered":"2024-03-08T06:00:42.206949+0000","last_clean":"2024-03-08T06:00:42.206949+0000","last_became_active":"2024-03-08T06:00:36.180087+0000","last_became_peered":"2024-03-08T06:00:36.180087+0000","last_unstale":"2024-03-08T06:00:42.206949+0000","last_undegraded":"2024-03-08T06:00:42.206949+0000","last_fullsized":"2024-03-08T06:00:42.206949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:30:07.063230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751552+0000","last_change":"2024-03-08T06:00:35.179401+0000","last_active":"2024-03-08T06:00:42.751552+0000","last_peered":"2024-03-08T06:00:42.751552+0000","last_clean":"2024-03-08T06:00:42.751552+0000","last_became_active":"2024-03-08T06:00:35.179121+0000","last_became_peered":"2024-03-08T06:00:35.179121+0000","last_unstale":"2024-03-08T06:00:42.751552+0000","last_undegraded":"2024-03-08T06:00:42.751552+0000","last_fullsized":"2024-03-08T06:00:42.751552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:30:06.058910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.207092+0000","last_change":"2024-03-08T06:00:40.222278+0000","last_active":"2024-03-08T06:00:42.207092+0000","last_peered":"2024-03-08T06:00:42.207092+0000","last_clean":"2024-03-08T06:00:42.207092+0000","last_became_active":"2024-03-08T06:00:40.221874+0000","last_became_peered":"2024-03-08T06:00:40.221874+0000","last_unstale":"2024-03-08T06:00:42.207092+0000","last_undegraded":"2024-03-08T06:00:42.207092+0000","last_fullsized":"2024-03-08T06:00:42.207092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:33.029056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251991+0000","last_change":"2024-03-08T06:00:36.180994+0000","last_active":"2024-03-08T06:00:43.251991+0000","last_peered":"2024-03-08T06:00:43.251991+0000","last_clean":"2024-03-08T06:00:43.251991+0000","last_became_active":"2024-03-08T06:00:36.178891+0000","last_became_peered":"2024-03-08T06:00:36.178891+0000","last_unstale":"2024-03-08T06:00:43.251991+0000","last_undegraded":"2024-03-08T06:00:43.251991+0000","last_fullsized":"2024-03-08T06:00:43.251991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:40:16.053974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253772+0000","last_change":"2024-03-08T06:00:35.166077+0000","last_active":"2024-03-08T06:00:43.253772+0000","last_peered":"2024-03-08T06:00:43.253772+0000","last_clean":"2024-03-08T06:00:43.253772+0000","last_became_active":"2024-03-08T06:00:35.165721+0000","last_became_peered":"2024-03-08T06:00:35.165721+0000","last_unstale":"2024-03-08T06:00:43.253772+0000","last_undegraded":"2024-03-08T06:00:43.253772+0000","last_fullsized":"2024-03-08T06:00:43.253772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206253+0000","last_change":"2024-03-08T06:00:40.772000+0000","last_active":"2024-03-08T06:00:42.206253+0000","last_peered":"2024-03-08T06:00:42.206253+0000","last_clean":"2024-03-08T06:00:42.206253+0000","last_became_active":"2024-03-08T06:00:40.771449+0000","last_became_peered":"2024-03-08T06:00:40.771449+0000","last_unstale":"2024-03-08T06:00:42.206253+0000","last_undegraded":"2024-03-08T06:00:42.206253+0000","last_fullsized":"2024-03-08T06:00:42.206253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:44:48.992193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205931+0000","last_change":"2024-03-08T06:00:36.185234+0000","last_active":"2024-03-08T06:00:42.205931+0000","last_peered":"2024-03-08T06:00:42.205931+0000","last_clean":"2024-03-08T06:00:42.205931+0000","last_became_active":"2024-03-08T06:00:36.184020+0000","last_became_peered":"2024-03-08T06:00:36.184020+0000","last_unstale":"2024-03-08T06:00:42.205931+0000","last_undegraded":"2024-03-08T06:00:42.205931+0000","last_fullsized":"2024-03-08T06:00:42.205931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:39:59.986300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252725+0000","last_change":"2024-03-08T06:00:35.166222+0000","last_active":"2024-03-08T06:00:43.252725+0000","last_peered":"2024-03-08T06:00:43.252725+0000","last_clean":"2024-03-08T06:00:43.252725+0000","last_became_active":"2024-03-08T06:00:35.165815+0000","last_became_peered":"2024-03-08T06:00:35.165815+0000","last_unstale":"2024-03-08T06:00:43.252725+0000","last_undegraded":"2024-03-08T06:00:43.252725+0000","last_fullsized":"2024-03-08T06:00:43.252725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.204928+0000","last_change":"2024-03-08T06:00:40.771309+0000","last_active":"2024-03-08T06:00:42.204928+0000","last_peered":"2024-03-08T06:00:42.204928+0000","last_clean":"2024-03-08T06:00:42.204928+0000","last_became_active":"2024-03-08T06:00:40.770916+0000","last_became_peered":"2024-03-08T06:00:40.770916+0000","last_unstale":"2024-03-08T06:00:42.204928+0000","last_undegraded":"2024-03-08T06:00:42.204928+0000","last_fullsized":"2024-03-08T06:00:42.204928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:26:51.436673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252117+0000","last_change":"2024-03-08T06:00:36.177665+0000","last_active":"2024-03-08T06:00:43.252117+0000","last_peered":"2024-03-08T06:00:43.252117+0000","last_clean":"2024-03-08T06:00:43.252117+0000","last_became_active":"2024-03-08T06:00:36.177409+0000","last_became_peered":"2024-03-08T06:00:36.177409+0000","last_unstale":"2024-03-08T06:00:43.252117+0000","last_undegraded":"2024-03-08T06:00:43.252117+0000","last_fullsized":"2024-03-08T06:00:43.252117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752203+0000","last_change":"2024-03-08T06:00:35.179418+0000","last_active":"2024-03-08T06:00:42.752203+0000","last_peered":"2024-03-08T06:00:42.752203+0000","last_clean":"2024-03-08T06:00:42.752203+0000","last_became_active":"2024-03-08T06:00:35.179132+0000","last_became_peered":"2024-03-08T06:00:35.179132+0000","last_unstale":"2024-03-08T06:00:42.752203+0000","last_undegraded":"2024-03-08T06:00:42.752203+0000","last_fullsized":"2024-03-08T06:00:42.752203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:11.262668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251653+0000","last_change":"2024-03-08T06:00:40.771662+0000","last_active":"2024-03-08T06:00:43.251653+0000","last_peered":"2024-03-08T06:00:43.251653+0000","last_clean":"2024-03-08T06:00:43.251653+0000","last_became_active":"2024-03-08T06:00:40.771168+0000","last_became_peered":"2024-03-08T06:00:40.771168+0000","last_unstale":"2024-03-08T06:00:43.251653+0000","last_undegraded":"2024-03-08T06:00:43.251653+0000","last_fullsized":"2024-03-08T06:00:43.251653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:16:51.281060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252306+0000","last_change":"2024-03-08T06:00:36.193542+0000","last_active":"2024-03-08T06:00:43.252306+0000","last_peered":"2024-03-08T06:00:43.252306+0000","last_clean":"2024-03-08T06:00:43.252306+0000","last_became_active":"2024-03-08T06:00:36.191695+0000","last_became_peered":"2024-03-08T06:00:36.191695+0000","last_unstale":"2024-03-08T06:00:43.252306+0000","last_undegraded":"2024-03-08T06:00:43.252306+0000","last_fullsized":"2024-03-08T06:00:43.252306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:16:14.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252207+0000","last_change":"2024-03-08T06:00:35.177137+0000","last_active":"2024-03-08T06:00:43.252207+0000","last_peered":"2024-03-08T06:00:43.252207+0000","last_clean":"2024-03-08T06:00:43.252207+0000","last_became_active":"2024-03-08T06:00:35.176790+0000","last_became_peered":"2024-03-08T06:00:35.176790+0000","last_unstale":"2024-03-08T06:00:43.252207+0000","last_undegraded":"2024-03-08T06:00:43.252207+0000","last_fullsized":"2024-03-08T06:00:43.252207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:11.262668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252112+0000","last_change":"2024-03-08T06:00:40.220329+0000","last_active":"2024-03-08T06:00:43.252112+0000","last_peered":"2024-03-08T06:00:43.252112+0000","last_clean":"2024-03-08T06:00:43.252112+0000","last_became_active":"2024-03-08T06:00:40.219921+0000","last_became_peered":"2024-03-08T06:00:40.219921+0000","last_unstale":"2024-03-08T06:00:43.252112+0000","last_undegraded":"2024-03-08T06:00:43.252112+0000","last_fullsized":"2024-03-08T06:00:43.252112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:08:55.908866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.207142+0000","last_change":"2024-03-08T06:00:36.187190+0000","last_active":"2024-03-08T06:00:42.207142+0000","last_peered":"2024-03-08T06:00:42.207142+0000","last_clean":"2024-03-08T06:00:42.207142+0000","last_became_active":"2024-03-08T06:00:36.186916+0000","last_became_peered":"2024-03-08T06:00:36.186916+0000","last_unstale":"2024-03-08T06:00:42.207142+0000","last_undegraded":"2024-03-08T06:00:42.207142+0000","last_fullsized":"2024-03-08T06:00:42.207142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:39:00.058426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251977+0000","last_change":"2024-03-08T06:00:35.179558+0000","last_active":"2024-03-08T06:00:43.251977+0000","last_peered":"2024-03-08T06:00:43.251977+0000","last_clean":"2024-03-08T06:00:43.251977+0000","last_became_active":"2024-03-08T06:00:35.179266+0000","last_became_peered":"2024-03-08T06:00:35.179266+0000","last_unstale":"2024-03-08T06:00:43.251977+0000","last_undegraded":"2024-03-08T06:00:43.251977+0000","last_fullsized":"2024-03-08T06:00:43.251977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:38.366092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252067+0000","last_change":"2024-03-08T06:00:40.224570+0000","last_active":"2024-03-08T06:00:43.252067+0000","last_peered":"2024-03-08T06:00:43.252067+0000","last_clean":"2024-03-08T06:00:43.252067+0000","last_became_active":"2024-03-08T06:00:40.222949+0000","last_became_peered":"2024-03-08T06:00:40.222949+0000","last_unstale":"2024-03-08T06:00:43.252067+0000","last_undegraded":"2024-03-08T06:00:43.252067+0000","last_fullsized":"2024-03-08T06:00:43.252067+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:16:51.281060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251279+0000","last_change":"2024-03-08T06:00:36.178768+0000","last_active":"2024-03-08T06:00:43.251279+0000","last_peered":"2024-03-08T06:00:43.251279+0000","last_clean":"2024-03-08T06:00:43.251279+0000","last_became_active":"2024-03-08T06:00:36.177890+0000","last_became_peered":"2024-03-08T06:00:36.177890+0000","last_unstale":"2024-03-08T06:00:43.251279+0000","last_undegraded":"2024-03-08T06:00:43.251279+0000","last_fullsized":"2024-03-08T06:00:43.251279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:01:37.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251961+0000","last_change":"2024-03-08T06:00:35.177031+0000","last_active":"2024-03-08T06:00:43.251961+0000","last_peered":"2024-03-08T06:00:43.251961+0000","last_clean":"2024-03-08T06:00:43.251961+0000","last_became_active":"2024-03-08T06:00:35.176687+0000","last_became_peered":"2024-03-08T06:00:35.176687+0000","last_unstale":"2024-03-08T06:00:43.251961+0000","last_undegraded":"2024-03-08T06:00:43.251961+0000","last_fullsized":"2024-03-08T06:00:43.251961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253204+0000","last_change":"2024-03-08T06:00:40.219392+0000","last_active":"2024-03-08T06:00:43.253204+0000","last_peered":"2024-03-08T06:00:43.253204+0000","last_clean":"2024-03-08T06:00:43.253204+0000","last_became_active":"2024-03-08T06:00:40.219040+0000","last_became_peered":"2024-03-08T06:00:40.219040+0000","last_unstale":"2024-03-08T06:00:43.253204+0000","last_undegraded":"2024-03-08T06:00:43.253204+0000","last_fullsized":"2024-03-08T06:00:43.253204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:33.029056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251221+0000","last_change":"2024-03-08T06:00:36.185525+0000","last_active":"2024-03-08T06:00:43.251221+0000","last_peered":"2024-03-08T06:00:43.251221+0000","last_clean":"2024-03-08T06:00:43.251221+0000","last_became_active":"2024-03-08T06:00:36.185111+0000","last_became_peered":"2024-03-08T06:00:36.185111+0000","last_unstale":"2024-03-08T06:00:43.251221+0000","last_undegraded":"2024-03-08T06:00:43.251221+0000","last_fullsized":"2024-03-08T06:00:43.251221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:52.669024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751556+0000","last_change":"2024-03-08T06:00:35.166949+0000","last_active":"2024-03-08T06:00:42.751556+0000","last_peered":"2024-03-08T06:00:42.751556+0000","last_clean":"2024-03-08T06:00:42.751556+0000","last_became_active":"2024-03-08T06:00:35.166680+0000","last_became_peered":"2024-03-08T06:00:35.166680+0000","last_unstale":"2024-03-08T06:00:42.751556+0000","last_undegraded":"2024-03-08T06:00:42.751556+0000","last_fullsized":"2024-03-08T06:00:42.751556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:58:33.115927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251873+0000","last_change":"2024-03-08T06:00:40.771268+0000","last_active":"2024-03-08T06:00:43.251873+0000","last_peered":"2024-03-08T06:00:43.251873+0000","last_clean":"2024-03-08T06:00:43.251873+0000","last_became_active":"2024-03-08T06:00:40.770846+0000","last_became_peered":"2024-03-08T06:00:40.770846+0000","last_unstale":"2024-03-08T06:00:43.251873+0000","last_undegraded":"2024-03-08T06:00:43.251873+0000","last_fullsized":"2024-03-08T06:00:43.251873+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:34:22.301392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205219+0000","last_change":"2024-03-08T06:00:36.200350+0000","last_active":"2024-03-08T06:00:42.205219+0000","last_peered":"2024-03-08T06:00:42.205219+0000","last_clean":"2024-03-08T06:00:42.205219+0000","last_became_active":"2024-03-08T06:00:36.200036+0000","last_became_peered":"2024-03-08T06:00:36.200036+0000","last_unstale":"2024-03-08T06:00:42.205219+0000","last_undegraded":"2024-03-08T06:00:42.205219+0000","last_fullsized":"2024-03-08T06:00:42.205219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:10:17.180499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251960+0000","last_change":"2024-03-08T06:00:35.176857+0000","last_active":"2024-03-08T06:00:43.251960+0000","last_peered":"2024-03-08T06:00:43.251960+0000","last_clean":"2024-03-08T06:00:43.251960+0000","last_became_active":"2024-03-08T06:00:35.176593+0000","last_became_peered":"2024-03-08T06:00:35.176593+0000","last_unstale":"2024-03-08T06:00:43.251960+0000","last_undegraded":"2024-03-08T06:00:43.251960+0000","last_fullsized":"2024-03-08T06:00:43.251960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:41.059111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751648+0000","last_change":"2024-03-08T06:00:26.178419+0000","last_active":"2024-03-08T06:00:42.751648+0000","last_peered":"2024-03-08T06:00:42.751648+0000","last_clean":"2024-03-08T06:00:42.751648+0000","last_became_active":"2024-03-08T06:00:26.175826+0000","last_became_peered":"2024-03-08T06:00:26.175826+0000","last_unstale":"2024-03-08T06:00:42.751648+0000","last_undegraded":"2024-03-08T06:00:42.751648+0000","last_fullsized":"2024-03-08T06:00:42.751648+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_clean_scrub_stamp":"2024-03-08T06:00:24.029492+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:36:00.682517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254739+0000","last_change":"2024-03-08T06:00:40.773106+0000","last_active":"2024-03-08T06:00:43.254739+0000","last_peered":"2024-03-08T06:00:43.254739+0000","last_clean":"2024-03-08T06:00:43.254739+0000","last_became_active":"2024-03-08T06:00:40.772665+0000","last_became_peered":"2024-03-08T06:00:40.772665+0000","last_unstale":"2024-03-08T06:00:43.254739+0000","last_undegraded":"2024-03-08T06:00:43.254739+0000","last_fullsized":"2024-03-08T06:00:43.254739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:02:19.499764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752004+0000","last_change":"2024-03-08T06:00:36.204915+0000","last_active":"2024-03-08T06:00:42.752004+0000","last_peered":"2024-03-08T06:00:42.752004+0000","last_clean":"2024-03-08T06:00:42.752004+0000","last_became_active":"2024-03-08T06:00:36.204663+0000","last_became_peered":"2024-03-08T06:00:36.204663+0000","last_unstale":"2024-03-08T06:00:42.752004+0000","last_undegraded":"2024-03-08T06:00:42.752004+0000","last_fullsized":"2024-03-08T06:00:42.752004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:05.458469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252634+0000","last_change":"2024-03-08T06:00:35.173339+0000","last_active":"2024-03-08T06:00:43.252634+0000","last_peered":"2024-03-08T06:00:43.252634+0000","last_clean":"2024-03-08T06:00:43.252634+0000","last_became_active":"2024-03-08T06:00:35.173024+0000","last_became_peered":"2024-03-08T06:00:35.173024+0000","last_unstale":"2024-03-08T06:00:43.252634+0000","last_undegraded":"2024-03-08T06:00:43.252634+0000","last_fullsized":"2024-03-08T06:00:43.252634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:41.059111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254629+0000","last_change":"2024-03-08T06:00:40.771194+0000","last_active":"2024-03-08T06:00:43.254629+0000","last_peered":"2024-03-08T06:00:43.254629+0000","last_clean":"2024-03-08T06:00:43.254629+0000","last_became_active":"2024-03-08T06:00:40.770762+0000","last_became_peered":"2024-03-08T06:00:40.770762+0000","last_unstale":"2024-03-08T06:00:43.254629+0000","last_undegraded":"2024-03-08T06:00:43.254629+0000","last_fullsized":"2024-03-08T06:00:43.254629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:21:03.123682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252111+0000","last_change":"2024-03-08T06:00:36.186111+0000","last_active":"2024-03-08T06:00:43.252111+0000","last_peered":"2024-03-08T06:00:43.252111+0000","last_clean":"2024-03-08T06:00:43.252111+0000","last_became_active":"2024-03-08T06:00:36.185865+0000","last_became_peered":"2024-03-08T06:00:36.185865+0000","last_unstale":"2024-03-08T06:00:43.252111+0000","last_undegraded":"2024-03-08T06:00:43.252111+0000","last_fullsized":"2024-03-08T06:00:43.252111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:19:26.058130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251711+0000","last_change":"2024-03-08T06:00:35.166769+0000","last_active":"2024-03-08T06:00:43.251711+0000","last_peered":"2024-03-08T06:00:43.251711+0000","last_clean":"2024-03-08T06:00:43.251711+0000","last_became_active":"2024-03-08T06:00:35.166396+0000","last_became_peered":"2024-03-08T06:00:35.166396+0000","last_unstale":"2024-03-08T06:00:43.251711+0000","last_undegraded":"2024-03-08T06:00:43.251711+0000","last_fullsized":"2024-03-08T06:00:43.251711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:30:06.058910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251907+0000","last_change":"2024-03-08T06:00:40.219431+0000","last_active":"2024-03-08T06:00:43.251907+0000","last_peered":"2024-03-08T06:00:43.251907+0000","last_clean":"2024-03-08T06:00:43.251907+0000","last_became_active":"2024-03-08T06:00:40.219091+0000","last_became_peered":"2024-03-08T06:00:40.219091+0000","last_unstale":"2024-03-08T06:00:43.251907+0000","last_undegraded":"2024-03-08T06:00:43.251907+0000","last_fullsized":"2024-03-08T06:00:43.251907+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:24:20.856936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751480+0000","last_change":"2024-03-08T06:00:36.202917+0000","last_active":"2024-03-08T06:00:42.751480+0000","last_peered":"2024-03-08T06:00:42.751480+0000","last_clean":"2024-03-08T06:00:42.751480+0000","last_became_active":"2024-03-08T06:00:36.202100+0000","last_became_peered":"2024-03-08T06:00:36.202100+0000","last_unstale":"2024-03-08T06:00:42.751480+0000","last_undegraded":"2024-03-08T06:00:42.751480+0000","last_fullsized":"2024-03-08T06:00:42.751480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:31.645981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252205+0000","last_change":"2024-03-08T06:00:35.171650+0000","last_active":"2024-03-08T06:00:43.252205+0000","last_peered":"2024-03-08T06:00:43.252205+0000","last_clean":"2024-03-08T06:00:43.252205+0000","last_became_active":"2024-03-08T06:00:35.171310+0000","last_became_peered":"2024-03-08T06:00:35.171310+0000","last_unstale":"2024-03-08T06:00:43.252205+0000","last_undegraded":"2024-03-08T06:00:43.252205+0000","last_fullsized":"2024-03-08T06:00:43.252205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251524+0000","last_change":"2024-03-08T06:00:40.774574+0000","last_active":"2024-03-08T06:00:43.251524+0000","last_peered":"2024-03-08T06:00:43.251524+0000","last_clean":"2024-03-08T06:00:43.251524+0000","last_became_active":"2024-03-08T06:00:40.774172+0000","last_became_peered":"2024-03-08T06:00:40.774172+0000","last_unstale":"2024-03-08T06:00:43.251524+0000","last_undegraded":"2024-03-08T06:00:43.251524+0000","last_fullsized":"2024-03-08T06:00:43.251524+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:48:26.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752182+0000","last_change":"2024-03-08T06:00:36.195883+0000","last_active":"2024-03-08T06:00:42.752182+0000","last_peered":"2024-03-08T06:00:42.752182+0000","last_clean":"2024-03-08T06:00:42.752182+0000","last_became_active":"2024-03-08T06:00:36.195641+0000","last_became_peered":"2024-03-08T06:00:36.195641+0000","last_unstale":"2024-03-08T06:00:42.752182+0000","last_undegraded":"2024-03-08T06:00:42.752182+0000","last_fullsized":"2024-03-08T06:00:42.752182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:02:15.458747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206956+0000","last_change":"2024-03-08T06:00:35.165484+0000","last_active":"2024-03-08T06:00:42.206956+0000","last_peered":"2024-03-08T06:00:42.206956+0000","last_clean":"2024-03-08T06:00:42.206956+0000","last_became_active":"2024-03-08T06:00:35.165147+0000","last_became_peered":"2024-03-08T06:00:35.165147+0000","last_unstale":"2024-03-08T06:00:42.206956+0000","last_undegraded":"2024-03-08T06:00:42.206956+0000","last_fullsized":"2024-03-08T06:00:42.206956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:07:54.742119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206806+0000","last_change":"2024-03-08T06:00:40.221435+0000","last_active":"2024-03-08T06:00:42.206806+0000","last_peered":"2024-03-08T06:00:42.206806+0000","last_clean":"2024-03-08T06:00:42.206806+0000","last_became_active":"2024-03-08T06:00:40.220886+0000","last_became_peered":"2024-03-08T06:00:40.220886+0000","last_unstale":"2024-03-08T06:00:42.206806+0000","last_undegraded":"2024-03-08T06:00:42.206806+0000","last_fullsized":"2024-03-08T06:00:42.206806+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:24:20.856936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751752+0000","last_change":"2024-03-08T06:00:36.202146+0000","last_active":"2024-03-08T06:00:42.751752+0000","last_peered":"2024-03-08T06:00:42.751752+0000","last_clean":"2024-03-08T06:00:42.751752+0000","last_became_active":"2024-03-08T06:00:36.201880+0000","last_became_peered":"2024-03-08T06:00:36.201880+0000","last_unstale":"2024-03-08T06:00:42.751752+0000","last_undegraded":"2024-03-08T06:00:42.751752+0000","last_fullsized":"2024-03-08T06:00:42.751752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:45:22.429641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252369+0000","last_change":"2024-03-08T06:00:35.168305+0000","last_active":"2024-03-08T06:00:43.252369+0000","last_peered":"2024-03-08T06:00:43.252369+0000","last_clean":"2024-03-08T06:00:43.252369+0000","last_became_active":"2024-03-08T06:00:35.168007+0000","last_became_peered":"2024-03-08T06:00:35.168007+0000","last_unstale":"2024-03-08T06:00:43.252369+0000","last_undegraded":"2024-03-08T06:00:43.252369+0000","last_fullsized":"2024-03-08T06:00:43.252369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:57:55.324372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251733+0000","last_change":"2024-03-08T06:00:40.217477+0000","last_active":"2024-03-08T06:00:43.251733+0000","last_peered":"2024-03-08T06:00:43.251733+0000","last_clean":"2024-03-08T06:00:43.251733+0000","last_became_active":"2024-03-08T06:00:40.217140+0000","last_became_peered":"2024-03-08T06:00:40.217140+0000","last_unstale":"2024-03-08T06:00:43.251733+0000","last_undegraded":"2024-03-08T06:00:43.251733+0000","last_fullsized":"2024-03-08T06:00:43.251733+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:08:55.908866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254784+0000","last_change":"2024-03-08T06:00:36.188881+0000","last_active":"2024-03-08T06:00:43.254784+0000","last_peered":"2024-03-08T06:00:43.254784+0000","last_clean":"2024-03-08T06:00:43.254784+0000","last_became_active":"2024-03-08T06:00:36.188323+0000","last_became_peered":"2024-03-08T06:00:36.188323+0000","last_unstale":"2024-03-08T06:00:43.254784+0000","last_undegraded":"2024-03-08T06:00:43.254784+0000","last_fullsized":"2024-03-08T06:00:43.254784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:11:34.950063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253637+0000","last_change":"2024-03-08T06:00:35.163917+0000","last_active":"2024-03-08T06:00:43.253637+0000","last_peered":"2024-03-08T06:00:43.253637+0000","last_clean":"2024-03-08T06:00:43.253637+0000","last_became_active":"2024-03-08T06:00:35.163650+0000","last_became_peered":"2024-03-08T06:00:35.163650+0000","last_unstale":"2024-03-08T06:00:43.253637+0000","last_undegraded":"2024-03-08T06:00:43.253637+0000","last_fullsized":"2024-03-08T06:00:43.253637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:11.262668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205799+0000","last_change":"2024-03-08T06:00:40.773617+0000","last_active":"2024-03-08T06:00:42.205799+0000","last_peered":"2024-03-08T06:00:42.205799+0000","last_clean":"2024-03-08T06:00:42.205799+0000","last_became_active":"2024-03-08T06:00:40.773144+0000","last_became_peered":"2024-03-08T06:00:40.773144+0000","last_unstale":"2024-03-08T06:00:42.205799+0000","last_undegraded":"2024-03-08T06:00:42.205799+0000","last_fullsized":"2024-03-08T06:00:42.205799+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:32:08.762183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751941+0000","last_change":"2024-03-08T06:00:36.202751+0000","last_active":"2024-03-08T06:00:42.751941+0000","last_peered":"2024-03-08T06:00:42.751941+0000","last_clean":"2024-03-08T06:00:42.751941+0000","last_became_active":"2024-03-08T06:00:36.202228+0000","last_became_peered":"2024-03-08T06:00:36.202228+0000","last_unstale":"2024-03-08T06:00:42.751941+0000","last_undegraded":"2024-03-08T06:00:42.751941+0000","last_fullsized":"2024-03-08T06:00:42.751941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:16:14.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251319+0000","last_change":"2024-03-08T06:00:35.176896+0000","last_active":"2024-03-08T06:00:43.251319+0000","last_peered":"2024-03-08T06:00:43.251319+0000","last_clean":"2024-03-08T06:00:43.251319+0000","last_became_active":"2024-03-08T06:00:35.176665+0000","last_became_peered":"2024-03-08T06:00:35.176665+0000","last_unstale":"2024-03-08T06:00:43.251319+0000","last_undegraded":"2024-03-08T06:00:43.251319+0000","last_fullsized":"2024-03-08T06:00:43.251319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:11.262668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205674+0000","last_change":"2024-03-08T06:00:40.771254+0000","last_active":"2024-03-08T06:00:42.205674+0000","last_peered":"2024-03-08T06:00:42.205674+0000","last_clean":"2024-03-08T06:00:42.205674+0000","last_became_active":"2024-03-08T06:00:40.770828+0000","last_became_peered":"2024-03-08T06:00:40.770828+0000","last_unstale":"2024-03-08T06:00:42.205674+0000","last_undegraded":"2024-03-08T06:00:42.205674+0000","last_fullsized":"2024-03-08T06:00:42.205674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:06:36.831014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251667+0000","last_change":"2024-03-08T06:00:36.192057+0000","last_active":"2024-03-08T06:00:43.251667+0000","last_peered":"2024-03-08T06:00:43.251667+0000","last_clean":"2024-03-08T06:00:43.251667+0000","last_became_active":"2024-03-08T06:00:36.191786+0000","last_became_peered":"2024-03-08T06:00:36.191786+0000","last_unstale":"2024-03-08T06:00:43.251667+0000","last_undegraded":"2024-03-08T06:00:43.251667+0000","last_fullsized":"2024-03-08T06:00:43.251667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:10:17.180499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251277+0000","last_change":"2024-03-08T06:00:35.164201+0000","last_active":"2024-03-08T06:00:43.251277+0000","last_peered":"2024-03-08T06:00:43.251277+0000","last_clean":"2024-03-08T06:00:43.251277+0000","last_became_active":"2024-03-08T06:00:35.163882+0000","last_became_peered":"2024-03-08T06:00:35.163882+0000","last_unstale":"2024-03-08T06:00:43.251277+0000","last_undegraded":"2024-03-08T06:00:43.251277+0000","last_fullsized":"2024-03-08T06:00:43.251277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:43:03.634887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252307+0000","last_change":"2024-03-08T06:00:40.773272+0000","last_active":"2024-03-08T06:00:43.252307+0000","last_peered":"2024-03-08T06:00:43.252307+0000","last_clean":"2024-03-08T06:00:43.252307+0000","last_became_active":"2024-03-08T06:00:40.772833+0000","last_became_peered":"2024-03-08T06:00:40.772833+0000","last_unstale":"2024-03-08T06:00:43.252307+0000","last_undegraded":"2024-03-08T06:00:43.252307+0000","last_fullsized":"2024-03-08T06:00:43.252307+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:53:52.773407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751887+0000","last_change":"2024-03-08T06:00:36.191731+0000","last_active":"2024-03-08T06:00:42.751887+0000","last_peered":"2024-03-08T06:00:42.751887+0000","last_clean":"2024-03-08T06:00:42.751887+0000","last_became_active":"2024-03-08T06:00:36.191469+0000","last_became_peered":"2024-03-08T06:00:36.191469+0000","last_unstale":"2024-03-08T06:00:42.751887+0000","last_undegraded":"2024-03-08T06:00:42.751887+0000","last_fullsized":"2024-03-08T06:00:42.751887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:59.082666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:45.428919+0000","last_change":"2024-03-08T06:00:35.171245+0000","last_active":"2024-03-08T06:00:45.428919+0000","last_peered":"2024-03-08T06:00:45.428919+0000","last_clean":"2024-03-08T06:00:45.428919+0000","last_became_active":"2024-03-08T06:00:35.170999+0000","last_became_peered":"2024-03-08T06:00:35.170999+0000","last_unstale":"2024-03-08T06:00:45.428919+0000","last_undegraded":"2024-03-08T06:00:45.428919+0000","last_fullsized":"2024-03-08T06:00:45.428919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252315+0000","last_change":"2024-03-08T06:00:40.222311+0000","last_active":"2024-03-08T06:00:43.252315+0000","last_peered":"2024-03-08T06:00:43.252315+0000","last_clean":"2024-03-08T06:00:43.252315+0000","last_became_active":"2024-03-08T06:00:40.221923+0000","last_became_peered":"2024-03-08T06:00:40.221923+0000","last_unstale":"2024-03-08T06:00:43.252315+0000","last_undegraded":"2024-03-08T06:00:43.252315+0000","last_fullsized":"2024-03-08T06:00:43.252315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:34:22.301392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206554+0000","last_change":"2024-03-08T06:00:36.190623+0000","last_active":"2024-03-08T06:00:42.206554+0000","last_peered":"2024-03-08T06:00:42.206554+0000","last_clean":"2024-03-08T06:00:42.206554+0000","last_became_active":"2024-03-08T06:00:36.190343+0000","last_became_peered":"2024-03-08T06:00:36.190343+0000","last_unstale":"2024-03-08T06:00:42.206554+0000","last_undegraded":"2024-03-08T06:00:42.206554+0000","last_fullsized":"2024-03-08T06:00:42.206554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:39.370412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252587+0000","last_change":"2024-03-08T06:00:35.169188+0000","last_active":"2024-03-08T06:00:43.252587+0000","last_peered":"2024-03-08T06:00:43.252587+0000","last_clean":"2024-03-08T06:00:43.252587+0000","last_became_active":"2024-03-08T06:00:35.168945+0000","last_became_peered":"2024-03-08T06:00:35.168945+0000","last_unstale":"2024-03-08T06:00:43.252587+0000","last_undegraded":"2024-03-08T06:00:43.252587+0000","last_fullsized":"2024-03-08T06:00:43.252587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:58:33.115927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206843+0000","last_change":"2024-03-08T06:00:40.219289+0000","last_active":"2024-03-08T06:00:42.206843+0000","last_peered":"2024-03-08T06:00:42.206843+0000","last_clean":"2024-03-08T06:00:42.206843+0000","last_became_active":"2024-03-08T06:00:40.218831+0000","last_became_peered":"2024-03-08T06:00:40.218831+0000","last_unstale":"2024-03-08T06:00:42.206843+0000","last_undegraded":"2024-03-08T06:00:42.206843+0000","last_fullsized":"2024-03-08T06:00:42.206843+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:08:55.908866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251809+0000","last_change":"2024-03-08T06:00:36.191492+0000","last_active":"2024-03-08T06:00:43.251809+0000","last_peered":"2024-03-08T06:00:43.251809+0000","last_clean":"2024-03-08T06:00:43.251809+0000","last_became_active":"2024-03-08T06:00:36.190895+0000","last_became_peered":"2024-03-08T06:00:36.190895+0000","last_unstale":"2024-03-08T06:00:43.251809+0000","last_undegraded":"2024-03-08T06:00:43.251809+0000","last_fullsized":"2024-03-08T06:00:43.251809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252270+0000","last_change":"2024-03-08T06:00:35.174075+0000","last_active":"2024-03-08T06:00:43.252270+0000","last_peered":"2024-03-08T06:00:43.252270+0000","last_clean":"2024-03-08T06:00:43.252270+0000","last_became_active":"2024-03-08T06:00:35.173803+0000","last_became_peered":"2024-03-08T06:00:35.173803+0000","last_unstale":"2024-03-08T06:00:43.252270+0000","last_undegraded":"2024-03-08T06:00:43.252270+0000","last_fullsized":"2024-03-08T06:00:43.252270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205087+0000","last_change":"2024-03-08T06:00:40.222466+0000","last_active":"2024-03-08T06:00:42.205087+0000","last_peered":"2024-03-08T06:00:42.205087+0000","last_clean":"2024-03-08T06:00:42.205087+0000","last_became_active":"2024-03-08T06:00:40.222083+0000","last_became_peered":"2024-03-08T06:00:40.222083+0000","last_unstale":"2024-03-08T06:00:42.205087+0000","last_undegraded":"2024-03-08T06:00:42.205087+0000","last_fullsized":"2024-03-08T06:00:42.205087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:21:08.958898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252185+0000","last_change":"2024-03-08T06:00:36.191479+0000","last_active":"2024-03-08T06:00:43.252185+0000","last_peered":"2024-03-08T06:00:43.252185+0000","last_clean":"2024-03-08T06:00:43.252185+0000","last_became_active":"2024-03-08T06:00:36.189667+0000","last_became_peered":"2024-03-08T06:00:36.189667+0000","last_unstale":"2024-03-08T06:00:43.252185+0000","last_undegraded":"2024-03-08T06:00:43.252185+0000","last_fullsized":"2024-03-08T06:00:43.252185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:59.082666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253827+0000","last_change":"2024-03-08T06:00:35.173447+0000","last_active":"2024-03-08T06:00:43.253827+0000","last_peered":"2024-03-08T06:00:43.253827+0000","last_clean":"2024-03-08T06:00:43.253827+0000","last_became_active":"2024-03-08T06:00:35.173050+0000","last_became_peered":"2024-03-08T06:00:35.173050+0000","last_unstale":"2024-03-08T06:00:43.253827+0000","last_undegraded":"2024-03-08T06:00:43.253827+0000","last_fullsized":"2024-03-08T06:00:43.253827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:41.059111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205049+0000","last_change":"2024-03-08T06:00:40.772940+0000","last_active":"2024-03-08T06:00:42.205049+0000","last_peered":"2024-03-08T06:00:42.205049+0000","last_clean":"2024-03-08T06:00:42.205049+0000","last_became_active":"2024-03-08T06:00:40.772593+0000","last_became_peered":"2024-03-08T06:00:40.772593+0000","last_unstale":"2024-03-08T06:00:42.205049+0000","last_undegraded":"2024-03-08T06:00:42.205049+0000","last_fullsized":"2024-03-08T06:00:42.205049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:58:58.974013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252132+0000","last_change":"2024-03-08T06:00:40.773346+0000","last_active":"2024-03-08T06:00:43.252132+0000","last_peered":"2024-03-08T06:00:43.252132+0000","last_clean":"2024-03-08T06:00:43.252132+0000","last_became_active":"2024-03-08T06:00:40.772977+0000","last_became_peered":"2024-03-08T06:00:40.772977+0000","last_unstale":"2024-03-08T06:00:43.252132+0000","last_undegraded":"2024-03-08T06:00:43.252132+0000","last_fullsized":"2024-03-08T06:00:43.252132+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:33.029056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251192+0000","last_change":"2024-03-08T06:00:36.182265+0000","last_active":"2024-03-08T06:00:43.251192+0000","last_peered":"2024-03-08T06:00:43.251192+0000","last_clean":"2024-03-08T06:00:43.251192+0000","last_became_active":"2024-03-08T06:00:36.181732+0000","last_became_peered":"2024-03-08T06:00:36.181732+0000","last_unstale":"2024-03-08T06:00:43.251192+0000","last_undegraded":"2024-03-08T06:00:43.251192+0000","last_fullsized":"2024-03-08T06:00:43.251192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:40:16.053974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251308+0000","last_change":"2024-03-08T06:00:35.173204+0000","last_active":"2024-03-08T06:00:43.251308+0000","last_peered":"2024-03-08T06:00:43.251308+0000","last_clean":"2024-03-08T06:00:43.251308+0000","last_became_active":"2024-03-08T06:00:35.172937+0000","last_became_peered":"2024-03-08T06:00:35.172937+0000","last_unstale":"2024-03-08T06:00:43.251308+0000","last_undegraded":"2024-03-08T06:00:43.251308+0000","last_fullsized":"2024-03-08T06:00:43.251308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254582+0000","last_change":"2024-03-08T06:00:40.223795+0000","last_active":"2024-03-08T06:00:43.254582+0000","last_peered":"2024-03-08T06:00:43.254582+0000","last_clean":"2024-03-08T06:00:43.254582+0000","last_became_active":"2024-03-08T06:00:40.223283+0000","last_became_peered":"2024-03-08T06:00:40.223283+0000","last_unstale":"2024-03-08T06:00:43.254582+0000","last_undegraded":"2024-03-08T06:00:43.254582+0000","last_fullsized":"2024-03-08T06:00:43.254582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:09.499486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205033+0000","last_change":"2024-03-08T06:00:36.185555+0000","last_active":"2024-03-08T06:00:42.205033+0000","last_peered":"2024-03-08T06:00:42.205033+0000","last_clean":"2024-03-08T06:00:42.205033+0000","last_became_active":"2024-03-08T06:00:36.185294+0000","last_became_peered":"2024-03-08T06:00:36.185294+0000","last_unstale":"2024-03-08T06:00:42.205033+0000","last_undegraded":"2024-03-08T06:00:42.205033+0000","last_fullsized":"2024-03-08T06:00:42.205033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:13.003781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205103+0000","last_change":"2024-03-08T06:00:35.171146+0000","last_active":"2024-03-08T06:00:42.205103+0000","last_peered":"2024-03-08T06:00:42.205103+0000","last_clean":"2024-03-08T06:00:42.205103+0000","last_became_active":"2024-03-08T06:00:35.170885+0000","last_became_peered":"2024-03-08T06:00:35.170885+0000","last_unstale":"2024-03-08T06:00:42.205103+0000","last_undegraded":"2024-03-08T06:00:42.205103+0000","last_fullsized":"2024-03-08T06:00:42.205103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:59:03.457310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253633+0000","last_change":"2024-03-08T06:00:35.163289+0000","last_active":"2024-03-08T06:00:43.253633+0000","last_peered":"2024-03-08T06:00:43.253633+0000","last_clean":"2024-03-08T06:00:43.253633+0000","last_became_active":"2024-03-08T06:00:35.163005+0000","last_became_peered":"2024-03-08T06:00:35.163005+0000","last_unstale":"2024-03-08T06:00:43.253633+0000","last_undegraded":"2024-03-08T06:00:43.253633+0000","last_fullsized":"2024-03-08T06:00:43.253633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:38.366092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252056+0000","last_change":"2024-03-08T06:00:36.194840+0000","last_active":"2024-03-08T06:00:43.252056+0000","last_peered":"2024-03-08T06:00:43.252056+0000","last_clean":"2024-03-08T06:00:43.252056+0000","last_became_active":"2024-03-08T06:00:36.194547+0000","last_became_peered":"2024-03-08T06:00:36.194547+0000","last_unstale":"2024-03-08T06:00:43.252056+0000","last_undegraded":"2024-03-08T06:00:43.252056+0000","last_fullsized":"2024-03-08T06:00:43.252056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:11:34.950063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253014+0000","last_change":"2024-03-08T06:00:40.219835+0000","last_active":"2024-03-08T06:00:43.253014+0000","last_peered":"2024-03-08T06:00:43.253014+0000","last_clean":"2024-03-08T06:00:43.253014+0000","last_became_active":"2024-03-08T06:00:40.219483+0000","last_became_peered":"2024-03-08T06:00:40.219483+0000","last_unstale":"2024-03-08T06:00:43.253014+0000","last_undegraded":"2024-03-08T06:00:43.253014+0000","last_fullsized":"2024-03-08T06:00:43.253014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:41:06.159304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205357+0000","last_change":"2024-03-08T06:00:35.171304+0000","last_active":"2024-03-08T06:00:42.205357+0000","last_peered":"2024-03-08T06:00:42.205357+0000","last_clean":"2024-03-08T06:00:42.205357+0000","last_became_active":"2024-03-08T06:00:35.170979+0000","last_became_peered":"2024-03-08T06:00:35.170979+0000","last_unstale":"2024-03-08T06:00:42.205357+0000","last_undegraded":"2024-03-08T06:00:42.205357+0000","last_fullsized":"2024-03-08T06:00:42.205357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:41.059111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206410+0000","last_change":"2024-03-08T06:00:36.179352+0000","last_active":"2024-03-08T06:00:42.206410+0000","last_peered":"2024-03-08T06:00:42.206410+0000","last_clean":"2024-03-08T06:00:42.206410+0000","last_became_active":"2024-03-08T06:00:36.179123+0000","last_became_peered":"2024-03-08T06:00:36.179123+0000","last_unstale":"2024-03-08T06:00:42.206410+0000","last_undegraded":"2024-03-08T06:00:42.206410+0000","last_fullsized":"2024-03-08T06:00:42.206410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:12.266989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205576+0000","last_change":"2024-03-08T06:00:40.220569+0000","last_active":"2024-03-08T06:00:42.205576+0000","last_peered":"2024-03-08T06:00:42.205576+0000","last_clean":"2024-03-08T06:00:42.205576+0000","last_became_active":"2024-03-08T06:00:40.220153+0000","last_became_peered":"2024-03-08T06:00:40.220153+0000","last_unstale":"2024-03-08T06:00:42.205576+0000","last_undegraded":"2024-03-08T06:00:42.205576+0000","last_fullsized":"2024-03-08T06:00:42.205576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:44:28.955585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254222+0000","last_change":"2024-03-08T06:00:36.189112+0000","last_active":"2024-03-08T06:00:43.254222+0000","last_peered":"2024-03-08T06:00:43.254222+0000","last_clean":"2024-03-08T06:00:43.254222+0000","last_became_active":"2024-03-08T06:00:36.188650+0000","last_became_peered":"2024-03-08T06:00:36.188650+0000","last_unstale":"2024-03-08T06:00:43.254222+0000","last_undegraded":"2024-03-08T06:00:43.254222+0000","last_fullsized":"2024-03-08T06:00:43.254222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:13.003781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252434+0000","last_change":"2024-03-08T06:00:36.188109+0000","last_active":"2024-03-08T06:00:43.252434+0000","last_peered":"2024-03-08T06:00:43.252434+0000","last_clean":"2024-03-08T06:00:43.252434+0000","last_became_active":"2024-03-08T06:00:36.187798+0000","last_became_peered":"2024-03-08T06:00:36.187798+0000","last_unstale":"2024-03-08T06:00:43.252434+0000","last_undegraded":"2024-03-08T06:00:43.252434+0000","last_fullsized":"2024-03-08T06:00:43.252434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:30:07.063230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251880+0000","last_change":"2024-03-08T06:00:36.193357+0000","last_active":"2024-03-08T06:00:43.251880+0000","last_peered":"2024-03-08T06:00:43.251880+0000","last_clean":"2024-03-08T06:00:43.251880+0000","last_became_active":"2024-03-08T06:00:36.191586+0000","last_became_peered":"2024-03-08T06:00:36.191586+0000","last_unstale":"2024-03-08T06:00:43.251880+0000","last_undegraded":"2024-03-08T06:00:43.251880+0000","last_fullsized":"2024-03-08T06:00:43.251880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:44:44.951176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251957+0000","last_change":"2024-03-08T06:00:36.186266+0000","last_active":"2024-03-08T06:00:43.251957+0000","last_peered":"2024-03-08T06:00:43.251957+0000","last_clean":"2024-03-08T06:00:43.251957+0000","last_became_active":"2024-03-08T06:00:36.185663+0000","last_became_peered":"2024-03-08T06:00:36.185663+0000","last_unstale":"2024-03-08T06:00:43.251957+0000","last_undegraded":"2024-03-08T06:00:43.251957+0000","last_fullsized":"2024-03-08T06:00:43.251957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:53:20.034194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205842+0000","last_change":"2024-03-08T06:00:36.192645+0000","last_active":"2024-03-08T06:00:42.205842+0000","last_peered":"2024-03-08T06:00:42.205842+0000","last_clean":"2024-03-08T06:00:42.205842+0000","last_became_active":"2024-03-08T06:00:36.192391+0000","last_became_peered":"2024-03-08T06:00:36.192391+0000","last_unstale":"2024-03-08T06:00:42.205842+0000","last_undegraded":"2024-03-08T06:00:42.205842+0000","last_fullsized":"2024-03-08T06:00:42.205842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:45:22.429641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252413+0000","last_change":"2024-03-08T06:00:36.181871+0000","last_active":"2024-03-08T06:00:43.252413+0000","last_peered":"2024-03-08T06:00:43.252413+0000","last_clean":"2024-03-08T06:00:43.252413+0000","last_became_active":"2024-03-08T06:00:36.181557+0000","last_became_peered":"2024-03-08T06:00:36.181557+0000","last_unstale":"2024-03-08T06:00:43.252413+0000","last_undegraded":"2024-03-08T06:00:43.252413+0000","last_fullsized":"2024-03-08T06:00:43.252413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:10:17.180499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254922+0000","last_change":"2024-03-08T06:00:36.185046+0000","last_active":"2024-03-08T06:00:43.254922+0000","last_peered":"2024-03-08T06:00:43.254922+0000","last_clean":"2024-03-08T06:00:43.254922+0000","last_became_active":"2024-03-08T06:00:36.184066+0000","last_became_peered":"2024-03-08T06:00:36.184066+0000","last_unstale":"2024-03-08T06:00:43.254922+0000","last_undegraded":"2024-03-08T06:00:43.254922+0000","last_fullsized":"2024-03-08T06:00:43.254922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205846+0000","last_change":"2024-03-08T06:00:36.186595+0000","last_active":"2024-03-08T06:00:42.205846+0000","last_peered":"2024-03-08T06:00:42.205846+0000","last_clean":"2024-03-08T06:00:42.205846+0000","last_became_active":"2024-03-08T06:00:36.186261+0000","last_became_peered":"2024-03-08T06:00:36.186261+0000","last_unstale":"2024-03-08T06:00:42.205846+0000","last_undegraded":"2024-03-08T06:00:42.205846+0000","last_fullsized":"2024-03-08T06:00:42.205846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:16:14.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251609+0000","last_change":"2024-03-08T06:00:36.186618+0000","last_active":"2024-03-08T06:00:43.251609+0000","last_peered":"2024-03-08T06:00:43.251609+0000","last_clean":"2024-03-08T06:00:43.251609+0000","last_became_active":"2024-03-08T06:00:36.186068+0000","last_became_peered":"2024-03-08T06:00:36.186068+0000","last_unstale":"2024-03-08T06:00:43.251609+0000","last_undegraded":"2024-03-08T06:00:43.251609+0000","last_fullsized":"2024-03-08T06:00:43.251609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:39:59.986300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205158+0000","last_change":"2024-03-08T06:00:36.186759+0000","last_active":"2024-03-08T06:00:42.205158+0000","last_peered":"2024-03-08T06:00:42.205158+0000","last_clean":"2024-03-08T06:00:42.205158+0000","last_became_active":"2024-03-08T06:00:36.186338+0000","last_became_peered":"2024-03-08T06:00:36.186338+0000","last_unstale":"2024-03-08T06:00:42.205158+0000","last_undegraded":"2024-03-08T06:00:42.205158+0000","last_fullsized":"2024-03-08T06:00:42.205158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:40:16.053974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251835+0000","last_change":"2024-03-08T06:00:36.197002+0000","last_active":"2024-03-08T06:00:43.251835+0000","last_peered":"2024-03-08T06:00:43.251835+0000","last_clean":"2024-03-08T06:00:43.251835+0000","last_became_active":"2024-03-08T06:00:36.196750+0000","last_became_peered":"2024-03-08T06:00:36.196750+0000","last_unstale":"2024-03-08T06:00:43.251835+0000","last_undegraded":"2024-03-08T06:00:43.251835+0000","last_fullsized":"2024-03-08T06:00:43.251835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:45:22.429641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251708+0000","last_change":"2024-03-08T06:00:36.184956+0000","last_active":"2024-03-08T06:00:43.251708+0000","last_peered":"2024-03-08T06:00:43.251708+0000","last_clean":"2024-03-08T06:00:43.251708+0000","last_became_active":"2024-03-08T06:00:36.184694+0000","last_became_peered":"2024-03-08T06:00:36.184694+0000","last_unstale":"2024-03-08T06:00:43.251708+0000","last_undegraded":"2024-03-08T06:00:43.251708+0000","last_fullsized":"2024-03-08T06:00:43.251708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:41:02.118287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254526+0000","last_change":"2024-03-08T06:00:36.187667+0000","last_active":"2024-03-08T06:00:43.254526+0000","last_peered":"2024-03-08T06:00:43.254526+0000","last_clean":"2024-03-08T06:00:43.254526+0000","last_became_active":"2024-03-08T06:00:36.187407+0000","last_became_peered":"2024-03-08T06:00:36.187407+0000","last_unstale":"2024-03-08T06:00:43.254526+0000","last_undegraded":"2024-03-08T06:00:43.254526+0000","last_fullsized":"2024-03-08T06:00:43.254526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:10:17.180499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752121+0000","last_change":"2024-03-08T06:00:36.198182+0000","last_active":"2024-03-08T06:00:42.752121+0000","last_peered":"2024-03-08T06:00:42.752121+0000","last_clean":"2024-03-08T06:00:42.752121+0000","last_became_active":"2024-03-08T06:00:36.197894+0000","last_became_peered":"2024-03-08T06:00:36.197894+0000","last_unstale":"2024-03-08T06:00:42.752121+0000","last_undegraded":"2024-03-08T06:00:42.752121+0000","last_fullsized":"2024-03-08T06:00:42.752121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:02.331109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253403+0000","last_change":"2024-03-08T06:00:36.180664+0000","last_active":"2024-03-08T06:00:43.253403+0000","last_peered":"2024-03-08T06:00:43.253403+0000","last_clean":"2024-03-08T06:00:43.253403+0000","last_became_active":"2024-03-08T06:00:36.180389+0000","last_became_peered":"2024-03-08T06:00:36.180389+0000","last_unstale":"2024-03-08T06:00:43.253403+0000","last_undegraded":"2024-03-08T06:00:43.253403+0000","last_fullsized":"2024-03-08T06:00:43.253403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:58:34.120247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252430+0000","last_change":"2024-03-08T06:00:36.189412+0000","last_active":"2024-03-08T06:00:43.252430+0000","last_peered":"2024-03-08T06:00:43.252430+0000","last_clean":"2024-03-08T06:00:43.252430+0000","last_became_active":"2024-03-08T06:00:36.187130+0000","last_became_peered":"2024-03-08T06:00:36.187130+0000","last_unstale":"2024-03-08T06:00:43.252430+0000","last_undegraded":"2024-03-08T06:00:43.252430+0000","last_fullsized":"2024-03-08T06:00:43.252430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:05.458469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251511+0000","last_change":"2024-03-08T06:00:36.186949+0000","last_active":"2024-03-08T06:00:43.251511+0000","last_peered":"2024-03-08T06:00:43.251511+0000","last_clean":"2024-03-08T06:00:43.251511+0000","last_became_active":"2024-03-08T06:00:36.186386+0000","last_became_peered":"2024-03-08T06:00:36.186386+0000","last_unstale":"2024-03-08T06:00:43.251511+0000","last_undegraded":"2024-03-08T06:00:43.251511+0000","last_fullsized":"2024-03-08T06:00:43.251511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251280+0000","last_change":"2024-03-08T06:00:36.178773+0000","last_active":"2024-03-08T06:00:43.251280+0000","last_peered":"2024-03-08T06:00:43.251280+0000","last_clean":"2024-03-08T06:00:43.251280+0000","last_became_active":"2024-03-08T06:00:36.178515+0000","last_became_peered":"2024-03-08T06:00:36.178515+0000","last_unstale":"2024-03-08T06:00:43.251280+0000","last_undegraded":"2024-03-08T06:00:43.251280+0000","last_fullsized":"2024-03-08T06:00:43.251280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:11:34.950063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251635+0000","last_change":"2024-03-08T06:00:36.179334+0000","last_active":"2024-03-08T06:00:43.251635+0000","last_peered":"2024-03-08T06:00:43.251635+0000","last_clean":"2024-03-08T06:00:43.251635+0000","last_became_active":"2024-03-08T06:00:36.179056+0000","last_became_peered":"2024-03-08T06:00:36.179056+0000","last_unstale":"2024-03-08T06:00:43.251635+0000","last_undegraded":"2024-03-08T06:00:43.251635+0000","last_fullsized":"2024-03-08T06:00:43.251635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:13.003781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205268+0000","last_change":"2024-03-08T06:00:36.182864+0000","last_active":"2024-03-08T06:00:42.205268+0000","last_peered":"2024-03-08T06:00:42.205268+0000","last_clean":"2024-03-08T06:00:42.205268+0000","last_became_active":"2024-03-08T06:00:36.182290+0000","last_became_peered":"2024-03-08T06:00:36.182290+0000","last_unstale":"2024-03-08T06:00:42.205268+0000","last_undegraded":"2024-03-08T06:00:42.205268+0000","last_fullsized":"2024-03-08T06:00:42.205268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206446+0000","last_change":"2024-03-08T06:00:36.184044+0000","last_active":"2024-03-08T06:00:42.206446+0000","last_peered":"2024-03-08T06:00:42.206446+0000","last_clean":"2024-03-08T06:00:42.206446+0000","last_became_active":"2024-03-08T06:00:36.183745+0000","last_became_peered":"2024-03-08T06:00:36.183745+0000","last_unstale":"2024-03-08T06:00:42.206446+0000","last_undegraded":"2024-03-08T06:00:42.206446+0000","last_fullsized":"2024-03-08T06:00:42.206446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:58:34.120247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254652+0000","last_change":"2024-03-08T06:00:36.200379+0000","last_active":"2024-03-08T06:00:43.254652+0000","last_peered":"2024-03-08T06:00:43.254652+0000","last_clean":"2024-03-08T06:00:43.254652+0000","last_became_active":"2024-03-08T06:00:36.200023+0000","last_became_peered":"2024-03-08T06:00:36.200023+0000","last_unstale":"2024-03-08T06:00:43.254652+0000","last_undegraded":"2024-03-08T06:00:43.254652+0000","last_fullsized":"2024-03-08T06:00:43.254652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:40:16.053974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254844+0000","last_change":"2024-03-08T06:00:36.200329+0000","last_active":"2024-03-08T06:00:43.254844+0000","last_peered":"2024-03-08T06:00:43.254844+0000","last_clean":"2024-03-08T06:00:43.254844+0000","last_became_active":"2024-03-08T06:00:36.200069+0000","last_became_peered":"2024-03-08T06:00:36.200069+0000","last_unstale":"2024-03-08T06:00:43.254844+0000","last_undegraded":"2024-03-08T06:00:43.254844+0000","last_fullsized":"2024-03-08T06:00:43.254844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751472+0000","last_change":"2024-03-08T06:00:36.205202+0000","last_active":"2024-03-08T06:00:42.751472+0000","last_peered":"2024-03-08T06:00:42.751472+0000","last_clean":"2024-03-08T06:00:42.751472+0000","last_became_active":"2024-03-08T06:00:36.204866+0000","last_became_peered":"2024-03-08T06:00:36.204866+0000","last_unstale":"2024-03-08T06:00:42.751472+0000","last_undegraded":"2024-03-08T06:00:42.751472+0000","last_fullsized":"2024-03-08T06:00:42.751472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:44:44.951176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.204931+0000","last_change":"2024-03-08T06:00:36.188976+0000","last_active":"2024-03-08T06:00:42.204931+0000","last_peered":"2024-03-08T06:00:42.204931+0000","last_clean":"2024-03-08T06:00:42.204931+0000","last_became_active":"2024-03-08T06:00:36.188717+0000","last_became_peered":"2024-03-08T06:00:36.188717+0000","last_unstale":"2024-03-08T06:00:42.204931+0000","last_undegraded":"2024-03-08T06:00:42.204931+0000","last_fullsized":"2024-03-08T06:00:42.204931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:11:34.950063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751727+0000","last_change":"2024-03-08T06:00:36.197547+0000","last_active":"2024-03-08T06:00:42.751727+0000","last_peered":"2024-03-08T06:00:42.751727+0000","last_clean":"2024-03-08T06:00:42.751727+0000","last_became_active":"2024-03-08T06:00:36.197295+0000","last_became_peered":"2024-03-08T06:00:36.197295+0000","last_unstale":"2024-03-08T06:00:42.751727+0000","last_undegraded":"2024-03-08T06:00:42.751727+0000","last_fullsized":"2024-03-08T06:00:42.751727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:39:59.986300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251773+0000","last_change":"2024-03-08T06:00:36.191642+0000","last_active":"2024-03-08T06:00:43.251773+0000","last_peered":"2024-03-08T06:00:43.251773+0000","last_clean":"2024-03-08T06:00:43.251773+0000","last_became_active":"2024-03-08T06:00:36.191242+0000","last_became_peered":"2024-03-08T06:00:36.191242+0000","last_unstale":"2024-03-08T06:00:43.251773+0000","last_undegraded":"2024-03-08T06:00:43.251773+0000","last_fullsized":"2024-03-08T06:00:43.251773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751820+0000","last_change":"2024-03-08T06:00:36.197506+0000","last_active":"2024-03-08T06:00:42.751820+0000","last_peered":"2024-03-08T06:00:42.751820+0000","last_clean":"2024-03-08T06:00:42.751820+0000","last_became_active":"2024-03-08T06:00:36.197208+0000","last_became_peered":"2024-03-08T06:00:36.197208+0000","last_unstale":"2024-03-08T06:00:42.751820+0000","last_undegraded":"2024-03-08T06:00:42.751820+0000","last_fullsized":"2024-03-08T06:00:42.751820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:59:39.518589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252005+0000","last_change":"2024-03-08T06:00:36.186669+0000","last_active":"2024-03-08T06:00:43.252005+0000","last_peered":"2024-03-08T06:00:43.252005+0000","last_clean":"2024-03-08T06:00:43.252005+0000","last_became_active":"2024-03-08T06:00:36.186417+0000","last_became_peered":"2024-03-08T06:00:36.186417+0000","last_unstale":"2024-03-08T06:00:43.252005+0000","last_undegraded":"2024-03-08T06:00:43.252005+0000","last_fullsized":"2024-03-08T06:00:43.252005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:59:39.518589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251432+0000","last_change":"2024-03-08T06:00:36.186124+0000","last_active":"2024-03-08T06:00:43.251432+0000","last_peered":"2024-03-08T06:00:43.251432+0000","last_clean":"2024-03-08T06:00:43.251432+0000","last_became_active":"2024-03-08T06:00:36.185808+0000","last_became_peered":"2024-03-08T06:00:36.185808+0000","last_unstale":"2024-03-08T06:00:43.251432+0000","last_undegraded":"2024-03-08T06:00:43.251432+0000","last_fullsized":"2024-03-08T06:00:43.251432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:01:37.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252630+0000","last_change":"2024-03-08T06:00:36.183150+0000","last_active":"2024-03-08T06:00:43.252630+0000","last_peered":"2024-03-08T06:00:43.252630+0000","last_clean":"2024-03-08T06:00:43.252630+0000","last_became_active":"2024-03-08T06:00:36.182716+0000","last_became_peered":"2024-03-08T06:00:36.182716+0000","last_unstale":"2024-03-08T06:00:43.252630+0000","last_undegraded":"2024-03-08T06:00:43.252630+0000","last_fullsized":"2024-03-08T06:00:43.252630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752346+0000","last_change":"2024-03-08T06:00:36.198478+0000","last_active":"2024-03-08T06:00:42.752346+0000","last_peered":"2024-03-08T06:00:42.752346+0000","last_clean":"2024-03-08T06:00:42.752346+0000","last_became_active":"2024-03-08T06:00:36.198218+0000","last_became_peered":"2024-03-08T06:00:36.198218+0000","last_unstale":"2024-03-08T06:00:42.752346+0000","last_undegraded":"2024-03-08T06:00:42.752346+0000","last_fullsized":"2024-03-08T06:00:42.752346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:19:26.058130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":876544,"data_stored":703400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":116292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":112540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":112576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":116292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":112450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":112610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":703456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:00:59.556 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-08T06:00:59.556 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-08T06:00:59.556 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-08T06:00:59.556 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-08T06:00:59.557 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-08T06:00:59.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.692+0000 7fa916207640 1 Processor -- start 2024-03-08T06:00:59.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.692+0000 7fa916207640 1 -- start start 2024-03-08T06:00:59.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.692+0000 7fa916207640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa9100d3710 0x7fa910168a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.692+0000 7fa916207640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91015ef30 con 0x7fa91015eab0 2024-03-08T06:00:59.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.692+0000 7fa916207640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91015f210 con 0x7fa91015dfc0 2024-03-08T06:00:59.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.692+0000 7fa916207640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9100b6c10 con 0x7fa9100d3710 2024-03-08T06:00:59.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.692+0000 7fa906ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa91015eab0 0x7fa91006c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57466/0 (socket says 172.21.15.76:57466) 2024-03-08T06:00:59.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.692+0000 7fa906ffd640 1 -- 172.21.15.76:0/659027672 learned_addr learned my addr 172.21.15.76:0/659027672 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:00:59.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.693+0000 7fa90ffff640 1 --2- 172.21.15.76:0/659027672 >> v2:172.21.15.81:3300/0 conn(0x7fa9100d3710 0x7fa910168a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.693+0000 7fa90effd640 1 -- 172.21.15.76:0/659027672 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171936082 0 0) 0x7fa91015ef30 con 0x7fa91015eab0 2024-03-08T06:00:59.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.693+0000 7fa90effd640 1 -- 172.21.15.76:0/659027672 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8fc003520 con 0x7fa91015eab0 2024-03-08T06:00:59.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.693+0000 7fa90ffff640 1 -- 172.21.15.76:0/659027672 >> v1:172.21.15.81:6789/0 conn(0x7fa91015dfc0 legacy=0x7fa91015e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.693+0000 7fa90ffff640 1 -- 172.21.15.76:0/659027672 >> v1:172.21.15.76:6789/0 conn(0x7fa91015eab0 legacy=0x7fa91006c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.693+0000 7fa90ffff640 1 -- 172.21.15.76:0/659027672 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9100d1f00 con 0x7fa9100d3710 2024-03-08T06:00:59.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.694+0000 7fa90ffff640 1 --2- 172.21.15.76:0/659027672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9100d3710 0x7fa910168a50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa908009d90 tx=0x7fa90802bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=503ac353c2f1cb2d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:59.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.694+0000 7fa90effd640 1 -- 172.21.15.76:0/659027672 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908039070 con 0x7fa9100d3710 2024-03-08T06:00:59.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.694+0000 7fa90effd640 1 -- 172.21.15.76:0/659027672 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90802cd80 con 0x7fa9100d3710 2024-03-08T06:00:59.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.694+0000 7fa90effd640 1 -- 172.21.15.76:0/659027672 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908034640 con 0x7fa9100d3710 2024-03-08T06:00:59.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.694+0000 7fa916207640 1 -- 172.21.15.76:0/659027672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9100d3710 msgr2=0x7fa910168a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.694+0000 7fa916207640 1 --2- 172.21.15.76:0/659027672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9100d3710 0x7fa910168a50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa908009d90 tx=0x7fa90802bfd0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.694+0000 7fa916207640 1 -- 172.21.15.76:0/659027672 shutdown_connections 2024-03-08T06:00:59.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.694+0000 7fa916207640 1 --2- 172.21.15.76:0/659027672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9100d3710 0x7fa910168a50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.694+0000 7fa916207640 1 -- 172.21.15.76:0/659027672 >> 172.21.15.76:0/659027672 conn(0x7fa910066de0 msgr2=0x7fa9100671f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:59.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.694+0000 7fa916207640 1 -- 172.21.15.76:0/659027672 shutdown_connections 2024-03-08T06:00:59.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.694+0000 7fa916207640 1 -- 172.21.15.76:0/659027672 wait complete. 2024-03-08T06:00:59.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.696+0000 7fa916207640 1 Processor -- start 2024-03-08T06:00:59.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.696+0000 7fa916207640 1 -- start start 2024-03-08T06:00:59.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.696+0000 7fa916207640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9100d3710 0x7fa9100cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.696+0000 7fa916207640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa91015dfc0 0x7fa9100abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.696+0000 7fa916207640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa91015eab0 0x7fa9100b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.696+0000 7fa916207640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91006c920 con 0x7fa9100d3710 2024-03-08T06:00:59.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.696+0000 7fa916207640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100b9720 con 0x7fa91015eab0 2024-03-08T06:00:59.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.696+0000 7fa916207640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91006caa0 con 0x7fa91015dfc0 2024-03-08T06:00:59.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa90f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9100d3710 0x7fa9100cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa90f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9100d3710 0x7fa9100cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51248/0 (socket says 172.21.15.76:51248) 2024-03-08T06:00:59.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa90f7fe640 1 -- 172.21.15.76:0/1366990978 learned_addr learned my addr 172.21.15.76:0/1366990978 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:00:59.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa90ffff640 1 --2- 172.21.15.76:0/1366990978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa91015eab0 0x7fa9100b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa906ffd640 1 --2- 172.21.15.76:0/1366990978 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa91015dfc0 0x7fa9100abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa90f7fe640 1 -- 172.21.15.76:0/1366990978 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa91015dfc0 msgr2=0x7fa9100abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa90f7fe640 1 --2- 172.21.15.76:0/1366990978 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa91015dfc0 0x7fa9100abc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa90f7fe640 1 -- 172.21.15.76:0/1366990978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa91015eab0 msgr2=0x7fa9100b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa90f7fe640 1 --2- 172.21.15.76:0/1366990978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa91015eab0 0x7fa9100b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa90f7fe640 1 -- 172.21.15.76:0/1366990978 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9100ac4c0 con 0x7fa9100d3710 2024-03-08T06:00:59.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa90ffff640 1 --2- 172.21.15.76:0/1366990978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa91015eab0 0x7fa9100b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:59.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa906ffd640 1 --2- 172.21.15.76:0/1366990978 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa91015dfc0 0x7fa9100abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:00:59.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa90f7fe640 1 --2- 172.21.15.76:0/1366990978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9100d3710 0x7fa9100cc910 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa90000c430 tx=0x7fa90000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:59.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa90cff9640 1 -- 172.21.15.76:0/1366990978 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9000052d0 con 0x7fa9100d3710 2024-03-08T06:00:59.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa90cff9640 1 -- 172.21.15.76:0/1366990978 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa900005490 con 0x7fa9100d3710 2024-03-08T06:00:59.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa916207640 1 -- 172.21.15.76:0/1366990978 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9100ac7a0 con 0x7fa9100d3710 2024-03-08T06:00:59.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa90cff9640 1 -- 172.21.15.76:0/1366990978 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900007260 con 0x7fa9100d3710 2024-03-08T06:00:59.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.697+0000 7fa916207640 1 -- 172.21.15.76:0/1366990978 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9100acd60 con 0x7fa9100d3710 2024-03-08T06:00:59.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.698+0000 7fa916207640 1 -- 172.21.15.76:0/1366990978 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91004f030 con 0x7fa9100d3710 2024-03-08T06:00:59.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.699+0000 7fa90cff9640 1 -- 172.21.15.76:0/1366990978 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa900007400 con 0x7fa9100d3710 2024-03-08T06:00:59.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.699+0000 7fa90cff9640 1 --2- 172.21.15.76:0/1366990978 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa8f006b4d0 0x7fa8f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:00:59.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.699+0000 7fa90cff9640 1 -- 172.21.15.76:0/1366990978 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa900089a60 con 0x7fa9100d3710 2024-03-08T06:00:59.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.699+0000 7fa906ffd640 1 --2- 172.21.15.76:0/1366990978 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa8f006b4d0 0x7fa8f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:00:59.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.700+0000 7fa906ffd640 1 --2- 172.21.15.76:0/1366990978 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa8f006b4d0 0x7fa8f006d990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8007fd0 tx=0x7fa8f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:00:59.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.703+0000 7fa90cff9640 1 -- 172.21.15.76:0/1366990978 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9000585b0 con 0x7fa9100d3710 2024-03-08T06:00:59.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.917+0000 7fa916207640 1 -- 172.21.15.76:0/1366990978 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa9100b9390 con 0x7fa9100d3710 2024-03-08T06:00:59.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.920+0000 7fa90cff9640 1 -- 172.21.15.76:0/1366990978 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171778 (secure 0 0 0) 0x7fa900057eb0 con 0x7fa9100d3710 2024-03-08T06:00:59.924 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:00:59.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.924+0000 7fa916207640 1 -- 172.21.15.76:0/1366990978 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa8f006b4d0 msgr2=0x7fa8f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.924+0000 7fa916207640 1 --2- 172.21.15.76:0/1366990978 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa8f006b4d0 0x7fa8f006d990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8007fd0 tx=0x7fa8f8009040 comp rx=0 tx=0).stop 2024-03-08T06:00:59.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.924+0000 7fa916207640 1 -- 172.21.15.76:0/1366990978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9100d3710 msgr2=0x7fa9100cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:00:59.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.924+0000 7fa916207640 1 --2- 172.21.15.76:0/1366990978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9100d3710 0x7fa9100cc910 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa90000c430 tx=0x7fa90000c8f0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.924+0000 7fa916207640 1 -- 172.21.15.76:0/1366990978 shutdown_connections 2024-03-08T06:00:59.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.924+0000 7fa916207640 1 --2- 172.21.15.76:0/1366990978 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa8f006b4d0 0x7fa8f006d990 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.924+0000 7fa916207640 1 --2- 172.21.15.76:0/1366990978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa91015eab0 0x7fa9100b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.924+0000 7fa916207640 1 --2- 172.21.15.76:0/1366990978 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa91015dfc0 0x7fa9100abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.924+0000 7fa916207640 1 --2- 172.21.15.76:0/1366990978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9100d3710 0x7fa9100cc910 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:00:59.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.924+0000 7fa916207640 1 -- 172.21.15.76:0/1366990978 >> 172.21.15.76:0/1366990978 conn(0x7fa910066de0 msgr2=0x7fa9100b8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:00:59.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.924+0000 7fa916207640 1 -- 172.21.15.76:0/1366990978 shutdown_connections 2024-03-08T06:00:59.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:00:59.924+0000 7fa916207640 1 -- 172.21.15.76:0/1366990978 wait complete. 2024-03-08T06:00:59.935 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":754497070},{"type":"v1","addr":"172.21.15.76:6833","nonce":754497070}]},"active_addr":"172.21.15.76:6833/754497070","active_change":"2024-03-08T06:00:15.912042+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":4262359870}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":957341329}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":59145125}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":486005638}]}]} 2024-03-08T06:00:59.938 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-08T06:00:59.939 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-08T06:00:59.939 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:00.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.036+0000 7f047c6fb640 1 Processor -- start 2024-03-08T06:01:00.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.036+0000 7f047c6fb640 1 -- start start 2024-03-08T06:01:00.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.037+0000 7f047c6fb640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f047415ab60 0x7f04740cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.037+0000 7f047c6fb640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f04740cbf00 0x7f04740cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.037+0000 7f047c6fb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04740b0340 con 0x7f04740b0db0 2024-03-08T06:01:00.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.037+0000 7f047c6fb640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04740cc4c0 con 0x7f04740cbf00 2024-03-08T06:01:00.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.037+0000 7f047c6fb640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04740b05b0 con 0x7f047415ab60 2024-03-08T06:01:00.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f047ac98640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f04740b0db0 0x7f04740b11c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57482/0 (socket says 172.21.15.76:57482) 2024-03-08T06:01:00.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f047ac98640 1 -- 172.21.15.76:0/3630917082 learned_addr learned my addr 172.21.15.76:0/3630917082 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:00.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479c96640 1 --2- 172.21.15.76:0/3630917082 >> v2:172.21.15.81:3300/0 conn(0x7f04740cbf00 0x7f04740cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f047a497640 1 --2- 172.21.15.76:0/3630917082 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f047415ab60 0x7f04740cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479495640 1 -- 172.21.15.76:0/3630917082 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910228874 0 0) 0x7f04740b0340 con 0x7f04740b0db0 2024-03-08T06:01:00.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479495640 1 -- 172.21.15.76:0/3630917082 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0460003520 con 0x7f04740b0db0 2024-03-08T06:01:00.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479495640 1 -- 172.21.15.76:0/3630917082 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 76646471 0 0) 0x7f0460003520 con 0x7f04740b0db0 2024-03-08T06:01:00.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479495640 1 -- 172.21.15.76:0/3630917082 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f047415ab60 msgr2=0x7f04740cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479495640 1 --2- 172.21.15.76:0/3630917082 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f047415ab60 0x7f04740cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479495640 1 -- 172.21.15.76:0/3630917082 >> v2:172.21.15.81:3300/0 conn(0x7f04740cbf00 msgr2=0x7f04740cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479495640 1 --2- 172.21.15.76:0/3630917082 >> v2:172.21.15.81:3300/0 conn(0x7f04740cbf00 0x7f04740cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479495640 1 -- 172.21.15.76:0/3630917082 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04740b0890 con 0x7f04740b0db0 2024-03-08T06:01:00.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479495640 1 -- 172.21.15.76:0/3630917082 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f046c001670 con 0x7f04740b0db0 2024-03-08T06:01:00.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479495640 1 -- 172.21.15.76:0/3630917082 >> v1:172.21.15.76:6789/0 conn(0x7f04740b0db0 legacy=0x7f04740b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479495640 1 --2- 172.21.15.76:0/3630917082 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f047415ab60 0x7f0460003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479495640 1 --2- 172.21.15.76:0/3630917082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0460003fa0 0x7f0460004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479495640 1 --2- 172.21.15.76:0/3630917082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0460004d40 0x7f04600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479495640 1 -- 172.21.15.76:0/3630917082 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04740cc4c0 con 0x7f0460003fa0 2024-03-08T06:01:00.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479495640 1 -- 172.21.15.76:0/3630917082 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04740b05b0 con 0x7f0460004d40 2024-03-08T06:01:00.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479495640 1 -- 172.21.15.76:0/3630917082 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046000b9f0 con 0x7f047415ab60 2024-03-08T06:01:00.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f0479c96640 1 --2- 172.21.15.76:0/3630917082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0460003fa0 0x7f0460004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f047a497640 1 --2- 172.21.15.76:0/3630917082 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f047415ab60 0x7f0460003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.038+0000 7f047ac98640 1 --2- 172.21.15.76:0/3630917082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0460004d40 0x7f04600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.039+0000 7f0479c96640 1 -- 172.21.15.76:0/3630917082 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f047415ab60 msgr2=0x7f0460003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.039+0000 7f0479c96640 1 --2- 172.21.15.76:0/3630917082 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f047415ab60 0x7f0460003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.039+0000 7f0479c96640 1 -- 172.21.15.76:0/3630917082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0460004d40 msgr2=0x7f04600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.039+0000 7f0479c96640 1 --2- 172.21.15.76:0/3630917082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0460004d40 0x7f04600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.039+0000 7f0479c96640 1 -- 172.21.15.76:0/3630917082 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0460003520 con 0x7f0460003fa0 2024-03-08T06:01:00.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.039+0000 7f0479c96640 1 --2- 172.21.15.76:0/3630917082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0460003fa0 0x7f0460004800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0464009ac0 tx=0x7f0460005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5214d9ec658c5ea8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.039+0000 7f0479495640 1 -- 172.21.15.76:0/3630917082 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0464039070 con 0x7f0460003fa0 2024-03-08T06:01:00.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.039+0000 7f0479495640 1 -- 172.21.15.76:0/3630917082 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046402c710 con 0x7f0460003fa0 2024-03-08T06:01:00.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.039+0000 7f0479495640 1 -- 172.21.15.76:0/3630917082 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0464009690 con 0x7f0460003fa0 2024-03-08T06:01:00.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.040+0000 7f047c6fb640 1 -- 172.21.15.76:0/3630917082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0460003fa0 msgr2=0x7f0460004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.040+0000 7f047c6fb640 1 --2- 172.21.15.76:0/3630917082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0460003fa0 0x7f0460004800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0464009ac0 tx=0x7f0460005830 comp rx=0 tx=0).stop 2024-03-08T06:01:00.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.040+0000 7f047ac98640 1 -- 172.21.15.76:0/3630917082 reap_dead start 2024-03-08T06:01:00.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.040+0000 7f047c6fb640 1 -- 172.21.15.76:0/3630917082 shutdown_connections 2024-03-08T06:01:00.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.040+0000 7f047c6fb640 1 -- 172.21.15.76:0/3630917082 >> 172.21.15.76:0/3630917082 conn(0x7f0474099a60 msgr2=0x7f04740b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.040+0000 7f047c6fb640 1 -- 172.21.15.76:0/3630917082 shutdown_connections 2024-03-08T06:01:00.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.040+0000 7f047c6fb640 1 -- 172.21.15.76:0/3630917082 wait complete. 2024-03-08T06:01:00.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.041+0000 7f047c6fb640 1 Processor -- start 2024-03-08T06:01:00.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.041+0000 7f047c6fb640 1 -- start start 2024-03-08T06:01:00.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f047c6fb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0460003fa0 0x7f04740ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f047c6fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04740b5a40 0x7f047406ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f047c6fb640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f047406b1c0 0x7f0474143c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f047c6fb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047415af40 con 0x7f04740b5a40 2024-03-08T06:01:00.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f047c6fb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04740cc7a0 con 0x7f0460003fa0 2024-03-08T06:01:00.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f047c6fb640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047415b0c0 con 0x7f047406b1c0 2024-03-08T06:01:00.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f0479c96640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04740b5a40 0x7f047406ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f0479c96640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04740b5a40 0x7f047406ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51266/0 (socket says 172.21.15.76:51266) 2024-03-08T06:01:00.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f0479c96640 1 -- 172.21.15.76:0/636517029 learned_addr learned my addr 172.21.15.76:0/636517029 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:00.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f047ac98640 1 --2- 172.21.15.76:0/636517029 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f047406b1c0 0x7f0474143c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f047a497640 1 --2- 172.21.15.76:0/636517029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0460003fa0 0x7f04740ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f0479c96640 1 -- 172.21.15.76:0/636517029 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f047406b1c0 msgr2=0x7f0474143c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f0479c96640 1 --2- 172.21.15.76:0/636517029 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f047406b1c0 0x7f0474143c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f0479c96640 1 -- 172.21.15.76:0/636517029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0460003fa0 msgr2=0x7f04740ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f0479c96640 1 --2- 172.21.15.76:0/636517029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0460003fa0 0x7f04740ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f0479c96640 1 -- 172.21.15.76:0/636517029 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04741449b0 con 0x7f04740b5a40 2024-03-08T06:01:00.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f047a497640 1 --2- 172.21.15.76:0/636517029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0460003fa0 0x7f04740ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:00.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.042+0000 7f0479c96640 1 --2- 172.21.15.76:0/636517029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04740b5a40 0x7f047406ac80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0464037040 tx=0x7f0464004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.043+0000 7f046affd640 1 -- 172.21.15.76:0/636517029 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0464042020 con 0x7f04740b5a40 2024-03-08T06:01:00.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.043+0000 7f046affd640 1 -- 172.21.15.76:0/636517029 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046403a040 con 0x7f04740b5a40 2024-03-08T06:01:00.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.043+0000 7f047c6fb640 1 -- 172.21.15.76:0/636517029 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04740c99c0 con 0x7f04740b5a40 2024-03-08T06:01:00.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.043+0000 7f046affd640 1 -- 172.21.15.76:0/636517029 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0464039070 con 0x7f04740b5a40 2024-03-08T06:01:00.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.043+0000 7f047c6fb640 1 -- 172.21.15.76:0/636517029 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04740c9f80 con 0x7f04740b5a40 2024-03-08T06:01:00.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.043+0000 7f047c6fb640 1 -- 172.21.15.76:0/636517029 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04740c83c0 con 0x7f04740b5a40 2024-03-08T06:01:00.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.044+0000 7f046affd640 1 -- 172.21.15.76:0/636517029 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0464045020 con 0x7f04740b5a40 2024-03-08T06:01:00.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.044+0000 7f046affd640 1 --2- 172.21.15.76:0/636517029 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f045806b4a0 0x7f045806d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.044+0000 7f046affd640 1 -- 172.21.15.76:0/636517029 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f04640aba70 con 0x7f04740b5a40 2024-03-08T06:01:00.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.044+0000 7f047a497640 1 --2- 172.21.15.76:0/636517029 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f045806b4a0 0x7f045806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.045+0000 7f047a497640 1 --2- 172.21.15.76:0/636517029 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f045806b4a0 0x7f045806d960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0470013370 tx=0x7f047000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.049+0000 7f046affd640 1 -- 172.21.15.76:0/636517029 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f04740c83c0 con 0x7f04740b5a40 2024-03-08T06:01:00.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.224+0000 7f047c6fb640 1 -- 172.21.15.76:0/636517029 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f047404f030 con 0x7f04740b5a40 2024-03-08T06:01:00.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.225+0000 7f046affd640 1 -- 172.21.15.76:0/636517029 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f046404c070 con 0x7f04740b5a40 2024-03-08T06:01:00.226 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:00.227 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":26,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:42.202011+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:00.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.227+0000 7f047c6fb640 1 -- 172.21.15.76:0/636517029 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f045806b4a0 msgr2=0x7f045806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.227+0000 7f047c6fb640 1 --2- 172.21.15.76:0/636517029 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f045806b4a0 0x7f045806d960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0470013370 tx=0x7f047000b040 comp rx=0 tx=0).stop 2024-03-08T06:01:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.227+0000 7f047c6fb640 1 -- 172.21.15.76:0/636517029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04740b5a40 msgr2=0x7f047406ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.227+0000 7f047c6fb640 1 --2- 172.21.15.76:0/636517029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04740b5a40 0x7f047406ac80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0464037040 tx=0x7f0464004730 comp rx=0 tx=0).stop 2024-03-08T06:01:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.227+0000 7f047c6fb640 1 -- 172.21.15.76:0/636517029 shutdown_connections 2024-03-08T06:01:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.227+0000 7f047c6fb640 1 --2- 172.21.15.76:0/636517029 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f045806b4a0 0x7f045806d960 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.227+0000 7f047c6fb640 1 --2- 172.21.15.76:0/636517029 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f047406b1c0 0x7f0474143c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.227+0000 7f047c6fb640 1 --2- 172.21.15.76:0/636517029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04740b5a40 0x7f047406ac80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.227+0000 7f047c6fb640 1 --2- 172.21.15.76:0/636517029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0460003fa0 0x7f04740ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.227+0000 7f047c6fb640 1 -- 172.21.15.76:0/636517029 >> 172.21.15.76:0/636517029 conn(0x7f0474099a60 msgr2=0x7f04740c6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.227+0000 7f047c6fb640 1 -- 172.21.15.76:0/636517029 shutdown_connections 2024-03-08T06:01:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.227+0000 7f047c6fb640 1 -- 172.21.15.76:0/636517029 wait complete. 2024-03-08T06:01:00.239 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-08T06:01:00.239 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:00.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.340+0000 7f60f646f640 1 Processor -- start 2024-03-08T06:01:00.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.340+0000 7f60f646f640 1 -- start start 2024-03-08T06:01:00.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60f646f640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f60f006ac80 0x7f60f00cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60f646f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f00d0150 con 0x7f60f006eee0 2024-03-08T06:01:00.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60f646f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f00d0430 con 0x7f60f00af9c0 2024-03-08T06:01:00.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60f646f640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f006e6e0 con 0x7f60f006ac80 2024-03-08T06:01:00.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60f4a0c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f60f006eee0 0x7f60f00caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57490/0 (socket says 172.21.15.76:57490) 2024-03-08T06:01:00.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60f4a0c640 1 -- 172.21.15.76:0/3529186650 learned_addr learned my addr 172.21.15.76:0/3529186650 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:00.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60effff640 1 --2- 172.21.15.76:0/3529186650 >> v2:172.21.15.81:3300/0 conn(0x7f60f006ac80 0x7f60f00cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60eeffd640 1 -- 172.21.15.76:0/3529186650 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49560636 0 0) 0x7f60f00d0150 con 0x7f60f006eee0 2024-03-08T06:01:00.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60eeffd640 1 -- 172.21.15.76:0/3529186650 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60d8003520 con 0x7f60f006eee0 2024-03-08T06:01:00.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60eeffd640 1 -- 172.21.15.76:0/3529186650 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 550576949 0 0) 0x7f60d8003520 con 0x7f60f006eee0 2024-03-08T06:01:00.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60eeffd640 1 -- 172.21.15.76:0/3529186650 >> v2:172.21.15.81:3300/0 conn(0x7f60f006ac80 msgr2=0x7f60f00cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60eeffd640 1 --2- 172.21.15.76:0/3529186650 >> v2:172.21.15.81:3300/0 conn(0x7f60f006ac80 0x7f60f00cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60eeffd640 1 -- 172.21.15.76:0/3529186650 >> v1:172.21.15.81:6789/0 conn(0x7f60f00af9c0 legacy=0x7f60f006b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60eeffd640 1 -- 172.21.15.76:0/3529186650 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f006e9c0 con 0x7f60f006eee0 2024-03-08T06:01:00.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60eeffd640 1 -- 172.21.15.76:0/3529186650 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f60e80019f0 con 0x7f60f006eee0 2024-03-08T06:01:00.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60eeffd640 1 -- 172.21.15.76:0/3529186650 >> v1:172.21.15.76:6789/0 conn(0x7f60f006eee0 legacy=0x7f60f00caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60eeffd640 1 --2- 172.21.15.76:0/3529186650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60d8003a20 0x7f60d8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60eeffd640 1 --2- 172.21.15.76:0/3529186650 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f60d8004800 0x7f60d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60eeffd640 1 --2- 172.21.15.76:0/3529186650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60d80051c0 0x7f60d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60eeffd640 1 -- 172.21.15.76:0/3529186650 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f006e6e0 con 0x7f60d80051c0 2024-03-08T06:01:00.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60eeffd640 1 -- 172.21.15.76:0/3529186650 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d800c0f0 con 0x7f60d8003a20 2024-03-08T06:01:00.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60eeffd640 1 -- 172.21.15.76:0/3529186650 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d800c2f0 con 0x7f60d8004800 2024-03-08T06:01:00.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60effff640 1 --2- 172.21.15.76:0/3529186650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60d8003a20 0x7f60d8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.341+0000 7f60f4a0c640 1 --2- 172.21.15.76:0/3529186650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60d80051c0 0x7f60d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.342+0000 7f60ef7fe640 1 --2- 172.21.15.76:0/3529186650 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f60d8004800 0x7f60d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.342+0000 7f60effff640 1 -- 172.21.15.76:0/3529186650 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f60d8004800 msgr2=0x7f60d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.342+0000 7f60effff640 1 --2- 172.21.15.76:0/3529186650 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f60d8004800 0x7f60d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.342+0000 7f60effff640 1 -- 172.21.15.76:0/3529186650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60d80051c0 msgr2=0x7f60d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.342+0000 7f60effff640 1 --2- 172.21.15.76:0/3529186650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60d80051c0 0x7f60d800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.342+0000 7f60effff640 1 -- 172.21.15.76:0/3529186650 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d8003520 con 0x7f60d8003a20 2024-03-08T06:01:00.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.342+0000 7f60f4a0c640 1 --2- 172.21.15.76:0/3529186650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60d80051c0 0x7f60d800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:00.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.342+0000 7f60f4a0c640 1 -- 172.21.15.76:0/3529186650 reap_dead start 2024-03-08T06:01:00.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.342+0000 7f60effff640 1 --2- 172.21.15.76:0/3529186650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60d8003a20 0x7f60d8003e40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009ac0 tx=0x7f60d8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2962f7489b264235 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.343+0000 7f60eeffd640 1 -- 172.21.15.76:0/3529186650 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0039070 con 0x7f60d8003a20 2024-03-08T06:01:00.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.343+0000 7f60eeffd640 1 -- 172.21.15.76:0/3529186650 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e002c6b0 con 0x7f60d8003a20 2024-03-08T06:01:00.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.343+0000 7f60eeffd640 1 -- 172.21.15.76:0/3529186650 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0009690 con 0x7f60d8003a20 2024-03-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.343+0000 7f60f646f640 1 -- 172.21.15.76:0/3529186650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60d8003a20 msgr2=0x7f60d8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.343+0000 7f60f646f640 1 --2- 172.21.15.76:0/3529186650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60d8003a20 0x7f60d8003e40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009ac0 tx=0x7f60d8005830 comp rx=0 tx=0).stop 2024-03-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.343+0000 7f60f646f640 1 -- 172.21.15.76:0/3529186650 shutdown_connections 2024-03-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.343+0000 7f60f646f640 1 --2- 172.21.15.76:0/3529186650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60d8003a20 0x7f60d8003e40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.343+0000 7f60f646f640 1 -- 172.21.15.76:0/3529186650 >> 172.21.15.76:0/3529186650 conn(0x7f60f0099a60 msgr2=0x7f60f00d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.343+0000 7f60f646f640 1 -- 172.21.15.76:0/3529186650 shutdown_connections 2024-03-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.343+0000 7f60f646f640 1 -- 172.21.15.76:0/3529186650 wait complete. 2024-03-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.344+0000 7f60f646f640 1 Processor -- start 2024-03-08T06:01:00.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.345+0000 7f60f646f640 1 -- start start 2024-03-08T06:01:00.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.345+0000 7f60f646f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60d8003a20 0x7f60f00b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.345+0000 7f60f646f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60f00b36a0 0x7f60f0160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.345+0000 7f60f646f640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f60f014d8b0 0x7f60f014dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.345+0000 7f60f646f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f006f370 con 0x7f60f00b36a0 2024-03-08T06:01:00.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.345+0000 7f60f646f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f015e280 con 0x7f60d8003a20 2024-03-08T06:01:00.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.345+0000 7f60f646f640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f006f4f0 con 0x7f60f014d8b0 2024-03-08T06:01:00.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.345+0000 7f60ef7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60f00b36a0 0x7f60f0160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.345+0000 7f60ef7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60f00b36a0 0x7f60f0160680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51288/0 (socket says 172.21.15.76:51288) 2024-03-08T06:01:00.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.345+0000 7f60ef7fe640 1 -- 172.21.15.76:0/3061410021 learned_addr learned my addr 172.21.15.76:0/3061410021 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:00.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.346+0000 7f60effff640 1 --2- 172.21.15.76:0/3061410021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60d8003a20 0x7f60f00b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.346+0000 7f60ef7fe640 1 -- 172.21.15.76:0/3061410021 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f60f014d8b0 msgr2=0x7f60f014dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.346+0000 7f60f4a0c640 1 --2- 172.21.15.76:0/3061410021 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f60f014d8b0 0x7f60f014dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.346+0000 7f60ef7fe640 1 --2- 172.21.15.76:0/3061410021 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f60f014d8b0 0x7f60f014dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.346+0000 7f60ef7fe640 1 -- 172.21.15.76:0/3061410021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60d8003a20 msgr2=0x7f60f00b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.346+0000 7f60ef7fe640 1 --2- 172.21.15.76:0/3061410021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60d8003a20 0x7f60f00b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.346+0000 7f60ef7fe640 1 -- 172.21.15.76:0/3061410021 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f00b11d0 con 0x7f60f00b36a0 2024-03-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.346+0000 7f60effff640 1 --2- 172.21.15.76:0/3061410021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60d8003a20 0x7f60f00b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.346+0000 7f60ef7fe640 1 --2- 172.21.15.76:0/3061410021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60f00b36a0 0x7f60f0160680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f60e400f5f0 tx=0x7f60e400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.346+0000 7f60ecff9640 1 -- 172.21.15.76:0/3061410021 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e40052e0 con 0x7f60f00b36a0 2024-03-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.346+0000 7f60ecff9640 1 -- 172.21.15.76:0/3061410021 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e4018070 con 0x7f60f00b36a0 2024-03-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.346+0000 7f60ecff9640 1 -- 172.21.15.76:0/3061410021 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e4003810 con 0x7f60f00b36a0 2024-03-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.346+0000 7f60f646f640 1 -- 172.21.15.76:0/3061410021 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f00b3c10 con 0x7f60f00b36a0 2024-03-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.346+0000 7f60f646f640 1 -- 172.21.15.76:0/3061410021 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f00b41d0 con 0x7f60f00b36a0 2024-03-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.347+0000 7f60f646f640 1 -- 172.21.15.76:0/3061410021 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f0161660 con 0x7f60f00b36a0 2024-03-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.347+0000 7f60ecff9640 1 -- 172.21.15.76:0/3061410021 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f60e40039b0 con 0x7f60f00b36a0 2024-03-08T06:01:00.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.348+0000 7f60ecff9640 1 --2- 172.21.15.76:0/3061410021 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f60dc06b500 0x7f60dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.348+0000 7f60ecff9640 1 -- 172.21.15.76:0/3061410021 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f60e408a360 con 0x7f60f00b36a0 2024-03-08T06:01:00.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.348+0000 7f60effff640 1 --2- 172.21.15.76:0/3061410021 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f60dc06b500 0x7f60dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.348+0000 7f60effff640 1 --2- 172.21.15.76:0/3061410021 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f60dc06b500 0x7f60dc06d9c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f60e0036000 tx=0x7f60e0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.352+0000 7f60ecff9640 1 -- 172.21.15.76:0/3061410021 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60f0161660 con 0x7f60f00b36a0 2024-03-08T06:01:00.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.524+0000 7f60f646f640 1 -- 172.21.15.76:0/3061410021 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60f004f030 con 0x7f60f00b36a0 2024-03-08T06:01:00.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.525+0000 7f60ecff9640 1 -- 172.21.15.76:0/3061410021 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f60e4058ea0 con 0x7f60f00b36a0 2024-03-08T06:01:00.526 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:00.526 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":26,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:42.202011+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:00.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.527+0000 7f60f646f640 1 -- 172.21.15.76:0/3061410021 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f60dc06b500 msgr2=0x7f60dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.527+0000 7f60f646f640 1 --2- 172.21.15.76:0/3061410021 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f60dc06b500 0x7f60dc06d9c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f60e0036000 tx=0x7f60e0030000 comp rx=0 tx=0).stop 2024-03-08T06:01:00.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.527+0000 7f60f646f640 1 -- 172.21.15.76:0/3061410021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60f00b36a0 msgr2=0x7f60f0160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.527+0000 7f60f646f640 1 --2- 172.21.15.76:0/3061410021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60f00b36a0 0x7f60f0160680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f60e400f5f0 tx=0x7f60e400fab0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.527+0000 7f60f646f640 1 -- 172.21.15.76:0/3061410021 shutdown_connections 2024-03-08T06:01:00.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.527+0000 7f60f646f640 1 --2- 172.21.15.76:0/3061410021 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f60dc06b500 0x7f60dc06d9c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.527+0000 7f60f646f640 1 --2- 172.21.15.76:0/3061410021 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f60f014d8b0 0x7f60f014dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.527+0000 7f60f646f640 1 --2- 172.21.15.76:0/3061410021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60f00b36a0 0x7f60f0160680 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.527+0000 7f60f646f640 1 --2- 172.21.15.76:0/3061410021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60d8003a20 0x7f60f00b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.527+0000 7f60f646f640 1 -- 172.21.15.76:0/3061410021 >> 172.21.15.76:0/3061410021 conn(0x7f60f0099a60 msgr2=0x7f60f015cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.528+0000 7f60f646f640 1 -- 172.21.15.76:0/3061410021 shutdown_connections 2024-03-08T06:01:00.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.528+0000 7f60f646f640 1 -- 172.21.15.76:0/3061410021 wait complete. 2024-03-08T06:01:00.540 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-08T06:01:00.540 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-08T06:01:00.540 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-08T06:01:00.541 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-08T06:01:00.541 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-08T06:01:00.542 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-08T06:01:00.542 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-08T06:01:00.543 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-08T06:01:00.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.712+0000 7f9fe4e21640 1 Processor -- start 2024-03-08T06:01:00.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.712+0000 7fcc50a33640 1 Processor -- start 2024-03-08T06:01:00.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.712+0000 7f1b7c49a640 1 Processor -- start 2024-03-08T06:01:00.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.712+0000 7f1b7c49a640 1 -- start start 2024-03-08T06:01:00.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.712+0000 7f9fe4e21640 1 -- start start 2024-03-08T06:01:00.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f2fcf59e640 1 Processor -- start 2024-03-08T06:01:00.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.713+0000 7fcc50a33640 1 -- start start 2024-03-08T06:01:00.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f2fcf59e640 1 -- start start 2024-03-08T06:01:00.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f0cc74c2640 1 Processor -- start 2024-03-08T06:01:00.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f0cc74c2640 1 -- start start 2024-03-08T06:01:00.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.713+0000 7f9fe4e21640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9fe00cd570 0x7f9fe006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.713+0000 7f9fe4e21640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fe00cef40 0x7f9fe006ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.713+0000 7f9fe4e21640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe006f980 con 0x7f9fe00cd950 2024-03-08T06:01:00.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.713+0000 7f9fe4e21640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe006fc60 con 0x7f9fe00cd570 2024-03-08T06:01:00.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.713+0000 7f9fe4e21640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe00cb400 con 0x7f9fe00cef40 2024-03-08T06:01:00.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f9fdeffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9fe00cd950 0x7f9fe00ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57494/0 (socket says 172.21.15.76:57494) 2024-03-08T06:01:00.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f9fdeffd640 1 -- 172.21.15.76:0/2182564672 learned_addr learned my addr 172.21.15.76:0/2182564672 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:00.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f9fdf7fe640 1 --2- 172.21.15.76:0/2182564672 >> v2:172.21.15.81:3300/0 conn(0x7f9fe00cd570 0x7f9fe006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f9fdf7fe640 1 -- 172.21.15.76:0/2182564672 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fe00cef40 msgr2=0x7f9fe006ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f9fdf7fe640 1 --2- 172.21.15.76:0/2182564672 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fe00cef40 0x7f9fe006ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f9fdf7fe640 1 -- 172.21.15.76:0/2182564672 >> v1:172.21.15.76:6789/0 conn(0x7f9fe00cd950 legacy=0x7f9fe00ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f9fdf7fe640 1 -- 172.21.15.76:0/2182564672 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe00cb6e0 con 0x7f9fe00cd570 2024-03-08T06:01:00.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f1b7c49a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1b6c0e6ca0 0x7f1b6c0e7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f1b7c49a640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b6c0534f0 0x7f1b6c053960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f1b7c49a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6c0e7a90 con 0x7f1b6c0e3160 2024-03-08T06:01:00.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f1b7c49a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b6c03aab0 con 0x7f1b6c0e6ca0 2024-03-08T06:01:00.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f1b7c49a640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c0e7650 con 0x7f1b6c0534f0 2024-03-08T06:01:00.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f1b7a236640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f1b6c0e3160 0x7f1b6c03a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:40050/0 (socket says 172.21.15.76:40050) 2024-03-08T06:01:00.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f1b7a236640 1 -- 172.21.15.76:0/3623206740 learned_addr learned my addr 172.21.15.76:0/3623206740 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:00.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f1b79234640 1 -- 172.21.15.76:0/3623206740 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555653239 0 0) 0x7f1b6c0e7a90 con 0x7f1b6c0e3160 2024-03-08T06:01:00.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.714+0000 7f1b79234640 1 -- 172.21.15.76:0/3623206740 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b60003520 con 0x7f1b6c0e3160 2024-03-08T06:01:00.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b7aa37640 1 --2- 172.21.15.76:0/3623206740 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b6c0534f0 0x7f1b6c053960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b79a35640 1 --2- 172.21.15.76:0/3623206740 >> v2:172.21.15.76:3300/0 conn(0x7f1b6c0e6ca0 0x7f1b6c0e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b79234640 1 -- 172.21.15.76:0/3623206740 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1428515160 0 0) 0x7f1b60003520 con 0x7f1b6c0e3160 2024-03-08T06:01:00.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b79234640 1 -- 172.21.15.76:0/3623206740 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b6c0534f0 msgr2=0x7f1b6c053960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b79234640 1 --2- 172.21.15.76:0/3623206740 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b6c0534f0 0x7f1b6c053960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b79234640 1 -- 172.21.15.76:0/3623206740 >> v2:172.21.15.76:3300/0 conn(0x7f1b6c0e6ca0 msgr2=0x7f1b6c0e7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b79234640 1 --2- 172.21.15.76:0/3623206740 >> v2:172.21.15.76:3300/0 conn(0x7f1b6c0e6ca0 0x7f1b6c0e7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b79234640 1 -- 172.21.15.76:0/3623206740 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c0071d0 con 0x7f1b6c0e3160 2024-03-08T06:01:00.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f2fcf59e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2fc8048ad0 0x7f2fc8052680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f2fcf59e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc803a2f0 con 0x7f2fc803b0b0 2024-03-08T06:01:00.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f2fcf59e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc803a5d0 con 0x7f2fc80eac70 2024-03-08T06:01:00.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f2fcf59e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc8052bc0 con 0x7f2fc8048ad0 2024-03-08T06:01:00.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b79234640 1 -- 172.21.15.76:0/3623206740 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f1b70001690 con 0x7f1b6c0e3160 2024-03-08T06:01:00.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b7aa37640 1 --2- 172.21.15.76:0/3623206740 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b6c0534f0 0x7f1b6c053960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:00.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b79234640 1 -- 172.21.15.76:0/3623206740 >> v1:172.21.15.81:6789/0 conn(0x7f1b6c0e3160 legacy=0x7f1b6c03a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b79234640 1 --2- 172.21.15.76:0/3623206740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b600039b0 0x7f1b60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b79234640 1 --2- 172.21.15.76:0/3623206740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b60004800 0x7f1b60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b79234640 1 --2- 172.21.15.76:0/3623206740 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b6c0534f0 0x7f1b60005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7fcc50a33640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcc4404f850 0x7fcc44007130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7fcc50a33640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc44007670 0x7fcc44053ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7fcc49d9b640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc44007670 0x7fcc44053ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7fcc49d9b640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc44007670 0x7fcc44053ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37140/0 (socket says 172.21.15.76:37140) 2024-03-08T06:01:00.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7fcc49d9b640 1 -- 172.21.15.76:0/1499450624 learned_addr learned my addr 172.21.15.76:0/1499450624 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:00.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7fcc50a33640 1 -- 172.21.15.76:0/1499450624 --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc440df5a0 con 0x7fcc44054530 2024-03-08T06:01:00.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7fcc50a33640 1 -- 172.21.15.76:0/1499450624 --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc44006090 con 0x7fcc4404f850 2024-03-08T06:01:00.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7fcc50a33640 1 -- 172.21.15.76:0/1499450624 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc44006370 con 0x7fcc44007670 2024-03-08T06:01:00.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7fcc4a59c640 1 --2- 172.21.15.76:0/1499450624 >> v2:172.21.15.81:3300/0 conn(0x7fcc4404f850 0x7fcc44007130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f0cc74c2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0cc015e120 0x7f0cc00cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f2fce59c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2fc80eac70 0x7f2fc80483c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:40052/0 (socket says 172.21.15.76:40052) 2024-03-08T06:01:00.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f2fce59c640 1 -- 172.21.15.76:0/1743039553 learned_addr learned my addr 172.21.15.76:0/1743039553 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:00.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f2fcd59a640 1 -- 172.21.15.76:0/1743039553 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355887192 0 0) 0x7f2fc803a5d0 con 0x7f2fc80eac70 2024-03-08T06:01:00.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f2fcd59a640 1 -- 172.21.15.76:0/1743039553 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fb8003520 con 0x7f2fc80eac70 2024-03-08T06:01:00.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f2fcdd9b640 1 --2- 172.21.15.76:0/1743039553 >> v2:172.21.15.76:3300/0 conn(0x7f2fc8048ad0 0x7f2fc8052680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f37f8c72640 1 Processor -- start 2024-03-08T06:01:00.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f37f8c72640 1 -- start start 2024-03-08T06:01:00.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b7a236640 1 --2- 172.21.15.76:0/3623206740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b600039b0 0x7f1b60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b7aa37640 1 --2- 172.21.15.76:0/3623206740 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b6c0534f0 0x7f1b60005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b79234640 1 -- 172.21.15.76:0/3623206740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c03aab0 con 0x7f1b60004800 2024-03-08T06:01:00.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b79234640 1 -- 172.21.15.76:0/3623206740 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c0e7650 con 0x7f1b600039b0 2024-03-08T06:01:00.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b79234640 1 -- 172.21.15.76:0/3623206740 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6000bbd0 con 0x7f1b6c0534f0 2024-03-08T06:01:00.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.715+0000 7f1b79a35640 1 --2- 172.21.15.76:0/3623206740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b60004800 0x7f1b60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7fcc49d9b640 1 -- 172.21.15.76:0/1499450624 >> v2:172.21.15.81:3300/0 conn(0x7fcc4404f850 msgr2=0x7fcc44007130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7fcc49d9b640 1 --2- 172.21.15.76:0/1499450624 >> v2:172.21.15.81:3300/0 conn(0x7fcc4404f850 0x7fcc44007130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7fcc49d9b640 1 -- 172.21.15.76:0/1499450624 >> v1:172.21.15.76:6789/0 conn(0x7fcc44054530 legacy=0x7fcc4404fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7fcc49d9b640 1 -- 172.21.15.76:0/1499450624 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc44006650 con 0x7fcc44007670 2024-03-08T06:01:00.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7fcc49d9b640 1 --2- 172.21.15.76:0/1499450624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc44007670 0x7fcc44053ff0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0747c0 tx=0x7fcc4c0a8ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=635cf826cbdd1ccb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f0cc74c2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc00b1550 con 0x7f0cc015e500 2024-03-08T06:01:00.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f0cc74c2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc00ae1d0 con 0x7f0cc00cf010 2024-03-08T06:01:00.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f0cc74c2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc00ae4b0 con 0x7f0cc015e120 2024-03-08T06:01:00.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f0cc525e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0cc015e120 0x7f0cc00cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f0cc525e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0cc015e120 0x7f0cc00cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51312/0 (socket says 172.21.15.76:51312) 2024-03-08T06:01:00.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f0cc525e640 1 -- 172.21.15.76:0/2141834813 learned_addr learned my addr 172.21.15.76:0/2141834813 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:00.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f0cc525e640 1 -- 172.21.15.76:0/2141834813 >> v1:172.21.15.81:6789/0 conn(0x7f0cc00cf010 legacy=0x7f0cc00aecb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-08T06:01:00.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f2fcd59a640 1 -- 172.21.15.76:0/1743039553 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1218371915 0 0) 0x7f2fb8003520 con 0x7f2fc80eac70 2024-03-08T06:01:00.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f2fcd59a640 1 -- 172.21.15.76:0/1743039553 >> v2:172.21.15.76:3300/0 conn(0x7f2fc8048ad0 msgr2=0x7f2fc8052680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f2fcd59a640 1 --2- 172.21.15.76:0/1743039553 >> v2:172.21.15.76:3300/0 conn(0x7f2fc8048ad0 0x7f2fc8052680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f2fcd59a640 1 -- 172.21.15.76:0/1743039553 >> v1:172.21.15.76:6789/0 conn(0x7f2fc803b0b0 legacy=0x7f2fc80537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f2fcd59a640 1 -- 172.21.15.76:0/1743039553 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc803a8b0 con 0x7f2fc80eac70 2024-03-08T06:01:00.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f2fcd59a640 1 -- 172.21.15.76:0/1743039553 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f2fc0001670 con 0x7f2fc80eac70 2024-03-08T06:01:00.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f2fcd59a640 1 -- 172.21.15.76:0/1743039553 >> v1:172.21.15.81:6789/0 conn(0x7f2fc80eac70 legacy=0x7f2fc80483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f2fcd59a640 1 --2- 172.21.15.76:0/1743039553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fb80039b0 0x7f2fb8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f2fcd59a640 1 --2- 172.21.15.76:0/1743039553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fb8004800 0x7f2fb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f2fce59c640 1 --2- 172.21.15.76:0/1743039553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fb80039b0 0x7f2fb8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f2fcd59a640 1 --2- 172.21.15.76:0/1743039553 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fb80052c0 0x7f2fb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f2fcd59a640 1 -- 172.21.15.76:0/1743039553 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb800c1d0 con 0x7f2fb8004800 2024-03-08T06:01:00.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f2fcd59a640 1 -- 172.21.15.76:0/1743039553 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb800c380 con 0x7f2fb80039b0 2024-03-08T06:01:00.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f2fced9d640 1 --2- 172.21.15.76:0/1743039553 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fb80052c0 0x7f2fb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f2fcdd9b640 1 --2- 172.21.15.76:0/1743039553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fb8004800 0x7f2fb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f2fcd59a640 1 -- 172.21.15.76:0/1743039553 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb800c580 con 0x7f2fb80052c0 2024-03-08T06:01:00.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f37f8c72640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f37ec00b1d0 0x7f37ec03a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f37f8c72640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f37ec03ae00 0x7f37ec006b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f37f8c72640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ec00cb30 con 0x7f37ec0e0020 2024-03-08T06:01:00.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f37f8c72640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ec0565d0 con 0x7f37ec03ae00 2024-03-08T06:01:00.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f37f8c72640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ec0568b0 con 0x7f37ec00b1d0 2024-03-08T06:01:00.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f37f3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f37ec03ae00 0x7f37ec006b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f37f3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f37ec03ae00 0x7f37ec006b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51316/0 (socket says 172.21.15.76:51316) 2024-03-08T06:01:00.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f37f3fff640 1 -- 172.21.15.76:0/2401197240 learned_addr learned my addr 172.21.15.76:0/2401197240 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:00.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f0a3f6d4640 1 Processor -- start 2024-03-08T06:01:00.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f0a3f6d4640 1 -- start start 2024-03-08T06:01:00.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f0a3f6d4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3806e7c0 0x7f0a380b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f1b7aa37640 1 -- 172.21.15.76:0/3623206740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b600039b0 msgr2=0x7f1b60003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f1b7aa37640 1 --2- 172.21.15.76:0/3623206740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b600039b0 0x7f1b60003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f1b7aa37640 1 -- 172.21.15.76:0/3623206740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b60004800 msgr2=0x7f1b60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f1b7aa37640 1 --2- 172.21.15.76:0/3623206740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b60004800 0x7f1b60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.716+0000 7f1b7aa37640 1 -- 172.21.15.76:0/3623206740 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b60003520 con 0x7f1b6c0534f0 2024-03-08T06:01:00.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f1b7aa37640 1 --2- 172.21.15.76:0/3623206740 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b6c0534f0 0x7f1b60005200 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1b74075290 tx=0x7f1b60005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8414c57bcf119b39 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7ffb0e9f6640 1 Processor -- start 2024-03-08T06:01:00.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f9fdf7fe640 1 --2- 172.21.15.76:0/2182564672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fe00cd570 0x7f9fe006ef30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009d90 tx=0x7f9fd802be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=597cfe1d542bef6b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7fcc4959a640 1 -- 172.21.15.76:0/1499450624 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c0aa930 con 0x7fcc44007670 2024-03-08T06:01:00.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7fcc4959a640 1 -- 172.21.15.76:0/1499450624 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4c0aaad0 con 0x7fcc44007670 2024-03-08T06:01:00.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7fcc4959a640 1 -- 172.21.15.76:0/1499450624 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c0ab440 con 0x7fcc44007670 2024-03-08T06:01:00.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f0a3f6d4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a380b2fb0 con 0x7f0a38067c50 2024-03-08T06:01:00.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f0a3f6d4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a380ced30 con 0x7f0a380b5990 2024-03-08T06:01:00.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f0a3f6d4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380b5de0 con 0x7f0a3806e7c0 2024-03-08T06:01:00.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7f0a3cc6f640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3806e7c0 0x7f0a380b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7f0a3cc6f640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3806e7c0 0x7f0a380b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37158/0 (socket says 172.21.15.76:37158) 2024-03-08T06:01:00.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7f0a3cc6f640 1 -- 172.21.15.76:0/1056204133 learned_addr learned my addr 172.21.15.76:0/1056204133 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:00.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f1b79234640 1 -- 172.21.15.76:0/3623206740 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b740b5070 con 0x7f1b6c0534f0 2024-03-08T06:01:00.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f1b79234640 1 -- 172.21.15.76:0/3623206740 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b740afd30 con 0x7f1b6c0534f0 2024-03-08T06:01:00.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f1b79234640 1 -- 172.21.15.76:0/3623206740 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b740626b0 con 0x7f1b6c0534f0 2024-03-08T06:01:00.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f37f3fff640 1 -- 172.21.15.76:0/2401197240 >> v2:172.21.15.81:3300/0 conn(0x7f37ec00b1d0 msgr2=0x7f37ec03a8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-08T06:01:00.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f0cc525e640 1 -- 172.21.15.76:0/2141834813 >> v1:172.21.15.76:6789/0 conn(0x7f0cc015e500 legacy=0x7f0cc00ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f0cc525e640 1 -- 172.21.15.76:0/2141834813 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc00ae790 con 0x7f0cc015e120 2024-03-08T06:01:00.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f37f3fff640 1 --2- 172.21.15.76:0/2401197240 >> v2:172.21.15.81:3300/0 conn(0x7f37ec00b1d0 0x7f37ec03a8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7f0a3cc6f640 1 -- 172.21.15.76:0/1056204133 >> v1:172.21.15.81:6789/0 conn(0x7f0a380b5990 legacy=0x7f0a380ce050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-08T06:01:00.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7f0a3cc6f640 1 -- 172.21.15.76:0/1056204133 >> v1:172.21.15.76:6789/0 conn(0x7f0a38067c50 legacy=0x7f0a38143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7f0a3cc6f640 1 -- 172.21.15.76:0/1056204133 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a380cf010 con 0x7f0a3806e7c0 2024-03-08T06:01:00.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f37f3fff640 1 -- 172.21.15.76:0/2401197240 >> v1:172.21.15.81:6789/0 conn(0x7f37ec0e0020 legacy=0x7f37ec008f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-08T06:01:00.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7f1b7c49a640 1 -- 172.21.15.76:0/3623206740 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b6c0534f0 msgr2=0x7f1b60005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7f1b7c49a640 1 --2- 172.21.15.76:0/3623206740 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b6c0534f0 0x7f1b60005200 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1b74075290 tx=0x7f1b60005830 comp rx=0 tx=0).stop 2024-03-08T06:01:00.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7f1b7aa37640 1 -- 172.21.15.76:0/3623206740 reap_dead start 2024-03-08T06:01:00.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7f37f3fff640 1 -- 172.21.15.76:0/2401197240 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec00cdb0 con 0x7f37ec03ae00 2024-03-08T06:01:00.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7f37f3fff640 1 --2- 172.21.15.76:0/2401197240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37ec03ae00 0x7f37ec006b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f37f402d3c0 tx=0x7f37f40a68b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7102dd88a2329f1f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7fcc50a33640 1 -- 172.21.15.76:0/1499450624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc44007670 msgr2=0x7fcc44053ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7fcc50a33640 1 --2- 172.21.15.76:0/1499450624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc44007670 0x7fcc44053ff0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0747c0 tx=0x7fcc4c0a8ad0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f0cc525e640 1 --2- 172.21.15.76:0/2141834813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc015e120 0x7f0cc00cb270 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009870 tx=0x7f0cbc02dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10698d52c1d4a57b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f1b7c49a640 1 -- 172.21.15.76:0/3623206740 shutdown_connections 2024-03-08T06:01:00.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f1b7c49a640 1 -- 172.21.15.76:0/3623206740 >> 172.21.15.76:0/3623206740 conn(0x7f1b6c009fb0 msgr2=0x7f1b6c00a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.717+0000 7ffb0e9f6640 1 -- start start 2024-03-08T06:01:00.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7ffb0e9f6640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb080ce750 0x7ffb08163830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7ffb0e9f6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ffb08163d70 0x7ffb081626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7ffb0e9f6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffb08162c20 0x7ffb080d20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7ffb0e9f6640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0815cb60 con 0x7ffb08162c20 2024-03-08T06:01:00.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7ffb0e9f6640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb080ceb30 con 0x7ffb08163d70 2024-03-08T06:01:00.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7ffb0e9f6640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb08163110 con 0x7ffb080ce750 2024-03-08T06:01:00.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7fcc50a33640 1 -- 172.21.15.76:0/1499450624 shutdown_connections 2024-03-08T06:01:00.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7fcc50a33640 1 --2- 172.21.15.76:0/1499450624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc44007670 0x7fcc44053ff0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7fcc50a33640 1 --2- 172.21.15.76:0/1499450624 >> v2:172.21.15.81:3300/0 conn(0x7fcc4404f850 0x7fcc44007130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7fcc50a33640 1 -- 172.21.15.76:0/1499450624 >> 172.21.15.76:0/1499450624 conn(0x7fcc44009fe0 msgr2=0x7fcc4400a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f0cb7fff640 1 -- 172.21.15.76:0/2141834813 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc03b070 con 0x7f0cc015e120 2024-03-08T06:01:00.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7f2fce59c640 1 -- 172.21.15.76:0/1743039553 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fb80052c0 msgr2=0x7f2fb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7f2fce59c640 1 --2- 172.21.15.76:0/1743039553 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fb80052c0 0x7f2fb800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7f2fce59c640 1 -- 172.21.15.76:0/1743039553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fb8004800 msgr2=0x7f2fb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7f2fce59c640 1 --2- 172.21.15.76:0/1743039553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fb8004800 0x7f2fb8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.718+0000 7f2fce59c640 1 -- 172.21.15.76:0/1743039553 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb8003520 con 0x7f2fb80039b0 2024-03-08T06:01:00.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f2fced9d640 1 -- 172.21.15.76:0/1743039553 reap_dead start 2024-03-08T06:01:00.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f37f27fc640 1 -- 172.21.15.76:0/2401197240 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f40b0020 con 0x7f37ec03ae00 2024-03-08T06:01:00.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7fcc50a33640 1 -- 172.21.15.76:0/1499450624 shutdown_connections 2024-03-08T06:01:00.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f37f27fc640 1 -- 172.21.15.76:0/2401197240 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f40ac8d0 con 0x7f37ec03ae00 2024-03-08T06:01:00.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f0cb7fff640 1 -- 172.21.15.76:0/2141834813 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc02e960 con 0x7f0cc015e120 2024-03-08T06:01:00.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f0cb7fff640 1 -- 172.21.15.76:0/2141834813 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc02ed30 con 0x7f0cc015e120 2024-03-08T06:01:00.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f9fde7fc640 1 -- 172.21.15.76:0/2182564672 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8039070 con 0x7f9fe00cd570 2024-03-08T06:01:00.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f9fde7fc640 1 -- 172.21.15.76:0/2182564672 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd802cbf0 con 0x7f9fe00cd570 2024-03-08T06:01:00.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f1b7c49a640 1 -- 172.21.15.76:0/3623206740 shutdown_connections 2024-03-08T06:01:00.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f1b7c49a640 1 -- 172.21.15.76:0/3623206740 wait complete. 2024-03-08T06:01:00.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f9fde7fc640 1 -- 172.21.15.76:0/2182564672 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8034440 con 0x7f9fe00cd570 2024-03-08T06:01:00.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f0a3cc6f640 1 --2- 172.21.15.76:0/1056204133 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3806e7c0 0x7f0a380b5450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0a280098a0 tx=0x7f0a2802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6df6fc3b2ca636e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7ffb07fff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb080ce750 0x7ffb08163830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7ffb07fff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb080ce750 0x7ffb08163830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37164/0 (socket says 172.21.15.76:37164) 2024-03-08T06:01:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7ffb07fff640 1 -- 172.21.15.76:0/2984129626 learned_addr learned my addr 172.21.15.76:0/2984129626 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7ffb077fe640 1 --2- 172.21.15.76:0/2984129626 >> v2:172.21.15.81:3300/0 conn(0x7ffb08163d70 0x7ffb081626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7ffb0cf93640 1 --2- 172.21.15.76:0/2984129626 >> v2:172.21.15.76:3300/0 conn(0x7ffb08162c20 0x7ffb080d20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7ffb07fff640 1 -- 172.21.15.76:0/2984129626 >> v2:172.21.15.81:3300/0 conn(0x7ffb08163d70 msgr2=0x7ffb081626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7ffb07fff640 1 --2- 172.21.15.76:0/2984129626 >> v2:172.21.15.81:3300/0 conn(0x7ffb08163d70 0x7ffb081626e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7ffb07fff640 1 -- 172.21.15.76:0/2984129626 >> v2:172.21.15.76:3300/0 conn(0x7ffb08162c20 msgr2=0x7ffb080d20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7ffb07fff640 1 --2- 172.21.15.76:0/2984129626 >> v2:172.21.15.76:3300/0 conn(0x7ffb08162c20 0x7ffb080d20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7ffb07fff640 1 -- 172.21.15.76:0/2984129626 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb081633f0 con 0x7ffb080ce750 2024-03-08T06:01:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f2fce59c640 1 --2- 172.21.15.76:0/1743039553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fb80039b0 0x7f2fb8003d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0004fd0 tx=0x7f2fb8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bc706b65ff8b8fe server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7ffb0cf93640 1 --2- 172.21.15.76:0/2984129626 >> v2:172.21.15.76:3300/0 conn(0x7ffb08162c20 0x7ffb080d20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:00.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f0a2ffff640 1 -- 172.21.15.76:0/1056204133 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28034070 con 0x7f0a3806e7c0 2024-03-08T06:01:00.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f0a2ffff640 1 -- 172.21.15.76:0/1056204133 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a28030840 con 0x7f0a3806e7c0 2024-03-08T06:01:00.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f0a2ffff640 1 -- 172.21.15.76:0/1056204133 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28030c10 con 0x7f0a3806e7c0 2024-03-08T06:01:00.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f2fcd59a640 1 -- 172.21.15.76:0/1743039553 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0004ba0 con 0x7f2fb80039b0 2024-03-08T06:01:00.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f2fcd59a640 1 -- 172.21.15.76:0/1743039553 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc0005ad0 con 0x7f2fb80039b0 2024-03-08T06:01:00.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f2fcd59a640 1 -- 172.21.15.76:0/1743039553 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc00080d0 con 0x7f2fb80039b0 2024-03-08T06:01:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f9fe4e21640 1 -- 172.21.15.76:0/2182564672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fe00cd570 msgr2=0x7f9fe006ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f9fe4e21640 1 --2- 172.21.15.76:0/2182564672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fe00cd570 0x7f9fe006ef30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009d90 tx=0x7f9fd802be40 comp rx=0 tx=0).stop 2024-03-08T06:01:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f9fe4e21640 1 -- 172.21.15.76:0/2182564672 shutdown_connections 2024-03-08T06:01:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f9fe4e21640 1 --2- 172.21.15.76:0/2182564672 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fe00cef40 0x7f9fe006ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f9fe4e21640 1 --2- 172.21.15.76:0/2182564672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fe00cd570 0x7f9fe006ef30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f9fe4e21640 1 -- 172.21.15.76:0/2182564672 >> 172.21.15.76:0/2182564672 conn(0x7f9fe0099a60 msgr2=0x7f9fe00b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f0cc74c2640 1 -- 172.21.15.76:0/2141834813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc015e120 msgr2=0x7f0cc00cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f0cc74c2640 1 --2- 172.21.15.76:0/2141834813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc015e120 0x7f0cc00cb270 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009870 tx=0x7f0cbc02dbb0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f0cc74c2640 1 -- 172.21.15.76:0/2141834813 shutdown_connections 2024-03-08T06:01:00.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7fcc50a33640 1 -- 172.21.15.76:0/1499450624 wait complete. 2024-03-08T06:01:00.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f0cc74c2640 1 --2- 172.21.15.76:0/2141834813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc015e120 0x7f0cc00cb270 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f0cc74c2640 1 -- 172.21.15.76:0/2141834813 >> 172.21.15.76:0/2141834813 conn(0x7f0cc00d0890 msgr2=0x7f0cc00d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.719+0000 7f37f27fc640 1 -- 172.21.15.76:0/2401197240 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f40acd40 con 0x7f37ec03ae00 2024-03-08T06:01:00.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f37f8c72640 1 -- 172.21.15.76:0/2401197240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37ec03ae00 msgr2=0x7f37ec006b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f37f8c72640 1 --2- 172.21.15.76:0/2401197240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37ec03ae00 0x7f37ec006b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f37f402d3c0 tx=0x7f37f40a68b0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f37f8c72640 1 -- 172.21.15.76:0/2401197240 shutdown_connections 2024-03-08T06:01:00.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f37f8c72640 1 --2- 172.21.15.76:0/2401197240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37ec03ae00 0x7f37ec006b00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f37f8c72640 1 --2- 172.21.15.76:0/2401197240 >> v2:172.21.15.81:3300/0 conn(0x7f37ec00b1d0 0x7f37ec03a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f37f8c72640 1 -- 172.21.15.76:0/2401197240 >> 172.21.15.76:0/2401197240 conn(0x7f37ec01ef70 msgr2=0x7f37ec055c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f0cc74c2640 1 -- 172.21.15.76:0/2141834813 shutdown_connections 2024-03-08T06:01:00.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7f37f8c72640 1 -- 172.21.15.76:0/2401197240 shutdown_connections 2024-03-08T06:01:00.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7f37f8c72640 1 -- 172.21.15.76:0/2401197240 wait complete. 2024-03-08T06:01:00.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7f2fcf59e640 1 -- 172.21.15.76:0/1743039553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fb80039b0 msgr2=0x7f2fb8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7f2fcf59e640 1 --2- 172.21.15.76:0/1743039553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fb80039b0 0x7f2fb8003d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0004fd0 tx=0x7f2fb8005830 comp rx=0 tx=0).stop 2024-03-08T06:01:00.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7f1b7c49a640 1 Processor -- start 2024-03-08T06:01:00.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7f0a3f6d4640 1 -- 172.21.15.76:0/1056204133 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3806e7c0 msgr2=0x7f0a380b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7f0a3f6d4640 1 --2- 172.21.15.76:0/1056204133 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3806e7c0 0x7f0a380b5450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0a280098a0 tx=0x7f0a2802b980 comp rx=0 tx=0).stop 2024-03-08T06:01:00.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7f0cc74c2640 1 -- 172.21.15.76:0/2141834813 wait complete. 2024-03-08T06:01:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7f1b7c49a640 1 -- start start 2024-03-08T06:01:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7f2fcf59e640 1 -- 172.21.15.76:0/1743039553 shutdown_connections 2024-03-08T06:01:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7f2fcf59e640 1 --2- 172.21.15.76:0/1743039553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fb80039b0 0x7f2fb8003d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7f2fcf59e640 1 -- 172.21.15.76:0/1743039553 >> 172.21.15.76:0/1743039553 conn(0x7f2fc801e6c0 msgr2=0x7f2fc80553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.720+0000 7f9fe4e21640 1 -- 172.21.15.76:0/2182564672 shutdown_connections 2024-03-08T06:01:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7f9fe4e21640 1 -- 172.21.15.76:0/2182564672 wait complete. 2024-03-08T06:01:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7ffb07fff640 1 --2- 172.21.15.76:0/2984129626 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb080ce750 0x7ffb08163830 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffb08163320 tx=0x7ffafc02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=221aaa7880f1969d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7ffb06ffd640 1 -- 172.21.15.76:0/2984129626 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc035020 con 0x7ffb080ce750 2024-03-08T06:01:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7ffb06ffd640 1 -- 172.21.15.76:0/2984129626 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc031640 con 0x7ffb080ce750 2024-03-08T06:01:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.721+0000 7ffb06ffd640 1 -- 172.21.15.76:0/2984129626 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc02d040 con 0x7ffb080ce750 2024-03-08T06:01:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f2fcf59e640 1 -- 172.21.15.76:0/1743039553 shutdown_connections 2024-03-08T06:01:00.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f1b7c49a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b6c0534f0 0x7f1b6c035380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f9fe4e21640 1 Processor -- start 2024-03-08T06:01:00.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f2fcf59e640 1 -- 172.21.15.76:0/1743039553 wait complete. 2024-03-08T06:01:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f1b7c49a640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b6c0e22d0 0x7f1b6c0d2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7fcc50a33640 1 Processor -- start 2024-03-08T06:01:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7ffb0e9f6640 1 -- 172.21.15.76:0/2984129626 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb080ce750 msgr2=0x7ffb08163830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f1b7c49a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b6c0e2750 0x7f1b6c0d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f1b7c49a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c007790 con 0x7f1b6c0534f0 2024-03-08T06:01:00.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f1b7c49a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c053060 con 0x7f1b6c0e2750 2024-03-08T06:01:00.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f1b7c49a640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c052cf0 con 0x7f1b6c0e22d0 2024-03-08T06:01:00.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7ffb0e9f6640 1 --2- 172.21.15.76:0/2984129626 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb080ce750 0x7ffb08163830 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffb08163320 tx=0x7ffafc02b8f0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7fcc50a33640 1 -- start start 2024-03-08T06:01:00.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f0cc74c2640 1 Processor -- start 2024-03-08T06:01:00.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f1b7a236640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b6c0534f0 0x7f1b6c035380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7ffb0e9f6640 1 -- 172.21.15.76:0/2984129626 shutdown_connections 2024-03-08T06:01:00.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7ffb0e9f6640 1 --2- 172.21.15.76:0/2984129626 >> v2:172.21.15.76:3300/0 conn(0x7ffb08162c20 0x7ffb080d20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7ffb0e9f6640 1 --2- 172.21.15.76:0/2984129626 >> v2:172.21.15.81:3300/0 conn(0x7ffb08163d70 0x7ffb081626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7ffb0e9f6640 1 --2- 172.21.15.76:0/2984129626 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb080ce750 0x7ffb08163830 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f0a3f6d4640 1 -- 172.21.15.76:0/1056204133 shutdown_connections 2024-03-08T06:01:00.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f0a3f6d4640 1 --2- 172.21.15.76:0/1056204133 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3806e7c0 0x7f0a380b5450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f0a3f6d4640 1 -- 172.21.15.76:0/1056204133 >> 172.21.15.76:0/1056204133 conn(0x7f0a380717b0 msgr2=0x7f0a38071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7ffb0e9f6640 1 -- 172.21.15.76:0/2984129626 >> 172.21.15.76:0/2984129626 conn(0x7ffb080670d0 msgr2=0x7ffb080d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f1b7a236640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b6c0534f0 0x7f1b6c035380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51338/0 (socket says 172.21.15.76:51338) 2024-03-08T06:01:00.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f1b7a236640 1 -- 172.21.15.76:0/1338492308 learned_addr learned my addr 172.21.15.76:0/1338492308 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:00.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.723+0000 7ffb0e9f6640 1 -- 172.21.15.76:0/2984129626 shutdown_connections 2024-03-08T06:01:00.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.723+0000 7f1b79a35640 1 --2- 172.21.15.76:0/1338492308 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b6c0e22d0 0x7f1b6c0d2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.723+0000 7f1b7a236640 1 -- 172.21.15.76:0/1338492308 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b6c0e22d0 msgr2=0x7f1b6c0d2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.723+0000 7f1b7a236640 1 --2- 172.21.15.76:0/1338492308 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b6c0e22d0 0x7f1b6c0d2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.723+0000 7f1b7a236640 1 -- 172.21.15.76:0/1338492308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b6c0e2750 msgr2=0x7f1b6c0d2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.723+0000 7f1b7a236640 1 --2- 172.21.15.76:0/1338492308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b6c0e2750 0x7f1b6c0d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.723+0000 7f1b7a236640 1 -- 172.21.15.76:0/1338492308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c0e6340 con 0x7f1b6c0534f0 2024-03-08T06:01:00.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.723+0000 7f9fe4e21640 1 -- start start 2024-03-08T06:01:00.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.723+0000 7f1b7a236640 1 --2- 172.21.15.76:0/1338492308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b6c0534f0 0x7f1b6c035380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1b70004fa0 tx=0x7f1b700074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.723+0000 7f0a3f6d4640 1 -- 172.21.15.76:0/1056204133 shutdown_connections 2024-03-08T06:01:00.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.723+0000 7f0a3f6d4640 1 -- 172.21.15.76:0/1056204133 wait complete. 2024-03-08T06:01:00.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f2fcf59e640 1 Processor -- start 2024-03-08T06:01:00.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.722+0000 7f0cc74c2640 1 -- start start 2024-03-08T06:01:00.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f0cc74c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc00cf010 0x7f0cc00ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f0cc74c2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cc015e500 0x7f0cc00c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f0cc74c2640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0cc0155bc0 0x7f0cc00c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f0cc74c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc00d1380 con 0x7f0cc00cf010 2024-03-08T06:01:00.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f0cc74c2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc00cf6b0 con 0x7f0cc015e500 2024-03-08T06:01:00.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f0cc74c2640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc00d1500 con 0x7f0cc0155bc0 2024-03-08T06:01:00.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f0cc525e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc00cf010 0x7f0cc00ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f0cc525e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc00cf010 0x7f0cc00ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51346/0 (socket says 172.21.15.76:51346) 2024-03-08T06:01:00.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f2fcf59e640 1 -- start start 2024-03-08T06:01:00.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f37f8c72640 1 Processor -- start 2024-03-08T06:01:00.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f0cc525e640 1 -- 172.21.15.76:0/1452056473 learned_addr learned my addr 172.21.15.76:0/1452056473 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:00.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f0cc4a5d640 1 --2- 172.21.15.76:0/1452056473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cc015e500 0x7f0cc00c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f0cc5a5f640 1 --2- 172.21.15.76:0/1452056473 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0cc0155bc0 0x7f0cc00c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f9fe4e21640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fe00cd570 0x7f9fe00b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f1b6affd640 1 -- 172.21.15.76:0/1338492308 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7000f990 con 0x7f1b6c0534f0 2024-03-08T06:01:00.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f1b6affd640 1 -- 172.21.15.76:0/1338492308 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b70018070 con 0x7f1b6c0534f0 2024-03-08T06:01:00.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f1b6affd640 1 -- 172.21.15.76:0/1338492308 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7000fd90 con 0x7f1b6c0534f0 2024-03-08T06:01:00.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.723+0000 7ffb0e9f6640 1 -- 172.21.15.76:0/2984129626 wait complete. 2024-03-08T06:01:00.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7ffb0e9f6640 1 Processor -- start 2024-03-08T06:01:00.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f9fe4e21640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fe00cd950 0x7f9fe0158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f0cc525e640 1 -- 172.21.15.76:0/1452056473 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0cc0155bc0 msgr2=0x7f0cc00c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f0cc525e640 1 --2- 172.21.15.76:0/1452056473 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0cc0155bc0 0x7f0cc00c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f0cc525e640 1 -- 172.21.15.76:0/1452056473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cc015e500 msgr2=0x7f0cc00c49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f0cc525e640 1 --2- 172.21.15.76:0/1452056473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cc015e500 0x7f0cc00c49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f0cc525e640 1 -- 172.21.15.76:0/1452056473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc00caa80 con 0x7f0cc00cf010 2024-03-08T06:01:00.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f0cc5a5f640 1 --2- 172.21.15.76:0/1452056473 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0cc0155bc0 0x7f0cc00c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:00.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f37f8c72640 1 -- start start 2024-03-08T06:01:00.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.724+0000 7f9fe4e21640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fe00cef40 0x7f9fe01efb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f9fdf7fe640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fe00cd570 0x7f9fe00b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f9fdf7fe640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fe00cd570 0x7f9fe00b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37180/0 (socket says 172.21.15.76:37180) 2024-03-08T06:01:00.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f9fdf7fe640 1 -- 172.21.15.76:0/4002860522 learned_addr learned my addr 172.21.15.76:0/4002860522 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:00.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f9fe4e21640 1 -- 172.21.15.76:0/4002860522 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe00702e0 con 0x7f9fe00cef40 2024-03-08T06:01:00.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7ffb0e9f6640 1 -- start start 2024-03-08T06:01:00.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f9fe4e21640 1 -- 172.21.15.76:0/4002860522 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe0070160 con 0x7f9fe00cd950 2024-03-08T06:01:00.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f1b7c49a640 1 -- 172.21.15.76:0/1338492308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6c0345c0 con 0x7f1b6c0534f0 2024-03-08T06:01:00.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f9fe4e21640 1 -- 172.21.15.76:0/4002860522 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe006f470 con 0x7f9fe00cd570 2024-03-08T06:01:00.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f1b7c49a640 1 -- 172.21.15.76:0/1338492308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6c034b80 con 0x7f1b6c0534f0 2024-03-08T06:01:00.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f9fdeffd640 1 --2- 172.21.15.76:0/4002860522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fe00cd950 0x7f9fe0158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f0cc525e640 1 --2- 172.21.15.76:0/1452056473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc00cf010 0x7f0cc00ca410 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009870 tx=0x7f0cbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f9fdffff640 1 --2- 172.21.15.76:0/4002860522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fe00cef40 0x7f9fe01efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f9fdf7fe640 1 -- 172.21.15.76:0/4002860522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fe00cd950 msgr2=0x7f9fe0158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f9fdf7fe640 1 --2- 172.21.15.76:0/4002860522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fe00cd950 0x7f9fe0158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f9fdf7fe640 1 -- 172.21.15.76:0/4002860522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fe00cef40 msgr2=0x7f9fe01efb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f9fdf7fe640 1 --2- 172.21.15.76:0/4002860522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fe00cef40 0x7f9fe01efb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f9fdf7fe640 1 -- 172.21.15.76:0/4002860522 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe01f00c0 con 0x7f9fe00cd570 2024-03-08T06:01:00.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f9fdffff640 1 --2- 172.21.15.76:0/4002860522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fe00cef40 0x7f9fe01efb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:00.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f2fcf59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fb80039b0 0x7f2fc8049000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f9fdeffd640 1 --2- 172.21.15.76:0/4002860522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fe00cd950 0x7f9fe0158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:00.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f0a3f6d4640 1 Processor -- start 2024-03-08T06:01:00.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7fcc50a33640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc44007670 0x7fcc440ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f9fdf7fe640 1 --2- 172.21.15.76:0/4002860522 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fe00cd570 0x7f9fe00b5cb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009d90 tx=0x7f9fd8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f0a3f6d4640 1 -- start start 2024-03-08T06:01:00.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f0cb5ffb640 1 -- 172.21.15.76:0/1452056473 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc03b070 con 0x7f0cc00cf010 2024-03-08T06:01:00.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f37f8c72640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37ec00b1d0 0x7f37ec036a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f37f8c72640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f37ec03ae00 0x7f37ec0d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f37f8c72640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37ec0e0020 0x7f37ec034dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f37f8c72640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec0e0400 con 0x7f37ec0e0020 2024-03-08T06:01:00.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f37f8c72640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec0534a0 con 0x7f37ec00b1d0 2024-03-08T06:01:00.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.725+0000 7f37f8c72640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec03b610 con 0x7f37ec03ae00 2024-03-08T06:01:00.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f9fc7fff640 1 -- 172.21.15.76:0/4002860522 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8044020 con 0x7f9fe00cd570 2024-03-08T06:01:00.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f2fcf59e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fc804b6a0 0x7f2fc8052040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f0cc74c2640 1 -- 172.21.15.76:0/1452056473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc00c5450 con 0x7f0cc00cf010 2024-03-08T06:01:00.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f9fe4e21640 1 -- 172.21.15.76:0/4002860522 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe00b5950 con 0x7f9fe00cd570 2024-03-08T06:01:00.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7ffb0e9f6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb080ce750 0x7ffb0814f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f9fc7fff640 1 -- 172.21.15.76:0/4002860522 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd803a040 con 0x7f9fe00cd570 2024-03-08T06:01:00.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f2fcf59e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fc8052580 0x7f2fc817c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7ffb0e9f6640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb08162c20 0x7ffb080d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f0cc74c2640 1 -- 172.21.15.76:0/1452056473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc00c5a10 con 0x7f0cc00cf010 2024-03-08T06:01:00.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f2fcf59e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc803a0a0 con 0x7f2fb80039b0 2024-03-08T06:01:00.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f2fcf59e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc80eb260 con 0x7f2fc804b6a0 2024-03-08T06:01:00.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f2fcf59e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc803ac20 con 0x7f2fc8052580 2024-03-08T06:01:00.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7ffb07fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb080ce750 0x7ffb0814f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7ffb077fe640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb08162c20 0x7ffb080d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7ffb077fe640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb08162c20 0x7ffb080d3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37220/0 (socket says 172.21.15.76:37220) 2024-03-08T06:01:00.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7ffb0e9f6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb08163d70 0x7ffb080d4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f37f3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37ec0e0020 0x7f37ec034dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f37f3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37ec0e0020 0x7f37ec034dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51368/0 (socket says 172.21.15.76:51368) 2024-03-08T06:01:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f37f3fff640 1 -- 172.21.15.76:0/378141522 learned_addr learned my addr 172.21.15.76:0/378141522 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f2fcdd9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fc804b6a0 0x7f2fc8052040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f2fcdd9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fc804b6a0 0x7f2fc8052040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.76:57230/0 (socket says 172.21.15.76:57230) 2024-03-08T06:01:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f2fcdd9b640 1 -- 172.21.15.76:0/1046250768 learned_addr learned my addr 172.21.15.76:0/1046250768 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:00.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f2fced9d640 1 --2- 172.21.15.76:0/1046250768 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fc8052580 0x7f2fc817c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f0a3f6d4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a38067c50 0x7f0a38152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f0a3f6d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3806e7c0 0x7f0a3814dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f0a3f6d4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a380b5990 0x7f0a38139720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f0a3f6d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380ce760 con 0x7f0a3806e7c0 2024-03-08T06:01:00.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f0a3f6d4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380b3940 con 0x7f0a38067c50 2024-03-08T06:01:00.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f0a3f6d4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380ce8e0 con 0x7f0a380b5990 2024-03-08T06:01:00.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f2fcdd9b640 1 -- 172.21.15.76:0/1046250768 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fc8052580 msgr2=0x7f2fc817c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f2fcdd9b640 1 --2- 172.21.15.76:0/1046250768 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fc8052580 0x7f2fc817c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f2fcdd9b640 1 -- 172.21.15.76:0/1046250768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fb80039b0 msgr2=0x7f2fc8049000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f2fce59c640 1 --2- 172.21.15.76:0/1046250768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fb80039b0 0x7f2fc8049000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.726+0000 7f9fe4e21640 1 -- 172.21.15.76:0/4002860522 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe00c82b0 con 0x7f9fe00cd570 2024-03-08T06:01:00.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f37f3fff640 1 -- 172.21.15.76:0/378141522 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f37ec03ae00 msgr2=0x7f37ec0d15b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-08T06:01:00.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f37f3fff640 1 --2- 172.21.15.76:0/378141522 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f37ec03ae00 0x7f37ec0d15b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f37f3fff640 1 -- 172.21.15.76:0/378141522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37ec00b1d0 msgr2=0x7f37ec036a30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-08T06:01:00.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f37f3fff640 1 --2- 172.21.15.76:0/378141522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37ec00b1d0 0x7f37ec036a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f37f3fff640 1 -- 172.21.15.76:0/378141522 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec0d2000 con 0x7f37ec0e0020 2024-03-08T06:01:00.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f0a3cc6f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3806e7c0 0x7f0a3814dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f9fc7fff640 1 -- 172.21.15.76:0/4002860522 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8039070 con 0x7f9fe00cd570 2024-03-08T06:01:00.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f0a3cc6f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3806e7c0 0x7f0a3814dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51388/0 (socket says 172.21.15.76:51388) 2024-03-08T06:01:00.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f0a3cc6f640 1 -- 172.21.15.76:0/3619709960 learned_addr learned my addr 172.21.15.76:0/3619709960 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:00.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f0cb5ffb640 1 -- 172.21.15.76:0/1452056473 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc036d80 con 0x7f0cc00cf010 2024-03-08T06:01:00.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f2fcdd9b640 1 --2- 172.21.15.76:0/1046250768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fb80039b0 0x7f2fc8049000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f2fcdd9b640 1 -- 172.21.15.76:0/1046250768 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc804bba0 con 0x7f2fc804b6a0 2024-03-08T06:01:00.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f2fce59c640 1 --2- 172.21.15.76:0/1046250768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fb80039b0 0x7f2fc8049000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:00.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f37f3fff640 1 --2- 172.21.15.76:0/378141522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37ec0e0020 0x7f37ec034dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f37f404fd70 tx=0x7f37f4073d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7f9fc5ffb640 1 -- 172.21.15.76:0/4002860522 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9fb8001030 con 0x7f9fe00cd570 2024-03-08T06:01:00.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f0a3cc6f640 1 -- 172.21.15.76:0/3619709960 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a380b5990 msgr2=0x7f0a38139720 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-08T06:01:00.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f0a3cc6f640 1 --2- 172.21.15.76:0/3619709960 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a380b5990 0x7f0a38139720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f0a3cc6f640 1 -- 172.21.15.76:0/3619709960 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a38067c50 msgr2=0x7f0a38152170 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-08T06:01:00.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f0a3cc6f640 1 --2- 172.21.15.76:0/3619709960 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a38067c50 0x7f0a38152170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.727+0000 7f0a3cc6f640 1 -- 172.21.15.76:0/3619709960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3814e5a0 con 0x7f0a3806e7c0 2024-03-08T06:01:00.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7f2fcdd9b640 1 --2- 172.21.15.76:0/1046250768 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fc804b6a0 0x7f2fc8052040 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400f520 tx=0x7f2fc400f9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7f37dffff640 1 -- 172.21.15.76:0/378141522 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f40c4070 con 0x7f37ec0e0020 2024-03-08T06:01:00.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7f37dffff640 1 -- 172.21.15.76:0/378141522 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f40b5d10 con 0x7f37ec0e0020 2024-03-08T06:01:00.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7f37dffff640 1 -- 172.21.15.76:0/378141522 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f40bf620 con 0x7f37ec0e0020 2024-03-08T06:01:00.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7f37f8c72640 1 -- 172.21.15.76:0/378141522 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec0d22e0 con 0x7f37ec0e0020 2024-03-08T06:01:00.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7f0cb5ffb640 1 -- 172.21.15.76:0/1452056473 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc041640 con 0x7f0cc00cf010 2024-03-08T06:01:00.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7f1b6affd640 1 -- 172.21.15.76:0/1338492308 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b7001d460 con 0x7f1b6c0534f0 2024-03-08T06:01:00.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7fcc50a33640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc4404f850 0x7fcc440dae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7fcc50a33640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc44054530 0x7fcc440dd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7fcc50a33640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc44050340 con 0x7fcc44054530 2024-03-08T06:01:00.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7fcc50a33640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc44053390 con 0x7fcc44007670 2024-03-08T06:01:00.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7fcc50a33640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4400ab00 con 0x7fcc4404f850 2024-03-08T06:01:00.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7fcc4a59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc44007670 0x7fcc440ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7f0cab7fe640 1 -- 172.21.15.76:0/1452056473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ca0001030 con 0x7f0cc00cf010 2024-03-08T06:01:00.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7f37f8c72640 1 -- 172.21.15.76:0/378141522 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ec036240 con 0x7f37ec0e0020 2024-03-08T06:01:00.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7fcc49d9b640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc4404f850 0x7fcc440dae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7fcc49d9b640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc4404f850 0x7fcc440dae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37230/0 (socket says 172.21.15.76:37230) 2024-03-08T06:01:00.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7fcc49d9b640 1 -- 172.21.15.76:0/2806783471 learned_addr learned my addr 172.21.15.76:0/2806783471 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:00.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.729+0000 7fcc4ad9d640 1 --2- 172.21.15.76:0/2806783471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc44054530 0x7fcc440dd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7f0a3cc6f640 1 --2- 172.21.15.76:0/3619709960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3806e7c0 0x7f0a3814dee0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0a280097f0 tx=0x7f0a28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7f0a2dffb640 1 -- 172.21.15.76:0/3619709960 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28045070 con 0x7f0a3806e7c0 2024-03-08T06:01:00.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7f0a3f6d4640 1 -- 172.21.15.76:0/3619709960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3814e880 con 0x7f0a3806e7c0 2024-03-08T06:01:00.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7f0a3f6d4640 1 -- 172.21.15.76:0/3619709960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a381502e0 con 0x7f0a3806e7c0 2024-03-08T06:01:00.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.729+0000 7f0a2dffb640 1 -- 172.21.15.76:0/3619709960 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2803f4c0 con 0x7f0a3806e7c0 2024-03-08T06:01:00.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.728+0000 7f1b6affd640 1 --2- 172.21.15.76:0/1338492308 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b5806b5d0 0x7f1b5806da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.729+0000 7f1b6affd640 1 -- 172.21.15.76:0/1338492308 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1b7008a550 con 0x7f1b6c0534f0 2024-03-08T06:01:00.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.729+0000 7f9fc7fff640 1 -- 172.21.15.76:0/4002860522 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9fd8047020 con 0x7f9fe00cd570 2024-03-08T06:01:00.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.729+0000 7f0a2dffb640 1 -- 172.21.15.76:0/3619709960 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28034040 con 0x7f0a3806e7c0 2024-03-08T06:01:00.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.729+0000 7f1b79a35640 1 --2- 172.21.15.76:0/1338492308 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b5806b5d0 0x7f1b5806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.729+0000 7f9fc7fff640 1 --2- 172.21.15.76:0/4002860522 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9fcc06b500 0x7f9fcc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.729+0000 7f37f8c72640 1 -- 172.21.15.76:0/378141522 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f37ec005710 con 0x7f37ec0e0020 2024-03-08T06:01:00.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.729+0000 7f9fc7fff640 1 -- 172.21.15.76:0/4002860522 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9fd80ad060 con 0x7f9fe00cd570 2024-03-08T06:01:00.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7f9fc7fff640 1 --2- 172.21.15.76:0/4002860522 >> [v2:172.21.15.81:6816/298940182,v1:172.21.15.81:6817/298940182] conn(0x7f9fcc074c80 0x7f9fcc0770c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7f9fdeffd640 1 --2- 172.21.15.76:0/4002860522 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9fcc06b500 0x7f9fcc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7f9fc7fff640 1 -- 172.21.15.76:0/4002860522 --> [v2:172.21.15.81:6816/298940182,v1:172.21.15.81:6817/298940182] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9fd80047f0 con 0x7f9fcc074c80 2024-03-08T06:01:00.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7f9fc7fff640 1 -- 172.21.15.76:0/4002860522 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fd80b4050 con 0x7f9fe00cd570 2024-03-08T06:01:00.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7fcc4a59c640 1 -- 172.21.15.76:0/2806783471 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc4404f850 msgr2=0x7fcc440dae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7fcc4a59c640 1 --2- 172.21.15.76:0/2806783471 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc4404f850 0x7fcc440dae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7fcc4a59c640 1 -- 172.21.15.76:0/2806783471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc44054530 msgr2=0x7fcc440dd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7fcc4a59c640 1 --2- 172.21.15.76:0/2806783471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc44054530 0x7fcc440dd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7fcc4a59c640 1 -- 172.21.15.76:0/2806783471 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4404ab60 con 0x7fcc44007670 2024-03-08T06:01:00.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7f0cb5ffb640 1 -- 172.21.15.76:0/1452056473 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0cbc041820 con 0x7f0cc00cf010 2024-03-08T06:01:00.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7f0a3f6d4640 1 -- 172.21.15.76:0/3619709960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a18001030 con 0x7f0a3806e7c0 2024-03-08T06:01:00.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.729+0000 7f2fbeffd640 1 -- 172.21.15.76:0/1046250768 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4019020 con 0x7f2fc804b6a0 2024-03-08T06:01:00.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.729+0000 7f2fcf59e640 1 -- 172.21.15.76:0/1046250768 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc804be80 con 0x7f2fc804b6a0 2024-03-08T06:01:00.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.729+0000 7f2fcf59e640 1 -- 172.21.15.76:0/1046250768 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc80ce2f0 con 0x7f2fc804b6a0 2024-03-08T06:01:00.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.729+0000 7f2fbeffd640 1 -- 172.21.15.76:0/1046250768 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc4014870 con 0x7f2fc804b6a0 2024-03-08T06:01:00.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7f9fdffff640 1 --2- 172.21.15.76:0/4002860522 >> [v2:172.21.15.81:6816/298940182,v1:172.21.15.81:6817/298940182] conn(0x7f9fcc074c80 0x7f9fcc0770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7f2fbeffd640 1 -- 172.21.15.76:0/1046250768 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4014c20 con 0x7f2fc804b6a0 2024-03-08T06:01:00.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.729+0000 7ffb07fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb080ce750 0x7ffb0814f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.76:57256/0 (socket says 172.21.15.76:57256) 2024-03-08T06:01:00.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7ffb0e9f6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080d1380 con 0x7ffb08163d70 2024-03-08T06:01:00.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7ffb0e9f6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080d1bb0 con 0x7ffb080ce750 2024-03-08T06:01:00.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7ffb0e9f6640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080d1500 con 0x7ffb08162c20 2024-03-08T06:01:00.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7ffb0cf93640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb08163d70 0x7ffb080d4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7ffb077fe640 1 -- 172.21.15.76:0/4181149575 learned_addr learned my addr 172.21.15.76:0/4181149575 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:00.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7f1b79a35640 1 --2- 172.21.15.76:0/1338492308 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b5806b5d0 0x7f1b5806da90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c006830 tx=0x7f1b64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7f9fdeffd640 1 --2- 172.21.15.76:0/4002860522 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9fcc06b500 0x7f9fcc06d9c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00b2930 tx=0x7f9fd0001520 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.730+0000 7f37dffff640 1 -- 172.21.15.76:0/378141522 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37f40b3020 con 0x7f37ec0e0020 2024-03-08T06:01:00.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7f37dffff640 1 --2- 172.21.15.76:0/378141522 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f37d806b520 0x7f37d806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7fcc4a59c640 1 --2- 172.21.15.76:0/2806783471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc44007670 0x7fcc440ccea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00d620 tx=0x7fcc3c00dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7f2fcf59e640 1 -- 172.21.15.76:0/1046250768 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2fb4001030 con 0x7f2fc804b6a0 2024-03-08T06:01:00.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7f1b7c49a640 1 --2- 172.21.15.76:0/1338492308 >> [v2:172.21.15.81:6808/173715776,v1:172.21.15.81:6809/173715776] conn(0x7f1b6c034d50 0x7f1b6c0e56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7f37dffff640 1 -- 172.21.15.76:0/378141522 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f37f4128660 con 0x7f37ec0e0020 2024-03-08T06:01:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7f1b7c49a640 1 -- 172.21.15.76:0/1338492308 --> [v2:172.21.15.81:6808/173715776,v1:172.21.15.81:6809/173715776] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b6c0052c0 con 0x7f1b6c034d50 2024-03-08T06:01:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7f1b7aa37640 1 --2- 172.21.15.76:0/1338492308 >> [v2:172.21.15.81:6808/173715776,v1:172.21.15.81:6809/173715776] conn(0x7f1b6c034d50 0x7f1b6c0e56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7f37dffff640 1 --2- 172.21.15.76:0/378141522 >> [v2:172.21.15.76:6808/2501234044,v1:172.21.15.76:6809/2501234044] conn(0x7f37d8074ca0 0x7f37d80770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7f37dffff640 1 -- 172.21.15.76:0/378141522 --> [v2:172.21.15.76:6808/2501234044,v1:172.21.15.76:6809/2501234044] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37f40b0e80 con 0x7f37d8074ca0 2024-03-08T06:01:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7fcc3affd640 1 -- 172.21.15.76:0/2806783471 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c00b620 con 0x7fcc44007670 2024-03-08T06:01:00.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7f37dffff640 1 -- 172.21.15.76:0/378141522 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37f4128a50 con 0x7f37ec0e0020 2024-03-08T06:01:00.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7fcc3affd640 1 -- 172.21.15.76:0/2806783471 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3c019070 con 0x7fcc44007670 2024-03-08T06:01:00.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f2fbeffd640 1 -- 172.21.15.76:0/1046250768 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2fc401c020 con 0x7f2fc804b6a0 2024-03-08T06:01:00.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7f37f37fe640 1 --2- 172.21.15.76:0/378141522 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f37d806b520 0x7f37d806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f37f2ffd640 1 --2- 172.21.15.76:0/378141522 >> [v2:172.21.15.76:6808/2501234044,v1:172.21.15.76:6809/2501234044] conn(0x7f37d8074ca0 0x7f37d80770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7ffb0cf93640 1 -- 172.21.15.76:0/4181149575 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb08162c20 msgr2=0x7ffb080d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7ffb0cf93640 1 --2- 172.21.15.76:0/4181149575 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb08162c20 0x7ffb080d3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7ffb0cf93640 1 -- 172.21.15.76:0/4181149575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb080ce750 msgr2=0x7ffb0814f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7ffb0cf93640 1 --2- 172.21.15.76:0/4181149575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb080ce750 0x7ffb0814f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7ffb0cf93640 1 -- 172.21.15.76:0/4181149575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb080cd330 con 0x7ffb08163d70 2024-03-08T06:01:00.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f1b7aa37640 1 --2- 172.21.15.76:0/1338492308 >> [v2:172.21.15.81:6808/173715776,v1:172.21.15.81:6809/173715776] conn(0x7f1b6c034d50 0x7f1b6c0e56f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7f0a2dffb640 1 -- 172.21.15.76:0/3619709960 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a28050060 con 0x7f0a3806e7c0 2024-03-08T06:01:00.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f0a2dffb640 1 --2- 172.21.15.76:0/3619709960 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0a1c06b5a0 0x7f0a1c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f0a2dffb640 1 -- 172.21.15.76:0/3619709960 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0a280b0710 con 0x7f0a3806e7c0 2024-03-08T06:01:00.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f0a3d470640 1 --2- 172.21.15.76:0/3619709960 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0a1c06b5a0 0x7f0a1c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f0a2dffb640 1 --2- 172.21.15.76:0/3619709960 >> [v2:172.21.15.81:6800/3228626670,v1:172.21.15.81:6801/3228626670] conn(0x7f0a1c074d20 0x7f0a1c077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f0a2dffb640 1 -- 172.21.15.76:0/3619709960 --> [v2:172.21.15.81:6800/3228626670,v1:172.21.15.81:6801/3228626670] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a1c077830 con 0x7f0a1c074d20 2024-03-08T06:01:00.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f0a2dffb640 1 -- 172.21.15.76:0/3619709960 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a280b7050 con 0x7f0a3806e7c0 2024-03-08T06:01:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f37f37fe640 1 --2- 172.21.15.76:0/378141522 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f37d806b520 0x7f37d806d9e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f37ec055e10 tx=0x7f37e8006bd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f2fbeffd640 1 --2- 172.21.15.76:0/1046250768 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2fb006b550 0x7f2fb006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7fcc50a33640 1 -- 172.21.15.76:0/2806783471 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4404ae40 con 0x7fcc44007670 2024-03-08T06:01:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f0a3dc71640 1 --2- 172.21.15.76:0/3619709960 >> [v2:172.21.15.81:6800/3228626670,v1:172.21.15.81:6801/3228626670] conn(0x7f0a1c074d20 0x7f0a1c077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7f0cb5ffb640 1 --2- 172.21.15.76:0/1452056473 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0ca406b4d0 0x7f0ca406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f0cc4a5d640 1 --2- 172.21.15.76:0/1452056473 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0ca406b4d0 0x7f0ca406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f0cb5ffb640 1 -- 172.21.15.76:0/1452056473 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0cbc0ac760 con 0x7f0cc00cf010 2024-03-08T06:01:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f0cb5ffb640 1 --2- 172.21.15.76:0/1452056473 >> [v2:172.21.15.76:6800/712186592,v1:172.21.15.76:6801/712186592] conn(0x7f0ca4074c50 0x7f0ca4077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f0cb5ffb640 1 -- 172.21.15.76:0/1452056473 --> [v2:172.21.15.76:6800/712186592,v1:172.21.15.76:6801/712186592] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cbc001d30 con 0x7f0ca4074c50 2024-03-08T06:01:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f0cb5ffb640 1 -- 172.21.15.76:0/1452056473 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cbc0b3050 con 0x7f0cc00cf010 2024-03-08T06:01:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f0cc5a5f640 1 --2- 172.21.15.76:0/1452056473 >> [v2:172.21.15.76:6800/712186592,v1:172.21.15.76:6801/712186592] conn(0x7f0ca4074c50 0x7f0ca4077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f2fbeffd640 1 -- 172.21.15.76:0/1046250768 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2fc408aac0 con 0x7f2fc804b6a0 2024-03-08T06:01:00.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f37f2ffd640 1 --2- 172.21.15.76:0/378141522 >> [v2:172.21.15.76:6808/2501234044,v1:172.21.15.76:6809/2501234044] conn(0x7f37d8074ca0 0x7f37d80770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.731+0000 7f9fdffff640 1 --2- 172.21.15.76:0/4002860522 >> [v2:172.21.15.81:6816/298940182,v1:172.21.15.81:6817/298940182] conn(0x7f9fcc074c80 0x7f9fcc0770c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.732+0000 7f9fc7fff640 1 -- 172.21.15.76:0/4002860522 <== osd.7 v2:172.21.15.81:6816/298940182 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9fd80047f0 con 0x7f9fcc074c80 2024-03-08T06:01:00.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7f2fce59c640 1 --2- 172.21.15.76:0/1046250768 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2fb006b550 0x7f2fb006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7f2fbeffd640 1 --2- 172.21.15.76:0/1046250768 >> [v2:172.21.15.76:6824/1112475349,v1:172.21.15.76:6825/1112475349] conn(0x7f2fb0074cd0 0x7f2fb0077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7ffb0cf93640 1 --2- 172.21.15.76:0/4181149575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb08163d70 0x7ffb080d4060 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffb08163040 tx=0x7ffb0000aad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7f0cc4a5d640 1 --2- 172.21.15.76:0/1452056473 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0ca406b4d0 0x7f0ca406d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00704d0 tx=0x7f0cb0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7f0cc5a5f640 1 --2- 172.21.15.76:0/1452056473 >> [v2:172.21.15.76:6800/712186592,v1:172.21.15.76:6801/712186592] conn(0x7f0ca4074c50 0x7f0ca4077090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7f2fbeffd640 1 -- 172.21.15.76:0/1046250768 --> [v2:172.21.15.76:6824/1112475349,v1:172.21.15.76:6825/1112475349] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2fc400ae90 con 0x7f2fb0074cd0 2024-03-08T06:01:00.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7f2fbeffd640 1 -- 172.21.15.76:0/1046250768 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fc408ce60 con 0x7f2fc804b6a0 2024-03-08T06:01:00.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7ffb04ff9640 1 -- 172.21.15.76:0/4181149575 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0001a070 con 0x7ffb08163d70 2024-03-08T06:01:00.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7f1b6affd640 1 -- 172.21.15.76:0/1338492308 <== osd.5 v2:172.21.15.81:6808/173715776 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1b6c0052c0 con 0x7f1b6c034d50 2024-03-08T06:01:00.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7f2fced9d640 1 --2- 172.21.15.76:0/1046250768 >> [v2:172.21.15.76:6824/1112475349,v1:172.21.15.76:6825/1112475349] conn(0x7f2fb0074cd0 0x7f2fb0077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7ffb04ff9640 1 -- 172.21.15.76:0/4181149575 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0000d6f0 con 0x7ffb08163d70 2024-03-08T06:01:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7ffb04ff9640 1 -- 172.21.15.76:0/4181149575 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0000dae0 con 0x7ffb08163d70 2024-03-08T06:01:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7f0a3d470640 1 --2- 172.21.15.76:0/3619709960 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0a1c06b5a0 0x7f0a1c06da60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0a30004350 tx=0x7f0a30007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7f0a3dc71640 1 --2- 172.21.15.76:0/3619709960 >> [v2:172.21.15.81:6800/3228626670,v1:172.21.15.81:6801/3228626670] conn(0x7f0a1c074d20 0x7f0a1c077160 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7ffb0e9f6640 1 -- 172.21.15.76:0/4181149575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0814f6d0 con 0x7ffb08163d70 2024-03-08T06:01:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7ffb0e9f6640 1 -- 172.21.15.76:0/4181149575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb080cd050 con 0x7ffb08163d70 2024-03-08T06:01:00.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.734+0000 7f0cb5ffb640 1 -- 172.21.15.76:0/1452056473 <== osd.1 v2:172.21.15.76:6800/712186592 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0cbc001d30 con 0x7f0ca4074c50 2024-03-08T06:01:00.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.733+0000 7f2fced9d640 1 --2- 172.21.15.76:0/1046250768 >> [v2:172.21.15.76:6824/1112475349,v1:172.21.15.76:6825/1112475349] conn(0x7f2fb0074cd0 0x7f2fb0077110 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.734+0000 7f2fce59c640 1 --2- 172.21.15.76:0/1046250768 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2fb006b550 0x7f2fb006da10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0034000 tx=0x7f2fc002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.734+0000 7f0a2dffb640 1 -- 172.21.15.76:0/3619709960 <== osd.4 v2:172.21.15.81:6800/3228626670 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0a1c077830 con 0x7f0a1c074d20 2024-03-08T06:01:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.734+0000 7fcc50a33640 1 -- 172.21.15.76:0/2806783471 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4404b400 con 0x7fcc44007670 2024-03-08T06:01:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.735+0000 7fcc3affd640 1 -- 172.21.15.76:0/2806783471 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c00b9e0 con 0x7fcc44007670 2024-03-08T06:01:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.735+0000 7fcc3affd640 1 -- 172.21.15.76:0/2806783471 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc3c00bbc0 con 0x7fcc44007670 2024-03-08T06:01:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.736+0000 7fcc3affd640 1 --2- 172.21.15.76:0/2806783471 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcc2c06f930 0x7fcc2c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.736+0000 7fcc3affd640 1 -- 172.21.15.76:0/2806783471 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcc3c08a000 con 0x7fcc44007670 2024-03-08T06:01:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.737+0000 7fcc50a33640 1 -- 172.21.15.76:0/2806783471 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc30001030 con 0x7fcc44007670 2024-03-08T06:01:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.735+0000 7f2fbeffd640 1 -- 172.21.15.76:0/1046250768 <== osd.2 v2:172.21.15.76:6824/1112475349 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2fc400ae90 con 0x7f2fb0074cd0 2024-03-08T06:01:00.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.735+0000 7f37dffff640 1 -- 172.21.15.76:0/378141522 <== osd.3 v2:172.21.15.76:6808/2501234044 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f37f40b0e80 con 0x7f37d8074ca0 2024-03-08T06:01:00.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.737+0000 7fcc3affd640 1 --2- 172.21.15.76:0/2806783471 >> [v2:172.21.15.76:6810/280957956,v1:172.21.15.76:6812/280957956] conn(0x7fcc2c079030 0x7fcc2c07b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.737+0000 7fcc3affd640 1 -- 172.21.15.76:0/2806783471 --> [v2:172.21.15.76:6810/280957956,v1:172.21.15.76:6812/280957956] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc2c07bb40 con 0x7fcc2c079030 2024-03-08T06:01:00.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.737+0000 7fcc3affd640 1 -- 172.21.15.76:0/2806783471 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc3c08f060 con 0x7fcc44007670 2024-03-08T06:01:00.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.737+0000 7fcc4ad9d640 1 --2- 172.21.15.76:0/2806783471 >> [v2:172.21.15.76:6810/280957956,v1:172.21.15.76:6812/280957956] conn(0x7fcc2c079030 0x7fcc2c07b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.738+0000 7fcc4ad9d640 1 --2- 172.21.15.76:0/2806783471 >> [v2:172.21.15.76:6810/280957956,v1:172.21.15.76:6812/280957956] conn(0x7fcc2c079030 0x7fcc2c07b470 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.738+0000 7ffaee7fc640 1 -- 172.21.15.76:0/4181149575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffb0804f030 con 0x7ffb08163d70 2024-03-08T06:01:00.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.739+0000 7fcc3affd640 1 -- 172.21.15.76:0/2806783471 <== osd.0 v2:172.21.15.76:6810/280957956 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcc2c07bb40 con 0x7fcc2c079030 2024-03-08T06:01:00.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.739+0000 7fcc49d9b640 1 --2- 172.21.15.76:0/2806783471 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcc2c06f930 0x7fcc2c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.740+0000 7ffb04ff9640 1 -- 172.21.15.76:0/4181149575 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb0001d070 con 0x7ffb08163d70 2024-03-08T06:01:00.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.740+0000 7fcc49d9b640 1 --2- 172.21.15.76:0/2806783471 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcc2c06f930 0x7fcc2c071df0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c064600 tx=0x7fcc4c06e030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.741+0000 7ffb04ff9640 1 --2- 172.21.15.76:0/4181149575 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffaf406b4d0 0x7ffaf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.741+0000 7ffb07fff640 1 --2- 172.21.15.76:0/4181149575 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffaf406b4d0 0x7ffaf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.742+0000 7ffb07fff640 1 --2- 172.21.15.76:0/4181149575 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffaf406b4d0 0x7ffaf406d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffafc036000 tx=0x7ffafc03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.742+0000 7ffb04ff9640 1 -- 172.21.15.76:0/4181149575 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ffb0008ad80 con 0x7ffb08163d70 2024-03-08T06:01:00.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.742+0000 7ffb04ff9640 1 --2- 172.21.15.76:0/4181149575 >> [v2:172.21.15.81:6824/3040851989,v1:172.21.15.81:6825/3040851989] conn(0x7ffaf4074bc0 0x7ffaf4077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:00.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.742+0000 7ffb04ff9640 1 -- 172.21.15.76:0/4181149575 --> [v2:172.21.15.81:6824/3040851989,v1:172.21.15.81:6825/3040851989] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffaf40776d0 con 0x7ffaf4074bc0 2024-03-08T06:01:00.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.742+0000 7ffb04ff9640 1 -- 172.21.15.76:0/4181149575 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb0008b0e0 con 0x7ffb08163d70 2024-03-08T06:01:00.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.743+0000 7ffb077fe640 1 --2- 172.21.15.76:0/4181149575 >> [v2:172.21.15.81:6824/3040851989,v1:172.21.15.81:6825/3040851989] conn(0x7ffaf4074bc0 0x7ffaf4077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:00.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.743+0000 7ffb077fe640 1 --2- 172.21.15.76:0/4181149575 >> [v2:172.21.15.81:6824/3040851989,v1:172.21.15.81:6825/3040851989] conn(0x7ffaf4074bc0 0x7ffaf4077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:00.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.745+0000 7ffb04ff9640 1 -- 172.21.15.76:0/4181149575 <== osd.6 v2:172.21.15.81:6824/3040851989 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ffaf40776d0 con 0x7ffaf4074bc0 2024-03-08T06:01:00.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.763+0000 7f9fc5ffb640 1 -- 172.21.15.76:0/4002860522 --> [v2:172.21.15.81:6816/298940182,v1:172.21.15.81:6817/298940182] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9fb8002dd0 con 0x7f9fcc074c80 2024-03-08T06:01:00.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.764+0000 7f37f8c72640 1 -- 172.21.15.76:0/378141522 --> [v2:172.21.15.76:6808/2501234044,v1:172.21.15.76:6809/2501234044] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f37ec005710 con 0x7f37d8074ca0 2024-03-08T06:01:00.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.764+0000 7f9fc7fff640 1 -- 172.21.15.76:0/4002860522 <== osd.7 v2:172.21.15.81:6816/298940182 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9fb8002dd0 con 0x7f9fcc074c80 2024-03-08T06:01:00.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.765+0000 7f37dffff640 1 -- 172.21.15.76:0/378141522 <== osd.3 v2:172.21.15.76:6808/2501234044 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f37ec005710 con 0x7f37d8074ca0 2024-03-08T06:01:00.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.765+0000 7f9fe4e21640 1 -- 172.21.15.76:0/4002860522 >> [v2:172.21.15.81:6816/298940182,v1:172.21.15.81:6817/298940182] conn(0x7f9fcc074c80 msgr2=0x7f9fcc0770c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.765+0000 7f9fe4e21640 1 --2- 172.21.15.76:0/4002860522 >> [v2:172.21.15.81:6816/298940182,v1:172.21.15.81:6817/298940182] conn(0x7f9fcc074c80 0x7f9fcc0770c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.765+0000 7f37f8c72640 1 -- 172.21.15.76:0/378141522 >> [v2:172.21.15.76:6808/2501234044,v1:172.21.15.76:6809/2501234044] conn(0x7f37d8074ca0 msgr2=0x7f37d80770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.765+0000 7f37f8c72640 1 --2- 172.21.15.76:0/378141522 >> [v2:172.21.15.76:6808/2501234044,v1:172.21.15.76:6809/2501234044] conn(0x7f37d8074ca0 0x7f37d80770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.765+0000 7f9fe4e21640 1 -- 172.21.15.76:0/4002860522 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9fcc06b500 msgr2=0x7f9fcc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.765+0000 7f9fe4e21640 1 --2- 172.21.15.76:0/4002860522 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9fcc06b500 0x7f9fcc06d9c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00b2930 tx=0x7f9fd0001520 comp rx=0 tx=0).stop 2024-03-08T06:01:00.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.765+0000 7f9fe4e21640 1 -- 172.21.15.76:0/4002860522 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fe00cd570 msgr2=0x7f9fe00b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.765+0000 7f9fe4e21640 1 --2- 172.21.15.76:0/4002860522 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fe00cd570 0x7f9fe00b5cb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009d90 tx=0x7f9fd8004a40 comp rx=0 tx=0).stop 2024-03-08T06:01:00.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.765+0000 7f9fdffff640 1 -- 172.21.15.76:0/4002860522 reap_dead start 2024-03-08T06:01:00.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.766+0000 7f37f8c72640 1 -- 172.21.15.76:0/378141522 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f37d806b520 msgr2=0x7f37d806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.766+0000 7f37f8c72640 1 --2- 172.21.15.76:0/378141522 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f37d806b520 0x7f37d806d9e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f37ec055e10 tx=0x7f37e8006bd0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.766+0000 7f37f8c72640 1 -- 172.21.15.76:0/378141522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37ec0e0020 msgr2=0x7f37ec034dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.766+0000 7f37f8c72640 1 --2- 172.21.15.76:0/378141522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37ec0e0020 0x7f37ec034dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f37f404fd70 tx=0x7f37f4073d00 comp rx=0 tx=0).stop 2024-03-08T06:01:00.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.766+0000 7f37f3fff640 1 -- 172.21.15.76:0/378141522 reap_dead start 2024-03-08T06:01:00.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.766+0000 7f37f8c72640 1 -- 172.21.15.76:0/378141522 shutdown_connections 2024-03-08T06:01:00.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.766+0000 7f37f8c72640 1 -- 172.21.15.76:0/378141522 >> 172.21.15.76:0/378141522 conn(0x7f37ec01ef70 msgr2=0x7f37ec055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.766+0000 7f37f8c72640 1 -- 172.21.15.76:0/378141522 shutdown_connections 2024-03-08T06:01:00.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.766+0000 7f9fe4e21640 1 -- 172.21.15.76:0/4002860522 shutdown_connections 2024-03-08T06:01:00.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.766+0000 7f9fe4e21640 1 -- 172.21.15.76:0/4002860522 >> 172.21.15.76:0/4002860522 conn(0x7f9fe0099a60 msgr2=0x7f9fe00af790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.766+0000 7f37f8c72640 1 -- 172.21.15.76:0/378141522 wait complete. 2024-03-08T06:01:00.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.767+0000 7f1b7c49a640 1 -- 172.21.15.76:0/1338492308 --> [v2:172.21.15.81:6808/173715776,v1:172.21.15.81:6809/173715776] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1b6c033a40 con 0x7f1b6c034d50 2024-03-08T06:01:00.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.766+0000 7f2fcf59e640 1 -- 172.21.15.76:0/1046250768 --> [v2:172.21.15.76:6824/1112475349,v1:172.21.15.76:6825/1112475349] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2fb4002dd0 con 0x7f2fb0074cd0 2024-03-08T06:01:00.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.766+0000 7f2fbeffd640 1 -- 172.21.15.76:0/1046250768 <== osd.2 v2:172.21.15.76:6824/1112475349 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2fb4002dd0 con 0x7f2fb0074cd0 2024-03-08T06:01:00.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.767+0000 7f9fe4e21640 1 -- 172.21.15.76:0/4002860522 shutdown_connections 2024-03-08T06:01:00.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.767+0000 7f9fe4e21640 1 -- 172.21.15.76:0/4002860522 wait complete. 2024-03-08T06:01:00.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.767+0000 7f2fbcff9640 1 -- 172.21.15.76:0/1046250768 >> [v2:172.21.15.76:6824/1112475349,v1:172.21.15.76:6825/1112475349] conn(0x7f2fb0074cd0 msgr2=0x7f2fb0077110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.767+0000 7f2fbcff9640 1 --2- 172.21.15.76:0/1046250768 >> [v2:172.21.15.76:6824/1112475349,v1:172.21.15.76:6825/1112475349] conn(0x7f2fb0074cd0 0x7f2fb0077110 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.767+0000 7f1b6affd640 1 -- 172.21.15.76:0/1338492308 <== osd.5 v2:172.21.15.81:6808/173715776 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1b6c033a40 con 0x7f1b6c034d50 2024-03-08T06:01:00.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f1b68ff9640 1 -- 172.21.15.76:0/1338492308 >> [v2:172.21.15.81:6808/173715776,v1:172.21.15.81:6809/173715776] conn(0x7f1b6c034d50 msgr2=0x7f1b6c0e56f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f1b68ff9640 1 --2- 172.21.15.76:0/1338492308 >> [v2:172.21.15.81:6808/173715776,v1:172.21.15.81:6809/173715776] conn(0x7f1b6c034d50 0x7f1b6c0e56f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f1b68ff9640 1 -- 172.21.15.76:0/1338492308 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b5806b5d0 msgr2=0x7f1b5806da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f1b68ff9640 1 --2- 172.21.15.76:0/1338492308 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b5806b5d0 0x7f1b5806da90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c006830 tx=0x7f1b64008040 comp rx=0 tx=0).stop 2024-03-08T06:01:00.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f1b68ff9640 1 -- 172.21.15.76:0/1338492308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b6c0534f0 msgr2=0x7f1b6c035380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f1b68ff9640 1 --2- 172.21.15.76:0/1338492308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b6c0534f0 0x7f1b6c035380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1b70004fa0 tx=0x7f1b700074f0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f1b7aa37640 1 -- 172.21.15.76:0/1338492308 reap_dead start 2024-03-08T06:01:00.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f2fbcff9640 1 -- 172.21.15.76:0/1046250768 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2fb006b550 msgr2=0x7f2fb006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f2fbcff9640 1 --2- 172.21.15.76:0/1046250768 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2fb006b550 0x7f2fb006da10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0034000 tx=0x7f2fc002f000 comp rx=0 tx=0).stop 2024-03-08T06:01:00.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f2fbcff9640 1 -- 172.21.15.76:0/1046250768 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fc804b6a0 msgr2=0x7f2fc8052040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f2fbcff9640 1 --2- 172.21.15.76:0/1046250768 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fc804b6a0 0x7f2fc8052040 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400f520 tx=0x7f2fc400f9e0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f2fced9d640 1 -- 172.21.15.76:0/1046250768 reap_dead start 2024-03-08T06:01:00.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f2fbcff9640 1 -- 172.21.15.76:0/1046250768 shutdown_connections 2024-03-08T06:01:00.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f2fbcff9640 1 -- 172.21.15.76:0/1046250768 >> 172.21.15.76:0/1046250768 conn(0x7f2fc801e6c0 msgr2=0x7f2fc8036b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f2fbcff9640 1 -- 172.21.15.76:0/1046250768 shutdown_connections 2024-03-08T06:01:00.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f2fbcff9640 1 -- 172.21.15.76:0/1046250768 wait complete. 2024-03-08T06:01:00.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f0cab7fe640 1 -- 172.21.15.76:0/1452056473 --> [v2:172.21.15.76:6800/712186592,v1:172.21.15.76:6801/712186592] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ca0002dd0 con 0x7f0ca4074c50 2024-03-08T06:01:00.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f1b68ff9640 1 -- 172.21.15.76:0/1338492308 shutdown_connections 2024-03-08T06:01:00.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.768+0000 7f1b68ff9640 1 -- 172.21.15.76:0/1338492308 >> 172.21.15.76:0/1338492308 conn(0x7f1b6c009fb0 msgr2=0x7f1b6c037c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.769+0000 7f0a3f6d4640 1 -- 172.21.15.76:0/3619709960 --> [v2:172.21.15.81:6800/3228626670,v1:172.21.15.81:6801/3228626670] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0a18002dd0 con 0x7f0a1c074d20 2024-03-08T06:01:00.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.769+0000 7f1b68ff9640 1 -- 172.21.15.76:0/1338492308 shutdown_connections 2024-03-08T06:01:00.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.769+0000 7f1b68ff9640 1 -- 172.21.15.76:0/1338492308 wait complete. 2024-03-08T06:01:00.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.769+0000 7f0cb5ffb640 1 -- 172.21.15.76:0/1452056473 <== osd.1 v2:172.21.15.76:6800/712186592 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0ca0002dd0 con 0x7f0ca4074c50 2024-03-08T06:01:00.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.769+0000 7f0cab7fe640 1 -- 172.21.15.76:0/1452056473 >> [v2:172.21.15.76:6800/712186592,v1:172.21.15.76:6801/712186592] conn(0x7f0ca4074c50 msgr2=0x7f0ca4077090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.769+0000 7f0cab7fe640 1 --2- 172.21.15.76:0/1452056473 >> [v2:172.21.15.76:6800/712186592,v1:172.21.15.76:6801/712186592] conn(0x7f0ca4074c50 0x7f0ca4077090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.769+0000 7f0a2dffb640 1 -- 172.21.15.76:0/3619709960 <== osd.4 v2:172.21.15.81:6800/3228626670 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0a18002dd0 con 0x7f0a1c074d20 2024-03-08T06:01:00.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0cab7fe640 1 -- 172.21.15.76:0/1452056473 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0ca406b4d0 msgr2=0x7f0ca406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0cab7fe640 1 --2- 172.21.15.76:0/1452056473 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0ca406b4d0 0x7f0ca406d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00704d0 tx=0x7f0cb0009040 comp rx=0 tx=0).stop 2024-03-08T06:01:00.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.769+0000 7f0a3f6d4640 1 -- 172.21.15.76:0/3619709960 >> [v2:172.21.15.81:6800/3228626670,v1:172.21.15.81:6801/3228626670] conn(0x7f0a1c074d20 msgr2=0x7f0a1c077160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0a3f6d4640 1 --2- 172.21.15.76:0/3619709960 >> [v2:172.21.15.81:6800/3228626670,v1:172.21.15.81:6801/3228626670] conn(0x7f0a1c074d20 0x7f0a1c077160 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0cab7fe640 1 -- 172.21.15.76:0/1452056473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc00cf010 msgr2=0x7f0cc00ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0cab7fe640 1 --2- 172.21.15.76:0/1452056473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc00cf010 0x7f0cc00ca410 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009870 tx=0x7f0cbc0046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0cc5a5f640 1 -- 172.21.15.76:0/1452056473 reap_dead start 2024-03-08T06:01:00.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0a3f6d4640 1 -- 172.21.15.76:0/3619709960 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0a1c06b5a0 msgr2=0x7f0a1c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0a3f6d4640 1 --2- 172.21.15.76:0/3619709960 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0a1c06b5a0 0x7f0a1c06da60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0a30004350 tx=0x7f0a30007d30 comp rx=0 tx=0).stop 2024-03-08T06:01:00.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0cab7fe640 1 -- 172.21.15.76:0/1452056473 shutdown_connections 2024-03-08T06:01:00.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0cab7fe640 1 -- 172.21.15.76:0/1452056473 >> 172.21.15.76:0/1452056473 conn(0x7f0cc00d0890 msgr2=0x7f0cc00aecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0a3f6d4640 1 -- 172.21.15.76:0/3619709960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3806e7c0 msgr2=0x7f0a3814dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0a3f6d4640 1 --2- 172.21.15.76:0/3619709960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3806e7c0 0x7f0a3814dee0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0a280097f0 tx=0x7f0a28004730 comp rx=0 tx=0).stop 2024-03-08T06:01:00.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0a3dc71640 1 -- 172.21.15.76:0/3619709960 reap_dead start 2024-03-08T06:01:00.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0cab7fe640 1 -- 172.21.15.76:0/1452056473 shutdown_connections 2024-03-08T06:01:00.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0cab7fe640 1 -- 172.21.15.76:0/1452056473 wait complete. 2024-03-08T06:01:00.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0a3f6d4640 1 -- 172.21.15.76:0/3619709960 shutdown_connections 2024-03-08T06:01:00.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0a3f6d4640 1 -- 172.21.15.76:0/3619709960 >> 172.21.15.76:0/3619709960 conn(0x7f0a380717b0 msgr2=0x7f0a3806fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.770+0000 7f0a3f6d4640 1 -- 172.21.15.76:0/3619709960 shutdown_connections 2024-03-08T06:01:00.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.771+0000 7f0a3f6d4640 1 -- 172.21.15.76:0/3619709960 wait complete. 2024-03-08T06:01:00.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.772+0000 7fcc50a33640 1 -- 172.21.15.76:0/2806783471 --> [v2:172.21.15.76:6810/280957956,v1:172.21.15.76:6812/280957956] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcc30002d10 con 0x7fcc2c079030 2024-03-08T06:01:00.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.773+0000 7fcc3affd640 1 -- 172.21.15.76:0/2806783471 <== osd.0 v2:172.21.15.76:6810/280957956 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcc30002d10 con 0x7fcc2c079030 2024-03-08T06:01:00.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.774+0000 7fcc38ff9640 1 -- 172.21.15.76:0/2806783471 >> [v2:172.21.15.76:6810/280957956,v1:172.21.15.76:6812/280957956] conn(0x7fcc2c079030 msgr2=0x7fcc2c07b470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.774+0000 7fcc38ff9640 1 --2- 172.21.15.76:0/2806783471 >> [v2:172.21.15.76:6810/280957956,v1:172.21.15.76:6812/280957956] conn(0x7fcc2c079030 0x7fcc2c07b470 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.774+0000 7fcc38ff9640 1 -- 172.21.15.76:0/2806783471 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcc2c06f930 msgr2=0x7fcc2c071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.774+0000 7fcc38ff9640 1 --2- 172.21.15.76:0/2806783471 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcc2c06f930 0x7fcc2c071df0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c064600 tx=0x7fcc4c06e030 comp rx=0 tx=0).stop 2024-03-08T06:01:00.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.774+0000 7fcc38ff9640 1 -- 172.21.15.76:0/2806783471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc44007670 msgr2=0x7fcc440ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.774+0000 7fcc38ff9640 1 --2- 172.21.15.76:0/2806783471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc44007670 0x7fcc440ccea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00d620 tx=0x7fcc3c00dae0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.774+0000 7ffaee7fc640 1 -- 172.21.15.76:0/4181149575 --> [v2:172.21.15.81:6824/3040851989,v1:172.21.15.81:6825/3040851989] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffb080b8760 con 0x7ffaf4074bc0 2024-03-08T06:01:00.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.775+0000 7fcc4ad9d640 1 -- 172.21.15.76:0/2806783471 reap_dead start 2024-03-08T06:01:00.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.775+0000 7fcc38ff9640 1 -- 172.21.15.76:0/2806783471 shutdown_connections 2024-03-08T06:01:00.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.775+0000 7fcc38ff9640 1 -- 172.21.15.76:0/2806783471 >> 172.21.15.76:0/2806783471 conn(0x7fcc44009fe0 msgr2=0x7fcc440de840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.775+0000 7fcc38ff9640 1 -- 172.21.15.76:0/2806783471 shutdown_connections 2024-03-08T06:01:00.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.775+0000 7fcc38ff9640 1 -- 172.21.15.76:0/2806783471 wait complete. 2024-03-08T06:01:00.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.775+0000 7ffb04ff9640 1 -- 172.21.15.76:0/4181149575 <== osd.6 v2:172.21.15.81:6824/3040851989 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffb080b8760 con 0x7ffaf4074bc0 2024-03-08T06:01:00.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.776+0000 7ffaee7fc640 1 -- 172.21.15.76:0/4181149575 >> [v2:172.21.15.81:6824/3040851989,v1:172.21.15.81:6825/3040851989] conn(0x7ffaf4074bc0 msgr2=0x7ffaf4077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.776+0000 7ffaee7fc640 1 --2- 172.21.15.76:0/4181149575 >> [v2:172.21.15.81:6824/3040851989,v1:172.21.15.81:6825/3040851989] conn(0x7ffaf4074bc0 0x7ffaf4077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.776+0000 7ffaee7fc640 1 -- 172.21.15.76:0/4181149575 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffaf406b4d0 msgr2=0x7ffaf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.777+0000 7ffaee7fc640 1 --2- 172.21.15.76:0/4181149575 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffaf406b4d0 0x7ffaf406d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffafc036000 tx=0x7ffafc03c040 comp rx=0 tx=0).stop 2024-03-08T06:01:00.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.777+0000 7ffaee7fc640 1 -- 172.21.15.76:0/4181149575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb08163d70 msgr2=0x7ffb080d4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:00.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.777+0000 7ffaee7fc640 1 --2- 172.21.15.76:0/4181149575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb08163d70 0x7ffb080d4060 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffb08163040 tx=0x7ffb0000aad0 comp rx=0 tx=0).stop 2024-03-08T06:01:00.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.777+0000 7ffb0cf93640 1 -- 172.21.15.76:0/4181149575 reap_dead start 2024-03-08T06:01:00.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.777+0000 7ffaee7fc640 1 -- 172.21.15.76:0/4181149575 shutdown_connections 2024-03-08T06:01:00.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.777+0000 7ffaee7fc640 1 -- 172.21.15.76:0/4181149575 >> 172.21.15.76:0/4181149575 conn(0x7ffb080670d0 msgr2=0x7ffb080b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:00.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.778+0000 7ffaee7fc640 1 -- 172.21.15.76:0/4181149575 shutdown_connections 2024-03-08T06:01:00.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:00.778+0000 7ffaee7fc640 1 -- 172.21.15.76:0/4181149575 wait complete. 2024-03-08T06:01:00.828 INFO:teuthology.orchestra.run.smithi076.stdout:55834574859 2024-03-08T06:01:00.829 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-08T06:01:00.830 INFO:teuthology.orchestra.run.smithi076.stdout:55834574859 2024-03-08T06:01:00.830 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-08T06:01:00.830 INFO:teuthology.orchestra.run.smithi076.stdout:55834574859 2024-03-08T06:01:00.831 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-08T06:01:00.831 INFO:teuthology.orchestra.run.smithi076.stdout:55834574859 2024-03-08T06:01:00.832 INFO:teuthology.orchestra.run.smithi076.stdout:55834574859 2024-03-08T06:01:00.832 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-08T06:01:00.833 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-08T06:01:00.833 INFO:teuthology.orchestra.run.smithi076.stdout:55834574859 2024-03-08T06:01:00.834 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-08T06:01:00.834 INFO:teuthology.orchestra.run.smithi076.stdout:55834574859 2024-03-08T06:01:00.835 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-08T06:01:00.835 INFO:teuthology.orchestra.run.smithi076.stdout:55834574860 2024-03-08T06:01:00.836 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-08T06:01:01.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.038+0000 7fc49ca42640 1 Processor -- start 2024-03-08T06:01:01.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.038+0000 7fc49ca42640 1 -- start start 2024-03-08T06:01:01.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.039+0000 7f560b59e640 1 Processor -- start 2024-03-08T06:01:01.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.039+0000 7f560b59e640 1 -- start start 2024-03-08T06:01:01.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.039+0000 7fc49ca42640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc49004a310 0x7fc490039b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.039+0000 7fc49ca42640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49003b330 con 0x7fc4900361a0 2024-03-08T06:01:01.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.039+0000 7fc49ca42640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49003b5d0 con 0x7fc4900529e0 2024-03-08T06:01:01.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.039+0000 7fc49ca42640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49003b8b0 con 0x7fc49004a310 2024-03-08T06:01:01.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.040+0000 7f907e872640 1 Processor -- start 2024-03-08T06:01:01.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.040+0000 7f907e872640 1 -- start start 2024-03-08T06:01:01.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.040+0000 7fc497fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc4900529e0 0x7fc490054860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:40086/0 (socket says 172.21.15.76:40086) 2024-03-08T06:01:01.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.040+0000 7fc497fff640 1 -- 172.21.15.76:0/2307668850 learned_addr learned my addr 172.21.15.76:0/2307668850 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:01.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.040+0000 7f1659a3b640 1 Processor -- start 2024-03-08T06:01:01.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.040+0000 7f1659a3b640 1 -- start start 2024-03-08T06:01:01.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.040+0000 7fc4977fe640 1 --2- 172.21.15.76:0/2307668850 >> v2:172.21.15.81:3300/0 conn(0x7fc49004a310 0x7fc490039b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.040+0000 7f560b59e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f56040535a0 0x7f56040e7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f560b59e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5604056ee0 con 0x7f5604006210 2024-03-08T06:01:01.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f560b59e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56040571c0 con 0x7f5604058b10 2024-03-08T06:01:01.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f560b59e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5604057af0 con 0x7f56040535a0 2024-03-08T06:01:01.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.040+0000 7f183e7ec640 1 Processor -- start 2024-03-08T06:01:01.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f183e7ec640 1 -- start start 2024-03-08T06:01:01.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f560ad9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5604006210 0x7f56040066a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57568/0 (socket says 172.21.15.76:57568) 2024-03-08T06:01:01.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f560ad9d640 1 -- 172.21.15.76:0/2460417168 learned_addr learned my addr 172.21.15.76:0/2460417168 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:01.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f907e872640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9070055540 0x7f9070009240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f907e872640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9070050bf0 0x7f9070006b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f907e872640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9070007090 0x7f90700e2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f907e872640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9070052010 con 0x7f9070055540 2024-03-08T06:01:01.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f907e872640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9070051400 con 0x7f9070050bf0 2024-03-08T06:01:01.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f907e872640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90700516e0 con 0x7f9070007090 2024-03-08T06:01:01.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f560959a640 1 -- 172.21.15.76:0/2460417168 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792487915 0 0) 0x7f5604056ee0 con 0x7f5604006210 2024-03-08T06:01:01.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f560959a640 1 -- 172.21.15.76:0/2460417168 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55f4003520 con 0x7f5604006210 2024-03-08T06:01:01.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f560a59c640 1 --2- 172.21.15.76:0/2460417168 >> v2:172.21.15.81:3300/0 conn(0x7f56040535a0 0x7f56040e7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f1659a3b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1654161cd0 0x7f16540cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f1659a3b640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16540b3830 0x7f165406da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f907d870640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9070055540 0x7f9070009240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f907d870640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9070055540 0x7f9070009240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51410/0 (socket says 172.21.15.76:51410) 2024-03-08T06:01:01.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f907d870640 1 -- 172.21.15.76:0/2419543875 learned_addr learned my addr 172.21.15.76:0/2419543875 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:01.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f907e071640 1 --2- 172.21.15.76:0/2419543875 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9070007090 0x7f90700e2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f1659a3b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16540b2710 con 0x7f1654166350 2024-03-08T06:01:01.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f1659a3b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16540cf1f0 con 0x7f1654161cd0 2024-03-08T06:01:01.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f1653fff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16540b3830 0x7f165406da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f1653fff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16540b3830 0x7f165406da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37246/0 (socket says 172.21.15.76:37246) 2024-03-08T06:01:01.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f1653fff640 1 -- 172.21.15.76:0/2467935586 learned_addr learned my addr 172.21.15.76:0/2467935586 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:01.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f1659a3b640 1 -- 172.21.15.76:0/2467935586 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16540b29b0 con 0x7f16540b3830 2024-03-08T06:01:01.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f15bc9e3640 1 Processor -- start 2024-03-08T06:01:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f15bc9e3640 1 -- start start 2024-03-08T06:01:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f183e7ec640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f18300e3360 0x7f1830006c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f183e7ec640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f18300537b0 0x7f1830053c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f183e7ec640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18300545e0 con 0x7f1830054150 2024-03-08T06:01:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f183e7ec640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1830007bd0 con 0x7f18300537b0 2024-03-08T06:01:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f183e7ec640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18300e3740 con 0x7f18300e3360 2024-03-08T06:01:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f0844db6640 1 Processor -- start 2024-03-08T06:01:01.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7f0844db6640 1 -- start start 2024-03-08T06:01:01.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f907d06f640 1 --2- 172.21.15.76:0/2419543875 >> v2:172.21.15.81:3300/0 conn(0x7f9070050bf0 0x7f9070006b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f907e071640 1 -- 172.21.15.76:0/2419543875 >> v2:172.21.15.81:3300/0 conn(0x7f9070050bf0 msgr2=0x7f9070006b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:01.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f907e071640 1 --2- 172.21.15.76:0/2419543875 >> v2:172.21.15.81:3300/0 conn(0x7f9070050bf0 0x7f9070006b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f183d7ea640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f18300e3360 0x7f1830006c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f183d7ea640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f18300e3360 0x7f1830006c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.76:57302/0 (socket says 172.21.15.76:57302) 2024-03-08T06:01:01.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f183d7ea640 1 -- 172.21.15.76:0/3004447835 learned_addr learned my addr 172.21.15.76:0/3004447835 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:01.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7fc4967fc640 1 -- 172.21.15.76:0/2307668850 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 62258737 0 0) 0x7fc49003b5d0 con 0x7fc4900529e0 2024-03-08T06:01:01.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.041+0000 7fc4967fc640 1 -- 172.21.15.76:0/2307668850 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc488003520 con 0x7fc4900529e0 2024-03-08T06:01:01.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7fc4977fe640 1 -- 172.21.15.76:0/2307668850 >> v1:172.21.15.81:6789/0 conn(0x7fc4900529e0 legacy=0x7fc490054860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7fc4977fe640 1 -- 172.21.15.76:0/2307668850 >> v1:172.21.15.76:6789/0 conn(0x7fc4900361a0 legacy=0x7fc49003c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7fc4977fe640 1 -- 172.21.15.76:0/2307668850 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49003bb90 con 0x7fc49004a310 2024-03-08T06:01:01.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f907e071640 1 -- 172.21.15.76:0/2419543875 >> v2:172.21.15.76:3300/0 conn(0x7f9070055540 msgr2=0x7f9070009240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:01.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f907e071640 1 --2- 172.21.15.76:0/2419543875 >> v2:172.21.15.76:3300/0 conn(0x7f9070055540 0x7f9070009240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7f907e071640 1 -- 172.21.15.76:0/2419543875 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90700519c0 con 0x7f9070007090 2024-03-08T06:01:01.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f1653fff640 1 -- 172.21.15.76:0/2467935586 >> v2:172.21.15.76:3300/0 conn(0x7f1654161cd0 msgr2=0x7f16540cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:01.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f1653fff640 1 --2- 172.21.15.76:0/2467935586 >> v2:172.21.15.76:3300/0 conn(0x7f1654161cd0 0x7f16540cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f1653fff640 1 -- 172.21.15.76:0/2467935586 >> v1:172.21.15.81:6789/0 conn(0x7f1654166350 legacy=0x7f16540b3120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-08T06:01:01.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f560a59c640 1 -- 172.21.15.76:0/2460417168 >> v1:172.21.15.81:6789/0 conn(0x7f5604058b10 legacy=0x7f5604053980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f560a59c640 1 -- 172.21.15.76:0/2460417168 >> v1:172.21.15.76:6789/0 conn(0x7f5604006210 legacy=0x7f56040066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f560a59c640 1 -- 172.21.15.76:0/2460417168 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56040574a0 con 0x7f56040535a0 2024-03-08T06:01:01.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f1653fff640 1 -- 172.21.15.76:0/2467935586 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16540b2c90 con 0x7f16540b3830 2024-03-08T06:01:01.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f182ffff640 1 -- 172.21.15.76:0/3004447835 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1797829384 0 0) 0x7f1828001110 con 0x7f1830054150 2024-03-08T06:01:01.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f182ffff640 1 -- 172.21.15.76:0/3004447835 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1824003520 con 0x7f1830054150 2024-03-08T06:01:01.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f183cfe9640 1 --2- 172.21.15.76:0/3004447835 >> v2:172.21.15.76:3300/0 conn(0x7f18300537b0 0x7f1830053c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7fc86a884640 1 Processor -- start 2024-03-08T06:01:01.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.042+0000 7fc86a884640 1 -- start start 2024-03-08T06:01:01.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7fc4977fe640 1 --2- 172.21.15.76:0/2307668850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc49004a310 0x7fc490039b70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc4840097a0 tx=0x7fc48402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c8d781385580542 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7fc86a884640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc85c04f930 0x7fc85c04fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7fc86a884640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85c037c50 0x7fc85c0380c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7fc86a884640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85c0536b0 con 0x7fc85c033d70 2024-03-08T06:01:01.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f907d06f640 1 --2- 172.21.15.76:0/2419543875 >> v2:172.21.15.81:3300/0 conn(0x7f9070050bf0 0x7f9070006b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:01.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f0844db6640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08400b27f0 0x7f08400cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7fc86a884640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85c0502d0 con 0x7fc85c04f930 2024-03-08T06:01:01.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7fc86a884640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c053990 con 0x7fc85c037c50 2024-03-08T06:01:01.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7fc868e21640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85c037c50 0x7fc85c0380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7fc868e21640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85c037c50 0x7fc85c0380c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37270/0 (socket says 172.21.15.76:37270) 2024-03-08T06:01:01.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7fc868e21640 1 -- 172.21.15.76:0/3664143720 learned_addr learned my addr 172.21.15.76:0/3664143720 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:01.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f0844db6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0840165c90 con 0x7f0840161fe0 2024-03-08T06:01:01.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f0844db6640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0840165f30 con 0x7f08400cfb90 2024-03-08T06:01:01.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f907e071640 1 --2- 172.21.15.76:0/2419543875 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9070007090 0x7f90700e2840 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9078000b60 tx=0x7f90780a6800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0c1591f488413d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f0844db6640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08400b4770 con 0x7f08400b27f0 2024-03-08T06:01:01.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f083e59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0840161fe0 0x7f08400b4060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57574/0 (socket says 172.21.15.76:57574) 2024-03-08T06:01:01.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f083e59c640 1 -- 172.21.15.76:0/24296726 learned_addr learned my addr 172.21.15.76:0/24296726 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:01.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f083dd9b640 1 --2- 172.21.15.76:0/24296726 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08400b27f0 0x7f08400cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f083d59a640 1 -- 172.21.15.76:0/24296726 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592591618 0 0) 0x7f0840165f30 con 0x7f08400cfb90 2024-03-08T06:01:01.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f15bc9e3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f15b00e7b30 0x7f15b0054270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f083d59a640 1 -- 172.21.15.76:0/24296726 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0828003520 con 0x7f08400cfb90 2024-03-08T06:01:01.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f182ffff640 1 -- 172.21.15.76:0/3004447835 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3158710004 0 0) 0x7f1824003520 con 0x7f1830054150 2024-03-08T06:01:01.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f182ffff640 1 -- 172.21.15.76:0/3004447835 >> v2:172.21.15.81:3300/0 conn(0x7f18300e3360 msgr2=0x7f1830006c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:01.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f182ffff640 1 --2- 172.21.15.76:0/3004447835 >> v2:172.21.15.81:3300/0 conn(0x7f18300e3360 0x7f1830006c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f182ffff640 1 -- 172.21.15.76:0/3004447835 >> v2:172.21.15.76:3300/0 conn(0x7f18300537b0 msgr2=0x7f1830053c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:01.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f182ffff640 1 --2- 172.21.15.76:0/3004447835 >> v2:172.21.15.76:3300/0 conn(0x7f18300537b0 0x7f1830053c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f182ffff640 1 -- 172.21.15.76:0/3004447835 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1830007330 con 0x7f1830054150 2024-03-08T06:01:01.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f182ffff640 1 -- 172.21.15.76:0/3004447835 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f1828001840 con 0x7f1830054150 2024-03-08T06:01:01.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f182ffff640 1 -- 172.21.15.76:0/3004447835 >> v1:172.21.15.81:6789/0 conn(0x7f1830054150 legacy=0x7f18300e6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f182ffff640 1 --2- 172.21.15.76:0/3004447835 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1824003a00 0x7f1824003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f182ffff640 1 --2- 172.21.15.76:0/3004447835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1824004800 0x7f1824004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.043+0000 7f182ffff640 1 --2- 172.21.15.76:0/3004447835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18240051c0 0x7f18240141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f083dd9b640 1 -- 172.21.15.76:0/24296726 >> v1:172.21.15.81:6789/0 conn(0x7f08400cfb90 legacy=0x7f084006d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f083dd9b640 1 -- 172.21.15.76:0/24296726 >> v1:172.21.15.76:6789/0 conn(0x7f0840161fe0 legacy=0x7f08400b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f083dd9b640 1 -- 172.21.15.76:0/24296726 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0840166210 con 0x7f08400b27f0 2024-03-08T06:01:01.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7fc4967fc640 1 -- 172.21.15.76:0/2307668850 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc484034070 con 0x7fc49004a310 2024-03-08T06:01:01.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7fc4967fc640 1 -- 172.21.15.76:0/2307668850 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc484030860 con 0x7fc49004a310 2024-03-08T06:01:01.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7fc4967fc640 1 -- 172.21.15.76:0/2307668850 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc484030c10 con 0x7fc49004a310 2024-03-08T06:01:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f15bc9e3640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15b0039160 0x7f15b0055100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f15bc9e3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b0039640 con 0x7f15b00e7750 2024-03-08T06:01:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f15bc9e3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b0051980 con 0x7f15b00e7b30 2024-03-08T06:01:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f15bc9e3640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b0051c60 con 0x7f15b0039160 2024-03-08T06:01:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f15b659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f15b00e7b30 0x7f15b0054270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f15b759e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15b0039160 0x7f15b0055100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f15b659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f15b00e7b30 0x7f15b0054270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51452/0 (socket says 172.21.15.76:51452) 2024-03-08T06:01:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f15b759e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15b0039160 0x7f15b0055100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37288/0 (socket says 172.21.15.76:37288) 2024-03-08T06:01:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f15b759e640 1 -- 172.21.15.76:0/2286088719 learned_addr learned my addr 172.21.15.76:0/2286088719 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f182ffff640 1 -- 172.21.15.76:0/3004447835 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18300e3740 con 0x7f18240051c0 2024-03-08T06:01:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f182ffff640 1 -- 172.21.15.76:0/3004447835 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18240148b0 con 0x7f1824004800 2024-03-08T06:01:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f182ffff640 1 -- 172.21.15.76:0/3004447835 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1824014ab0 con 0x7f1824003a00 2024-03-08T06:01:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f907c86e640 1 -- 172.21.15.76:0/2419543875 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90780ac620 con 0x7f9070007090 2024-03-08T06:01:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f907c86e640 1 -- 172.21.15.76:0/2419543875 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90780ac7c0 con 0x7f9070007090 2024-03-08T06:01:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f907c86e640 1 -- 172.21.15.76:0/2419543875 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90780acc30 con 0x7f9070007090 2024-03-08T06:01:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f183dfeb640 1 --2- 172.21.15.76:0/3004447835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18240051c0 0x7f18240141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f560a59c640 1 --2- 172.21.15.76:0/2460417168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56040535a0 0x7f56040e7b10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0097a0 tx=0x7f55fc02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38553cf5e6d63e31 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f560959a640 1 -- 172.21.15.76:0/2460417168 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc034070 con 0x7f56040535a0 2024-03-08T06:01:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f560959a640 1 -- 172.21.15.76:0/2460417168 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc030610 con 0x7f56040535a0 2024-03-08T06:01:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f183cfe9640 1 --2- 172.21.15.76:0/3004447835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1824004800 0x7f1824004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7fc868e21640 1 -- 172.21.15.76:0/3664143720 >> v2:172.21.15.76:3300/0 conn(0x7fc85c04f930 msgr2=0x7fc85c04fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7fc868e21640 1 --2- 172.21.15.76:0/3664143720 >> v2:172.21.15.76:3300/0 conn(0x7fc85c04f930 0x7fc85c04fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7fc868e21640 1 -- 172.21.15.76:0/3664143720 >> v1:172.21.15.81:6789/0 conn(0x7fc85c033d70 legacy=0x7fc85c006f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7fc868e21640 1 -- 172.21.15.76:0/3664143720 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c053c70 con 0x7fc85c037c50 2024-03-08T06:01:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f560959a640 1 -- 172.21.15.76:0/2460417168 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc034070 con 0x7f56040535a0 2024-03-08T06:01:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f1653fff640 1 --2- 172.21.15.76:0/2467935586 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16540b3830 0x7f165406da20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f164c009e50 tx=0x7f164c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17491b0223f46fc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f1651ffb640 1 -- 172.21.15.76:0/2467935586 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c02ca70 con 0x7f16540b3830 2024-03-08T06:01:01.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f1651ffb640 1 -- 172.21.15.76:0/2467935586 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f164c02cc10 con 0x7f16540b3830 2024-03-08T06:01:01.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f1651ffb640 1 -- 172.21.15.76:0/2467935586 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c0343f0 con 0x7f16540b3830 2024-03-08T06:01:01.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7fc868e21640 1 --2- 172.21.15.76:0/3664143720 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85c037c50 0x7fc85c0380c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc86402d3c0 tx=0x7fc8640a68b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da72bd57dc2c5d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f083dd9b640 1 --2- 172.21.15.76:0/24296726 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08400b27f0 0x7f08400cf650 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0834009750 tx=0x7f083402b6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1422fa9bb88da38 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f15b759e640 1 -- 172.21.15.76:0/2286088719 >> v2:172.21.15.76:3300/0 conn(0x7f15b00e7b30 msgr2=0x7f15b0054270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:01.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f183d7ea640 1 --2- 172.21.15.76:0/3004447835 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1824003a00 0x7f1824003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f15b759e640 1 --2- 172.21.15.76:0/2286088719 >> v2:172.21.15.76:3300/0 conn(0x7f15b00e7b30 0x7f15b0054270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f183dfeb640 1 -- 172.21.15.76:0/3004447835 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1824003a00 msgr2=0x7f1824003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:01.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f183dfeb640 1 --2- 172.21.15.76:0/3004447835 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1824003a00 0x7f1824003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f183dfeb640 1 -- 172.21.15.76:0/3004447835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1824004800 msgr2=0x7f1824004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:01.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f15b759e640 1 -- 172.21.15.76:0/2286088719 >> v1:172.21.15.76:6789/0 conn(0x7f15b00e7750 legacy=0x7f15b0007c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:01.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f183dfeb640 1 --2- 172.21.15.76:0/3004447835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1824004800 0x7f1824004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f183dfeb640 1 -- 172.21.15.76:0/3004447835 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1824003520 con 0x7f18240051c0 2024-03-08T06:01:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.044+0000 7f15b759e640 1 -- 172.21.15.76:0/2286088719 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b0051f40 con 0x7f15b0039160 2024-03-08T06:01:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f907e872640 1 -- 172.21.15.76:0/2419543875 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9070007090 msgr2=0x7f90700e2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f083d59a640 1 -- 172.21.15.76:0/24296726 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0834037070 con 0x7f08400b27f0 2024-03-08T06:01:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f083d59a640 1 -- 172.21.15.76:0/24296726 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0834030510 con 0x7f08400b27f0 2024-03-08T06:01:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f083d59a640 1 -- 172.21.15.76:0/24296726 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08380019f0 con 0x7f08400b27f0 2024-03-08T06:01:01.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f907e872640 1 --2- 172.21.15.76:0/2419543875 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9070007090 0x7f90700e2840 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9078000b60 tx=0x7f90780a6800 comp rx=0 tx=0).stop 2024-03-08T06:01:01.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f907e872640 1 -- 172.21.15.76:0/2419543875 shutdown_connections 2024-03-08T06:01:01.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f560b59e640 1 -- 172.21.15.76:0/2460417168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56040535a0 msgr2=0x7f56040e7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f560b59e640 1 --2- 172.21.15.76:0/2460417168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56040535a0 0x7f56040e7b10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0097a0 tx=0x7f55fc02b960 comp rx=0 tx=0).stop 2024-03-08T06:01:01.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f907e872640 1 --2- 172.21.15.76:0/2419543875 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9070007090 0x7f90700e2840 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f907e872640 1 --2- 172.21.15.76:0/2419543875 >> v2:172.21.15.81:3300/0 conn(0x7f9070050bf0 0x7f9070006b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f907e872640 1 --2- 172.21.15.76:0/2419543875 >> v2:172.21.15.76:3300/0 conn(0x7f9070055540 0x7f9070009240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7f907e872640 1 -- 172.21.15.76:0/2419543875 >> 172.21.15.76:0/2419543875 conn(0x7f9070055bc0 msgr2=0x7f9070055fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:01.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f15b759e640 1 --2- 172.21.15.76:0/2286088719 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15b0039160 0x7f15b0055100 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f15b0007b90 tx=0x7f15a802b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=803c937010f73d75 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7fc862ffd640 1 -- 172.21.15.76:0/3664143720 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8640b0020 con 0x7fc85c037c50 2024-03-08T06:01:01.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7fc862ffd640 1 -- 172.21.15.76:0/3664143720 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8640ac8d0 con 0x7fc85c037c50 2024-03-08T06:01:01.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.045+0000 7fc862ffd640 1 -- 172.21.15.76:0/3664143720 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8640acd00 con 0x7fc85c037c50 2024-03-08T06:01:01.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7fc49ca42640 1 -- 172.21.15.76:0/2307668850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc49004a310 msgr2=0x7fc490039b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7fc49ca42640 1 --2- 172.21.15.76:0/2307668850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc49004a310 0x7fc490039b70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc4840097a0 tx=0x7fc48402b9a0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f1659a3b640 1 -- 172.21.15.76:0/2467935586 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16540b3830 msgr2=0x7f165406da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7fc49ca42640 1 -- 172.21.15.76:0/2307668850 shutdown_connections 2024-03-08T06:01:01.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7fc49ca42640 1 --2- 172.21.15.76:0/2307668850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc49004a310 0x7fc490039b70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7fc49ca42640 1 -- 172.21.15.76:0/2307668850 >> 172.21.15.76:0/2307668850 conn(0x7fc4900537a0 msgr2=0x7fc490053b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:01.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f560b59e640 1 -- 172.21.15.76:0/2460417168 shutdown_connections 2024-03-08T06:01:01.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f560b59e640 1 --2- 172.21.15.76:0/2460417168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56040535a0 0x7f56040e7b10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f560b59e640 1 -- 172.21.15.76:0/2460417168 >> 172.21.15.76:0/2460417168 conn(0x7f560403aa40 msgr2=0x7f560403ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:01.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f15b5d9b640 1 -- 172.21.15.76:0/2286088719 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a80317c0 con 0x7f15b0039160 2024-03-08T06:01:01.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f1659a3b640 1 --2- 172.21.15.76:0/2467935586 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16540b3830 0x7f165406da20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f164c009e50 tx=0x7f164c02bed0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f183dfeb640 1 -- 172.21.15.76:0/3004447835 reap_dead start 2024-03-08T06:01:01.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f560b59e640 1 -- 172.21.15.76:0/2460417168 shutdown_connections 2024-03-08T06:01:01.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f15b5d9b640 1 -- 172.21.15.76:0/2286088719 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a8037070 con 0x7f15b0039160 2024-03-08T06:01:01.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7fc49ca42640 1 -- 172.21.15.76:0/2307668850 shutdown_connections 2024-03-08T06:01:01.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f560b59e640 1 -- 172.21.15.76:0/2460417168 wait complete. 2024-03-08T06:01:01.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f15b5d9b640 1 -- 172.21.15.76:0/2286088719 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8031bc0 con 0x7f15b0039160 2024-03-08T06:01:01.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7fc86a884640 1 -- 172.21.15.76:0/3664143720 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85c037c50 msgr2=0x7fc85c0380c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7fc86a884640 1 --2- 172.21.15.76:0/3664143720 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85c037c50 0x7fc85c0380c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc86402d3c0 tx=0x7fc8640a68b0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f0844db6640 1 -- 172.21.15.76:0/24296726 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08400b27f0 msgr2=0x7f08400cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f0844db6640 1 --2- 172.21.15.76:0/24296726 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08400b27f0 0x7f08400cf650 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0834009750 tx=0x7f083402b6f0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f907e872640 1 -- 172.21.15.76:0/2419543875 shutdown_connections 2024-03-08T06:01:01.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f1659a3b640 1 -- 172.21.15.76:0/2467935586 shutdown_connections 2024-03-08T06:01:01.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f1659a3b640 1 --2- 172.21.15.76:0/2467935586 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16540b3830 0x7f165406da20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f1659a3b640 1 --2- 172.21.15.76:0/2467935586 >> v2:172.21.15.76:3300/0 conn(0x7f1654161cd0 0x7f16540cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f1659a3b640 1 -- 172.21.15.76:0/2467935586 >> 172.21.15.76:0/2467935586 conn(0x7f16540670c0 msgr2=0x7f165406acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:01.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f1659a3b640 1 -- 172.21.15.76:0/2467935586 shutdown_connections 2024-03-08T06:01:01.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f1659a3b640 1 -- 172.21.15.76:0/2467935586 wait complete. 2024-03-08T06:01:01.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7fc86a884640 1 -- 172.21.15.76:0/3664143720 shutdown_connections 2024-03-08T06:01:01.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7fc86a884640 1 --2- 172.21.15.76:0/3664143720 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85c037c50 0x7fc85c0380c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7fc86a884640 1 --2- 172.21.15.76:0/3664143720 >> v2:172.21.15.76:3300/0 conn(0x7fc85c04f930 0x7fc85c04fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7fc86a884640 1 -- 172.21.15.76:0/3664143720 >> 172.21.15.76:0/3664143720 conn(0x7fc85c009fe0 msgr2=0x7fc85c00a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:01.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f0844db6640 1 -- 172.21.15.76:0/24296726 shutdown_connections 2024-03-08T06:01:01.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f0844db6640 1 --2- 172.21.15.76:0/24296726 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08400b27f0 0x7f08400cf650 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7fc86a884640 1 -- 172.21.15.76:0/3664143720 shutdown_connections 2024-03-08T06:01:01.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f183dfeb640 1 --2- 172.21.15.76:0/3004447835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18240051c0 0x7f18240141d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f182800c240 tx=0x7f1824005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a02c4edcf93c7aa server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7f0844db6640 1 -- 172.21.15.76:0/24296726 >> 172.21.15.76:0/24296726 conn(0x7f084006ac80 msgr2=0x7f084006b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:01.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.047+0000 7f907e872640 1 -- 172.21.15.76:0/2419543875 wait complete. 2024-03-08T06:01:01.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.047+0000 7f0844db6640 1 -- 172.21.15.76:0/24296726 shutdown_connections 2024-03-08T06:01:01.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7fc49ca42640 1 -- 172.21.15.76:0/2307668850 wait complete. 2024-03-08T06:01:01.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.047+0000 7f182ffff640 1 -- 172.21.15.76:0/3004447835 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828039070 con 0x7f18240051c0 2024-03-08T06:01:01.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.047+0000 7f182ffff640 1 -- 172.21.15.76:0/3004447835 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182802cb50 con 0x7f18240051c0 2024-03-08T06:01:01.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.046+0000 7fc86a884640 1 -- 172.21.15.76:0/3664143720 wait complete. 2024-03-08T06:01:01.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.048+0000 7f183e7ec640 1 -- 172.21.15.76:0/3004447835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18240051c0 msgr2=0x7f18240141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.048+0000 7f183e7ec640 1 --2- 172.21.15.76:0/3004447835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18240051c0 0x7f18240141d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f182800c240 tx=0x7f1824005830 comp rx=0 tx=0).stop 2024-03-08T06:01:01.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.048+0000 7fc49ca42640 1 Processor -- start 2024-03-08T06:01:01.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.047+0000 7f15bc9e3640 1 -- 172.21.15.76:0/2286088719 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15b0039160 msgr2=0x7f15b0055100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.048+0000 7f15bc9e3640 1 --2- 172.21.15.76:0/2286088719 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15b0039160 0x7f15b0055100 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f15b0007b90 tx=0x7f15a802b860 comp rx=0 tx=0).stop 2024-03-08T06:01:01.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.048+0000 7fc86a884640 1 Processor -- start 2024-03-08T06:01:01.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.048+0000 7fc49ca42640 1 -- start start 2024-03-08T06:01:01.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.048+0000 7f183e7ec640 1 -- 172.21.15.76:0/3004447835 shutdown_connections 2024-03-08T06:01:01.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.048+0000 7f183e7ec640 1 --2- 172.21.15.76:0/3004447835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18240051c0 0x7f18240141d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.048+0000 7f183e7ec640 1 -- 172.21.15.76:0/3004447835 >> 172.21.15.76:0/3004447835 conn(0x7f183003aa40 msgr2=0x7f183003ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:01.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.048+0000 7f1659a3b640 1 Processor -- start 2024-03-08T06:01:01.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.048+0000 7fc86a884640 1 -- start start 2024-03-08T06:01:01.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.048+0000 7f907e872640 1 Processor -- start 2024-03-08T06:01:01.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.047+0000 7f0844db6640 1 -- 172.21.15.76:0/24296726 wait complete. 2024-03-08T06:01:01.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.048+0000 7f0844db6640 1 Processor -- start 2024-03-08T06:01:01.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7f183e7ec640 1 -- 172.21.15.76:0/3004447835 shutdown_connections 2024-03-08T06:01:01.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7f15bc9e3640 1 -- 172.21.15.76:0/2286088719 shutdown_connections 2024-03-08T06:01:01.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7f15bc9e3640 1 --2- 172.21.15.76:0/2286088719 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15b0039160 0x7f15b0055100 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7f15bc9e3640 1 --2- 172.21.15.76:0/2286088719 >> v2:172.21.15.76:3300/0 conn(0x7f15b00e7b30 0x7f15b0054270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7f15bc9e3640 1 -- 172.21.15.76:0/2286088719 >> 172.21.15.76:0/2286088719 conn(0x7f15b0053750 msgr2=0x7f15b0053b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:01.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7f1659a3b640 1 -- start start 2024-03-08T06:01:01.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7f0844db6640 1 -- start start 2024-03-08T06:01:01.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7f183e7ec640 1 -- 172.21.15.76:0/3004447835 wait complete. 2024-03-08T06:01:01.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7fc86a884640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85c033d70 0x7fc85c0e7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7fc86a884640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc85c037c50 0x7fc85c04e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7fc86a884640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc85c04f930 0x7fc85c0d3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7fc86a884640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c054510 con 0x7fc85c04f930 2024-03-08T06:01:01.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7fc86a884640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c054230 con 0x7fc85c037c50 2024-03-08T06:01:01.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7fc86a884640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c034150 con 0x7fc85c033d70 2024-03-08T06:01:01.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.048+0000 7f560b59e640 1 Processor -- start 2024-03-08T06:01:01.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.048+0000 7f560b59e640 1 -- start start 2024-03-08T06:01:01.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7f560b59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5604006210 0x7f560404f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7f560b59e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56040535a0 0x7f56040cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7f560b59e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5604058b10 0x7f5604039e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7fc49ca42640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4900361a0 0x7fc4900dd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7f560b59e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5604057f80 con 0x7f5604006210 2024-03-08T06:01:01.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7f560b59e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56040e1fa0 con 0x7f56040535a0 2024-03-08T06:01:01.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7f560b59e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5604056b20 con 0x7f5604058b10 2024-03-08T06:01:01.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7fc49ca42640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc4900529e0 0x7fc490051390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7fc49ca42640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4900518d0 0x7fc490181250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7fc868e21640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc85c04f930 0x7fc85c0d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7fc868e21640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc85c04f930 0x7fc85c0d3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51454/0 (socket says 172.21.15.76:51454) 2024-03-08T06:01:01.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7fc868e21640 1 -- 172.21.15.76:0/1734842674 learned_addr learned my addr 172.21.15.76:0/1734842674 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:01.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7fc863fff640 1 --2- 172.21.15.76:0/1734842674 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85c033d70 0x7fc85c0e7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.049+0000 7f907e872640 1 -- start start 2024-03-08T06:01:01.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f907e872640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9070007090 0x7f907004dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f5609d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56040535a0 0x7f56040cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f907e872640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9070050bf0 0x7f907004efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f907e872640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9070055540 0x7f9070186d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f907e872640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9070051050 con 0x7f9070050bf0 2024-03-08T06:01:01.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f907e872640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9070055920 con 0x7f9070007090 2024-03-08T06:01:01.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f907e872640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9070180030 con 0x7f9070055540 2024-03-08T06:01:01.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f5609d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56040535a0 0x7f56040cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.76:57324/0 (socket says 172.21.15.76:57324) 2024-03-08T06:01:01.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f5609d9b640 1 -- 172.21.15.76:0/3113036432 learned_addr learned my addr 172.21.15.76:0/3113036432 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:01.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7fc49ca42640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4900eb4c0 con 0x7fc4900518d0 2024-03-08T06:01:01.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7fc49ca42640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4900eb340 con 0x7fc4900361a0 2024-03-08T06:01:01.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7fc49ca42640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc490054f70 con 0x7fc4900529e0 2024-03-08T06:01:01.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f0844db6640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08400b27f0 0x7f084015f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7fc497fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4900518d0 0x7fc490181250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7fc4977fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4900361a0 0x7fc4900dd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7fc497fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4900518d0 0x7fc490181250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51470/0 (socket says 172.21.15.76:51470) 2024-03-08T06:01:01.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7fc497fff640 1 -- 172.21.15.76:0/1266182071 learned_addr learned my addr 172.21.15.76:0/1266182071 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:01.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7fc8637fe640 1 --2- 172.21.15.76:0/1734842674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc85c037c50 0x7fc85c04e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f15bc9e3640 1 -- 172.21.15.76:0/2286088719 shutdown_connections 2024-03-08T06:01:01.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f0844db6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08400cfb90 0x7f084015feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f15bc9e3640 1 -- 172.21.15.76:0/2286088719 wait complete. 2024-03-08T06:01:01.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f907e071640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9070055540 0x7f9070186d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f907d06f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9070050bf0 0x7f907004efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f907d06f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9070050bf0 0x7f907004efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51478/0 (socket says 172.21.15.76:51478) 2024-03-08T06:01:01.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f907d06f640 1 -- 172.21.15.76:0/2077858444 learned_addr learned my addr 172.21.15.76:0/2077858444 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:01.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f1659a3b640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16540b3830 0x7f165414d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f0844db6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0840161fe0 0x7f084014fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f0844db6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08401623c0 con 0x7f0840161fe0 2024-03-08T06:01:01.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f0844db6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08400b49b0 con 0x7f08400cfb90 2024-03-08T06:01:01.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f0844db6640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0840166a10 con 0x7f08400b27f0 2024-03-08T06:01:01.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f907d870640 1 --2- 172.21.15.76:0/2077858444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9070007090 0x7f907004dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f1659a3b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1654161cd0 0x7f165415c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f083dd9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08400cfb90 0x7f084015feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f183e7ec640 1 Processor -- start 2024-03-08T06:01:01.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f083dd9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08400cfb90 0x7f084015feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.76:57366/0 (socket says 172.21.15.76:57366) 2024-03-08T06:01:01.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f083dd9b640 1 -- 172.21.15.76:0/3553323628 learned_addr learned my addr 172.21.15.76:0/3553323628 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:01.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f083e59c640 1 --2- 172.21.15.76:0/3553323628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08400b27f0 0x7f084015f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7fc863fff640 1 -- 172.21.15.76:0/1734842674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc85c037c50 msgr2=0x7fc85c04e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7fc863fff640 1 --2- 172.21.15.76:0/1734842674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc85c037c50 0x7fc85c04e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7fc863fff640 1 -- 172.21.15.76:0/1734842674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc85c04f930 msgr2=0x7fc85c0d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7fc863fff640 1 --2- 172.21.15.76:0/1734842674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc85c04f930 0x7fc85c0d3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7fc863fff640 1 -- 172.21.15.76:0/1734842674 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c0d4670 con 0x7fc85c033d70 2024-03-08T06:01:01.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f1652ffd640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16540b3830 0x7f165414d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f1652ffd640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16540b3830 0x7f165414d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37334/0 (socket says 172.21.15.76:37334) 2024-03-08T06:01:01.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f1652ffd640 1 -- 172.21.15.76:0/225287124 learned_addr learned my addr 172.21.15.76:0/225287124 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:01.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7fc496ffd640 1 --2- 172.21.15.76:0/1266182071 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc4900529e0 0x7fc490051390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7fc4977fe640 1 -- 172.21.15.76:0/1266182071 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc4900529e0 msgr2=0x7fc490051390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f16527fc640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1654161cd0 0x7f165415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f183e7ec640 1 -- start start 2024-03-08T06:01:01.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7fc4977fe640 1 --2- 172.21.15.76:0/1266182071 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc4900529e0 0x7fc490051390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7fc4977fe640 1 -- 172.21.15.76:0/1266182071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4900518d0 msgr2=0x7fc490181250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7fc4977fe640 1 --2- 172.21.15.76:0/1266182071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4900518d0 0x7fc490181250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7fc4977fe640 1 -- 172.21.15.76:0/1266182071 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49004b9e0 con 0x7fc4900361a0 2024-03-08T06:01:01.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f1659a3b640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1654166350 0x7f1654160930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f560ad9d640 1 --2- 172.21.15.76:0/3113036432 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5604058b10 0x7f5604039e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f560a59c640 1 --2- 172.21.15.76:0/3113036432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5604006210 0x7f560404f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f5609d9b640 1 -- 172.21.15.76:0/3113036432 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5604058b10 msgr2=0x7f5604039e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f5609d9b640 1 --2- 172.21.15.76:0/3113036432 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5604058b10 0x7f5604039e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f5609d9b640 1 -- 172.21.15.76:0/3113036432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5604006210 msgr2=0x7f560404f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f1659a3b640 1 -- 172.21.15.76:0/225287124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16541667b0 con 0x7f1654166350 2024-03-08T06:01:01.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f5609d9b640 1 --2- 172.21.15.76:0/3113036432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5604006210 0x7f560404f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.050+0000 7f5609d9b640 1 -- 172.21.15.76:0/3113036432 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560404f8d0 con 0x7f56040535a0 2024-03-08T06:01:01.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f5609d9b640 1 --2- 172.21.15.76:0/3113036432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56040535a0 0x7f56040cdaf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f560000c3c0 tx=0x7f560000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7fc863fff640 1 --2- 172.21.15.76:0/1734842674 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85c033d70 0x7fc85c0e7020 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc85800c390 tx=0x7fc85800c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f1659a3b640 1 -- 172.21.15.76:0/225287124 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16540cf770 con 0x7f1654161cd0 2024-03-08T06:01:01.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f1659a3b640 1 -- 172.21.15.76:0/225287124 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16541620b0 con 0x7f16540b3830 2024-03-08T06:01:01.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f55faffd640 1 -- 172.21.15.76:0/3113036432 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600005220 con 0x7f56040535a0 2024-03-08T06:01:01.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f083ed9d640 1 --2- 172.21.15.76:0/3553323628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0840161fe0 0x7f084014fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f1653fff640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1654166350 0x7f1654160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f907d870640 1 -- 172.21.15.76:0/2077858444 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9070055540 msgr2=0x7f9070186d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f907d870640 1 --2- 172.21.15.76:0/2077858444 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9070055540 0x7f9070186d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f907d870640 1 -- 172.21.15.76:0/2077858444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9070050bf0 msgr2=0x7f907004efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f907d870640 1 --2- 172.21.15.76:0/2077858444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9070050bf0 0x7f907004efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f907d870640 1 -- 172.21.15.76:0/2077858444 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90700df230 con 0x7f9070007090 2024-03-08T06:01:01.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f55faffd640 1 -- 172.21.15.76:0/3113036432 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56000053c0 con 0x7f56040535a0 2024-03-08T06:01:01.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f55faffd640 1 -- 172.21.15.76:0/3113036432 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600007250 con 0x7f56040535a0 2024-03-08T06:01:01.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f907d06f640 1 --2- 172.21.15.76:0/2077858444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9070050bf0 0x7f907004efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:01.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7fc4977fe640 1 --2- 172.21.15.76:0/1266182071 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4900361a0 0x7fc4900dd0f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc484033000 tx=0x7fc4840046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7fc483fff640 1 -- 172.21.15.76:0/1266182071 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc484048070 con 0x7fc4900361a0 2024-03-08T06:01:01.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f083e59c640 1 -- 172.21.15.76:0/3553323628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08400cfb90 msgr2=0x7f084015feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f083e59c640 1 --2- 172.21.15.76:0/3553323628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08400cfb90 0x7f084015feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f083e59c640 1 -- 172.21.15.76:0/3553323628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0840161fe0 msgr2=0x7f084014fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f083e59c640 1 --2- 172.21.15.76:0/3553323628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0840161fe0 0x7f084014fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7f083e59c640 1 -- 172.21.15.76:0/3553323628 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0840160510 con 0x7f08400b27f0 2024-03-08T06:01:01.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7fc483fff640 1 -- 172.21.15.76:0/1266182071 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc484030660 con 0x7fc4900361a0 2024-03-08T06:01:01.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7fc483fff640 1 -- 172.21.15.76:0/1266182071 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc484034050 con 0x7fc4900361a0 2024-03-08T06:01:01.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7fc49ca42640 1 -- 172.21.15.76:0/1266182071 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49004bcc0 con 0x7fc4900361a0 2024-03-08T06:01:01.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f1652ffd640 1 -- 172.21.15.76:0/225287124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1654161cd0 msgr2=0x7f165415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f1652ffd640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1654161cd0 0x7f165415c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7fc49ca42640 1 -- 172.21.15.76:0/1266182071 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4900dd630 con 0x7fc4900361a0 2024-03-08T06:01:01.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f1652ffd640 1 -- 172.21.15.76:0/225287124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1654166350 msgr2=0x7f1654160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f183e7ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18240051c0 0x7f18300d1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f183e7ec640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f18300de550 0x7f1830052cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f183e7ec640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18300521d0 0x7f183000b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f183e7ec640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18300572a0 con 0x7f18240051c0 2024-03-08T06:01:01.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f183e7ec640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18300078f0 con 0x7f18300521d0 2024-03-08T06:01:01.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f183e7ec640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1830053c10 con 0x7f18300de550 2024-03-08T06:01:01.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f183d7ea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18240051c0 0x7f18300d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f183d7ea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18240051c0 0x7f18300d1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51512/0 (socket says 172.21.15.76:51512) 2024-03-08T06:01:01.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f183d7ea640 1 -- 172.21.15.76:0/2451316532 learned_addr learned my addr 172.21.15.76:0/2451316532 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:01.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f1652ffd640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1654166350 0x7f1654160930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f1652ffd640 1 -- 172.21.15.76:0/225287124 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16541650e0 con 0x7f16540b3830 2024-03-08T06:01:01.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7fc860ff9640 1 -- 172.21.15.76:0/1734842674 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858017070 con 0x7fc85c033d70 2024-03-08T06:01:01.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.051+0000 7fc860ff9640 1 -- 172.21.15.76:0/1734842674 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8580052d0 con 0x7fc85c033d70 2024-03-08T06:01:01.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7fc86a884640 1 -- 172.21.15.76:0/1734842674 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c0d4950 con 0x7fc85c033d70 2024-03-08T06:01:01.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7fc86a884640 1 -- 172.21.15.76:0/1734842674 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c0d4f10 con 0x7fc85c033d70 2024-03-08T06:01:01.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f1653fff640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1654166350 0x7f1654160930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:01.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f16527fc640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1654161cd0 0x7f165415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:01.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f907d870640 1 --2- 172.21.15.76:0/2077858444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9070007090 0x7f907004dfb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f907400e870 tx=0x7f907400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f183dfeb640 1 --2- 172.21.15.76:0/2451316532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18300521d0 0x7f183000b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.052+0000 7f1652ffd640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16540b3830 0x7f165414d900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f164400b4a0 tx=0x7f164400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f183cfe9640 1 --2- 172.21.15.76:0/2451316532 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f18300de550 0x7f1830052cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f183d7ea640 1 -- 172.21.15.76:0/2451316532 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f18300de550 msgr2=0x7f1830052cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f183d7ea640 1 --2- 172.21.15.76:0/2451316532 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f18300de550 0x7f1830052cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f183d7ea640 1 -- 172.21.15.76:0/2451316532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18300521d0 msgr2=0x7f183000b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f183d7ea640 1 --2- 172.21.15.76:0/2451316532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18300521d0 0x7f183000b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f183d7ea640 1 -- 172.21.15.76:0/2451316532 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18300d18b0 con 0x7f18240051c0 2024-03-08T06:01:01.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f906e7fc640 1 -- 172.21.15.76:0/2077858444 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074017070 con 0x7f9070007090 2024-03-08T06:01:01.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f907e872640 1 -- 172.21.15.76:0/2077858444 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90700df510 con 0x7f9070007090 2024-03-08T06:01:01.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f083e59c640 1 --2- 172.21.15.76:0/3553323628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08400b27f0 0x7f084015f970 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0830000f60 tx=0x7f083000c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f163bfff640 1 -- 172.21.15.76:0/225287124 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1644014070 con 0x7f16540b3830 2024-03-08T06:01:01.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f560b59e640 1 -- 172.21.15.76:0/3113036432 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5604052020 con 0x7f56040535a0 2024-03-08T06:01:01.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f560b59e640 1 -- 172.21.15.76:0/3113036432 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56040dd980 con 0x7f56040535a0 2024-03-08T06:01:01.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f15bc9e3640 1 Processor -- start 2024-03-08T06:01:01.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f163bfff640 1 -- 172.21.15.76:0/225287124 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1644010610 con 0x7f16540b3830 2024-03-08T06:01:01.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f163bfff640 1 -- 172.21.15.76:0/225287124 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16440109e0 con 0x7f16540b3830 2024-03-08T06:01:01.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f907e872640 1 -- 172.21.15.76:0/2077858444 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90700dfad0 con 0x7f9070007090 2024-03-08T06:01:01.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f183d7ea640 1 --2- 172.21.15.76:0/2451316532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18240051c0 0x7f18300d1370 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1838075290 tx=0x7f1838068730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f15bc9e3640 1 -- start start 2024-03-08T06:01:01.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f1659a3b640 1 -- 172.21.15.76:0/225287124 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16541644d0 con 0x7f16540b3830 2024-03-08T06:01:01.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f1659a3b640 1 -- 172.21.15.76:0/225287124 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1654164a90 con 0x7f16540b3830 2024-03-08T06:01:01.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f906e7fc640 1 -- 172.21.15.76:0/2077858444 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907400ee60 con 0x7f9070007090 2024-03-08T06:01:01.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f182dffb640 1 -- 172.21.15.76:0/2451316532 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183809a070 con 0x7f18240051c0 2024-03-08T06:01:01.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f183e7ec640 1 -- 172.21.15.76:0/2451316532 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1830051130 con 0x7f18240051c0 2024-03-08T06:01:01.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f182dffb640 1 -- 172.21.15.76:0/2451316532 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1838061ab0 con 0x7f18240051c0 2024-03-08T06:01:01.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f182dffb640 1 -- 172.21.15.76:0/2451316532 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18380626b0 con 0x7f18240051c0 2024-03-08T06:01:01.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f183e7ec640 1 -- 172.21.15.76:0/2451316532 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18300516f0 con 0x7f18240051c0 2024-03-08T06:01:01.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.054+0000 7fc49ca42640 1 -- 172.21.15.76:0/1266182071 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4900079b0 con 0x7fc4900361a0 2024-03-08T06:01:01.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.054+0000 7fc860ff9640 1 -- 172.21.15.76:0/1734842674 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858003830 con 0x7fc85c033d70 2024-03-08T06:01:01.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.054+0000 7f906e7fc640 1 -- 172.21.15.76:0/2077858444 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907400cb80 con 0x7f9070007090 2024-03-08T06:01:01.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.054+0000 7f15bc9e3640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15b0039160 0x7f15b0172ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.055+0000 7fc86a884640 1 -- 172.21.15.76:0/1734842674 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85c0500c0 con 0x7fc85c033d70 2024-03-08T06:01:01.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.053+0000 7f082effd640 1 -- 172.21.15.76:0/3553323628 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830017070 con 0x7f08400b27f0 2024-03-08T06:01:01.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.054+0000 7f082effd640 1 -- 172.21.15.76:0/3553323628 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0830005ce0 con 0x7f08400b27f0 2024-03-08T06:01:01.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.054+0000 7f082effd640 1 -- 172.21.15.76:0/3553323628 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08300072d0 con 0x7f08400b27f0 2024-03-08T06:01:01.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.054+0000 7f0844db6640 1 -- 172.21.15.76:0/3553323628 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08401607f0 con 0x7f08400b27f0 2024-03-08T06:01:01.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.054+0000 7f0844db6640 1 -- 172.21.15.76:0/3553323628 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0840160db0 con 0x7f08400b27f0 2024-03-08T06:01:01.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.055+0000 7fc483fff640 1 -- 172.21.15.76:0/1266182071 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc48404b070 con 0x7fc4900361a0 2024-03-08T06:01:01.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.055+0000 7f560b59e640 1 -- 172.21.15.76:0/3113036432 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5604051500 con 0x7f56040535a0 2024-03-08T06:01:01.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.055+0000 7f0844db6640 1 -- 172.21.15.76:0/3553323628 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0824005290 con 0x7f08400b27f0 2024-03-08T06:01:01.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.055+0000 7fc483fff640 1 --2- 172.21.15.76:0/1266182071 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc47c06b5a0 0x7fc47c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.056+0000 7fc483fff640 1 -- 172.21.15.76:0/1266182071 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc4840ac5f0 con 0x7fc4900361a0 2024-03-08T06:01:01.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.056+0000 7f55faffd640 1 -- 172.21.15.76:0/3113036432 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5600019020 con 0x7f56040535a0 2024-03-08T06:01:01.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.056+0000 7f182dffb640 1 -- 172.21.15.76:0/2451316532 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f183809d070 con 0x7f18240051c0 2024-03-08T06:01:01.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.056+0000 7f183e7ec640 1 -- 172.21.15.76:0/2451316532 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18300052b0 con 0x7f18240051c0 2024-03-08T06:01:01.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.056+0000 7f55faffd640 1 --2- 172.21.15.76:0/3113036432 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f55ec06b4d0 0x7f55ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.056+0000 7f182dffb640 1 --2- 172.21.15.76:0/2451316532 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f181c06b500 0x7f181c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7fc860ff9640 1 -- 172.21.15.76:0/1734842674 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc85801a070 con 0x7fc85c033d70 2024-03-08T06:01:01.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7f560a59c640 1 --2- 172.21.15.76:0/3113036432 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f55ec06b4d0 0x7f55ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7f1659a3b640 1 -- 172.21.15.76:0/225287124 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16540ccd00 con 0x7f16540b3830 2024-03-08T06:01:01.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.056+0000 7f15bc9e3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b00e7750 0x7f15b0173000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.056+0000 7f15bc9e3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15b00e7b30 0x7f15b0180d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.056+0000 7f15bc9e3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b0055640 con 0x7f15b00e7750 2024-03-08T06:01:01.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.056+0000 7f15bc9e3640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b004a170 con 0x7f15b00e7b30 2024-03-08T06:01:01.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.056+0000 7f15bc9e3640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b00557c0 con 0x7f15b0039160 2024-03-08T06:01:01.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7f15b659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b00e7750 0x7f15b0173000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7f15b659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b00e7750 0x7f15b0173000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51514/0 (socket says 172.21.15.76:51514) 2024-03-08T06:01:01.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7f15b659c640 1 -- 172.21.15.76:0/1187115749 learned_addr learned my addr 172.21.15.76:0/1187115749 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:01.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7f55faffd640 1 -- 172.21.15.76:0/3113036432 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f56000896d0 con 0x7f56040535a0 2024-03-08T06:01:01.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7fc860ff9640 1 --2- 172.21.15.76:0/1734842674 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc84806b5a0 0x7fc84806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7fc860ff9640 1 -- 172.21.15.76:0/1734842674 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc858089c30 con 0x7fc85c033d70 2024-03-08T06:01:01.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7f15b659c640 1 -- 172.21.15.76:0/1187115749 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15b0039160 msgr2=0x7f15b0172ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T06:01:01.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7f15b659c640 1 --2- 172.21.15.76:0/1187115749 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15b0039160 0x7f15b0172ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7f15b659c640 1 -- 172.21.15.76:0/1187115749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15b00e7b30 msgr2=0x7f15b0180d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T06:01:01.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7f560a59c640 1 --2- 172.21.15.76:0/3113036432 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f55ec06b4d0 0x7f55ec06d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f55fc036000 tx=0x7f55fc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7f15b659c640 1 --2- 172.21.15.76:0/1187115749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15b00e7b30 0x7f15b0180d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7f182dffb640 1 -- 172.21.15.76:0/2451316532 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1838104f10 con 0x7f18240051c0 2024-03-08T06:01:01.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.058+0000 7f183cfe9640 1 --2- 172.21.15.76:0/2451316532 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f181c06b500 0x7f181c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7f082effd640 1 -- 172.21.15.76:0/3553323628 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f083001a070 con 0x7f08400b27f0 2024-03-08T06:01:01.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7f082effd640 1 --2- 172.21.15.76:0/3553323628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f082006b5a0 0x7f082006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7f082effd640 1 -- 172.21.15.76:0/3553323628 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0830089760 con 0x7f08400b27f0 2024-03-08T06:01:01.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.058+0000 7fc496ffd640 1 --2- 172.21.15.76:0/1266182071 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc47c06b5a0 0x7fc47c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.058+0000 7f15b659c640 1 -- 172.21.15.76:0/1187115749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b00e1030 con 0x7f15b00e7750 2024-03-08T06:01:01.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.057+0000 7fc8637fe640 1 --2- 172.21.15.76:0/1734842674 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc84806b5a0 0x7fc84806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.056+0000 7f906e7fc640 1 -- 172.21.15.76:0/2077858444 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f907401a070 con 0x7f9070007090 2024-03-08T06:01:01.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.056+0000 7f907e872640 1 -- 172.21.15.76:0/2077858444 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9060005290 con 0x7f9070007090 2024-03-08T06:01:01.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.058+0000 7f15b659c640 1 --2- 172.21.15.76:0/1187115749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b00e7750 0x7f15b0173000 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00e840 tx=0x7f15ac00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.058+0000 7f083dd9b640 1 --2- 172.21.15.76:0/3553323628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f082006b5a0 0x7f082006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.059+0000 7f183cfe9640 1 --2- 172.21.15.76:0/2451316532 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f181c06b500 0x7f181c06d9c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f18300de930 tx=0x7f1834008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.058+0000 7f163bfff640 1 -- 172.21.15.76:0/225287124 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1644017070 con 0x7f16540b3830 2024-03-08T06:01:01.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.058+0000 7f906e7fc640 1 --2- 172.21.15.76:0/2077858444 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f905c06b5a0 0x7f905c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.058+0000 7f906e7fc640 1 -- 172.21.15.76:0/2077858444 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f907408a650 con 0x7f9070007090 2024-03-08T06:01:01.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.058+0000 7f907d06f640 1 --2- 172.21.15.76:0/2077858444 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f905c06b5a0 0x7f905c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.059+0000 7f907d06f640 1 --2- 172.21.15.76:0/2077858444 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f905c06b5a0 0x7f905c06da60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f907004a140 tx=0x7f9068008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.059+0000 7fc496ffd640 1 --2- 172.21.15.76:0/1266182071 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc47c06b5a0 0x7fc47c06da60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc49004c6b0 tx=0x7fc48c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.059+0000 7f163bfff640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f164006b5a0 0x7f164006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.059+0000 7fc8637fe640 1 --2- 172.21.15.76:0/1734842674 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc84806b5a0 0x7fc84806da60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc85c036c00 tx=0x7fc850008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.059+0000 7f163bfff640 1 -- 172.21.15.76:0/225287124 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1644089eb0 con 0x7f16540b3830 2024-03-08T06:01:01.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.059+0000 7f083dd9b640 1 --2- 172.21.15.76:0/3553323628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f082006b5a0 0x7f082006da60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0840161840 tx=0x7f0834008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.059+0000 7f16527fc640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f164006b5a0 0x7f164006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.059+0000 7f16527fc640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f164006b5a0 0x7f164006da60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f16541fb6a0 tx=0x7f1648008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.060+0000 7fc483fff640 1 -- 172.21.15.76:0/1266182071 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc48407aac0 con 0x7fc4900361a0 2024-03-08T06:01:01.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.060+0000 7fc860ff9640 1 -- 172.21.15.76:0/1734842674 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc85c0500c0 con 0x7fc85c033d70 2024-03-08T06:01:01.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.062+0000 7f159f7fe640 1 -- 172.21.15.76:0/1187115749 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac017070 con 0x7f15b00e7750 2024-03-08T06:01:01.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.062+0000 7f159f7fe640 1 -- 172.21.15.76:0/1187115749 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac00c940 con 0x7f15b00e7750 2024-03-08T06:01:01.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.062+0000 7f15bc9e3640 1 -- 172.21.15.76:0/1187115749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b00e1310 con 0x7f15b00e7750 2024-03-08T06:01:01.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.062+0000 7f15bc9e3640 1 -- 172.21.15.76:0/1187115749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b0181290 con 0x7f15b00e7750 2024-03-08T06:01:01.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.062+0000 7f182dffb640 1 -- 172.21.15.76:0/2451316532 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f18300d18b0 con 0x7f18240051c0 2024-03-08T06:01:01.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.062+0000 7f159f7fe640 1 -- 172.21.15.76:0/1187115749 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00cd10 con 0x7f15b00e7750 2024-03-08T06:01:01.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.064+0000 7f15bc9e3640 1 -- 172.21.15.76:0/1187115749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b0005710 con 0x7f15b00e7750 2024-03-08T06:01:01.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.065+0000 7f159f7fe640 1 -- 172.21.15.76:0/1187115749 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15ac01a070 con 0x7f15b00e7750 2024-03-08T06:01:01.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.065+0000 7f55faffd640 1 -- 172.21.15.76:0/3113036432 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5604051500 con 0x7f56040535a0 2024-03-08T06:01:01.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.065+0000 7f082effd640 1 -- 172.21.15.76:0/3553323628 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0830057ba0 con 0x7f08400b27f0 2024-03-08T06:01:01.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.065+0000 7f159f7fe640 1 --2- 172.21.15.76:0/1187115749 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f15a406b5a0 0x7f15a406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:01.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.065+0000 7f159f7fe640 1 -- 172.21.15.76:0/1187115749 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f15ac08a7b0 con 0x7f15b00e7750 2024-03-08T06:01:01.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.066+0000 7f15b6d9d640 1 --2- 172.21.15.76:0/1187115749 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f15a406b5a0 0x7f15a406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:01.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.066+0000 7f15b6d9d640 1 --2- 172.21.15.76:0/1187115749 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f15a406b5a0 0x7f15a406da60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f15b8050230 tx=0x7f15b80742d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:01.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.069+0000 7f163bfff640 1 -- 172.21.15.76:0/225287124 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16540ccd00 con 0x7f16540b3830 2024-03-08T06:01:01.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.070+0000 7f906e7fc640 1 -- 172.21.15.76:0/2077858444 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9074059170 con 0x7f9070007090 2024-03-08T06:01:01.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.074+0000 7f159f7fe640 1 -- 172.21.15.76:0/1187115749 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15ac0592d0 con 0x7f15b00e7750 2024-03-08T06:01:01.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.463+0000 7fc86a884640 1 -- 172.21.15.76:0/1734842674 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fc85c00ac50 con 0x7fc85c033d70 2024-03-08T06:01:01.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.464+0000 7fc860ff9640 1 -- 172.21.15.76:0/1734842674 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc85c00ac50 con 0x7fc85c033d70 2024-03-08T06:01:01.465 INFO:teuthology.orchestra.run.smithi076.stdout:55834574859 2024-03-08T06:01:01.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.467+0000 7fc49ca42640 1 -- 172.21.15.76:0/1266182071 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc4900eba00 con 0x7fc4900361a0 2024-03-08T06:01:01.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.467+0000 7fc84e7fc640 1 -- 172.21.15.76:0/1734842674 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc84806b5a0 msgr2=0x7fc84806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.467+0000 7fc84e7fc640 1 --2- 172.21.15.76:0/1734842674 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc84806b5a0 0x7fc84806da60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc85c036c00 tx=0x7fc850008040 comp rx=0 tx=0).stop 2024-03-08T06:01:01.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.467+0000 7fc84e7fc640 1 -- 172.21.15.76:0/1734842674 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85c033d70 msgr2=0x7fc85c0e7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.467+0000 7fc84e7fc640 1 --2- 172.21.15.76:0/1734842674 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85c033d70 0x7fc85c0e7020 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc85800c390 tx=0x7fc85800c850 comp rx=0 tx=0).stop 2024-03-08T06:01:01.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.468+0000 7fc84e7fc640 1 -- 172.21.15.76:0/1734842674 shutdown_connections 2024-03-08T06:01:01.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.468+0000 7fc84e7fc640 1 --2- 172.21.15.76:0/1734842674 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc84806b5a0 0x7fc84806da60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.468+0000 7fc84e7fc640 1 --2- 172.21.15.76:0/1734842674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc85c04f930 0x7fc85c0d3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.468+0000 7fc84e7fc640 1 --2- 172.21.15.76:0/1734842674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc85c037c50 0x7fc85c04e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.468+0000 7fc84e7fc640 1 --2- 172.21.15.76:0/1734842674 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85c033d70 0x7fc85c0e7020 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.468+0000 7fc84e7fc640 1 -- 172.21.15.76:0/1734842674 >> 172.21.15.76:0/1734842674 conn(0x7fc85c009fe0 msgr2=0x7fc85c0349e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:01.469 INFO:teuthology.orchestra.run.smithi076.stdout:55834574858 2024-03-08T06:01:01.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.468+0000 7fc483fff640 1 -- 172.21.15.76:0/1266182071 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc48407eb60 con 0x7fc4900361a0 2024-03-08T06:01:01.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.468+0000 7fc84e7fc640 1 -- 172.21.15.76:0/1734842674 shutdown_connections 2024-03-08T06:01:01.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.468+0000 7fc84e7fc640 1 -- 172.21.15.76:0/1734842674 wait complete. 2024-03-08T06:01:01.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.469+0000 7f183e7ec640 1 -- 172.21.15.76:0/2451316532 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1830056cb0 con 0x7f18240051c0 2024-03-08T06:01:01.471 INFO:teuthology.orchestra.run.smithi076.stdout:55834574858 2024-03-08T06:01:01.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.469+0000 7f182dffb640 1 -- 172.21.15.76:0/2451316532 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f18380d3360 con 0x7f18240051c0 2024-03-08T06:01:01.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.471+0000 7f0844db6640 1 -- 172.21.15.76:0/3553323628 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0824005580 con 0x7f08400b27f0 2024-03-08T06:01:01.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.471+0000 7f082effd640 1 -- 172.21.15.76:0/3553323628 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f083005bc40 con 0x7f08400b27f0 2024-03-08T06:01:01.472 INFO:teuthology.orchestra.run.smithi076.stdout:55834574858 2024-03-08T06:01:01.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.472+0000 7f560b59e640 1 -- 172.21.15.76:0/3113036432 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f560400dfb0 con 0x7f56040535a0 2024-03-08T06:01:01.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.472+0000 7fc49ca42640 1 -- 172.21.15.76:0/1266182071 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc47c06b5a0 msgr2=0x7fc47c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.472+0000 7fc49ca42640 1 --2- 172.21.15.76:0/1266182071 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc47c06b5a0 0x7fc47c06da60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc49004c6b0 tx=0x7fc48c009040 comp rx=0 tx=0).stop 2024-03-08T06:01:01.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.472+0000 7fc49ca42640 1 -- 172.21.15.76:0/1266182071 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4900361a0 msgr2=0x7fc4900dd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.472+0000 7fc49ca42640 1 --2- 172.21.15.76:0/1266182071 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4900361a0 0x7fc4900dd0f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc484033000 tx=0x7fc4840046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.472+0000 7f55faffd640 1 -- 172.21.15.76:0/3113036432 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5600058200 con 0x7f56040535a0 2024-03-08T06:01:01.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.472+0000 7fc49ca42640 1 -- 172.21.15.76:0/1266182071 shutdown_connections 2024-03-08T06:01:01.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.472+0000 7fc49ca42640 1 --2- 172.21.15.76:0/1266182071 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc47c06b5a0 0x7fc47c06da60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.472+0000 7fc49ca42640 1 --2- 172.21.15.76:0/1266182071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4900518d0 0x7fc490181250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.472+0000 7fc49ca42640 1 --2- 172.21.15.76:0/1266182071 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc4900529e0 0x7fc490051390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.472+0000 7fc49ca42640 1 --2- 172.21.15.76:0/1266182071 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4900361a0 0x7fc4900dd0f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.472+0000 7fc49ca42640 1 -- 172.21.15.76:0/1266182071 >> 172.21.15.76:0/1266182071 conn(0x7fc4900537a0 msgr2=0x7fc490036780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:01.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.472+0000 7fc49ca42640 1 -- 172.21.15.76:0/1266182071 shutdown_connections 2024-03-08T06:01:01.475 INFO:teuthology.orchestra.run.smithi076.stdout:55834574858 2024-03-08T06:01:01.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.472+0000 7fc49ca42640 1 -- 172.21.15.76:0/1266182071 wait complete. 2024-03-08T06:01:01.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.473+0000 7f183e7ec640 1 -- 172.21.15.76:0/2451316532 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f181c06b500 msgr2=0x7f181c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.473+0000 7f183e7ec640 1 --2- 172.21.15.76:0/2451316532 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f181c06b500 0x7f181c06d9c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f18300de930 tx=0x7f1834008040 comp rx=0 tx=0).stop 2024-03-08T06:01:01.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.473+0000 7f183e7ec640 1 -- 172.21.15.76:0/2451316532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18240051c0 msgr2=0x7f18300d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.473+0000 7f183e7ec640 1 --2- 172.21.15.76:0/2451316532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18240051c0 0x7f18300d1370 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1838075290 tx=0x7f1838068730 comp rx=0 tx=0).stop 2024-03-08T06:01:01.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.473+0000 7f183e7ec640 1 -- 172.21.15.76:0/2451316532 shutdown_connections 2024-03-08T06:01:01.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.473+0000 7f183e7ec640 1 --2- 172.21.15.76:0/2451316532 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f181c06b500 0x7f181c06d9c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.473+0000 7f183e7ec640 1 --2- 172.21.15.76:0/2451316532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18300521d0 0x7f183000b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.473+0000 7f183e7ec640 1 --2- 172.21.15.76:0/2451316532 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f18300de550 0x7f1830052cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.473+0000 7f183e7ec640 1 --2- 172.21.15.76:0/2451316532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18240051c0 0x7f18300d1370 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1838075290 tx=0x7f1838068730 comp rx=0 tx=0).stop 2024-03-08T06:01:01.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.473+0000 7f183e7ec640 1 -- 172.21.15.76:0/2451316532 >> 172.21.15.76:0/2451316532 conn(0x7f183003aa40 msgr2=0x7f18300e6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:01.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.474+0000 7f183e7ec640 1 -- 172.21.15.76:0/2451316532 shutdown_connections 2024-03-08T06:01:01.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.474+0000 7f183e7ec640 1 -- 172.21.15.76:0/2451316532 wait complete. 2024-03-08T06:01:01.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.475+0000 7f0844db6640 1 -- 172.21.15.76:0/3553323628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f082006b5a0 msgr2=0x7f082006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.475+0000 7f0844db6640 1 --2- 172.21.15.76:0/3553323628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f082006b5a0 0x7f082006da60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0840161840 tx=0x7f0834008030 comp rx=0 tx=0).stop 2024-03-08T06:01:01.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.475+0000 7f0844db6640 1 -- 172.21.15.76:0/3553323628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08400b27f0 msgr2=0x7f084015f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.475+0000 7f0844db6640 1 --2- 172.21.15.76:0/3553323628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08400b27f0 0x7f084015f970 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0830000f60 tx=0x7f083000c760 comp rx=0 tx=0).stop 2024-03-08T06:01:01.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.475+0000 7f0844db6640 1 -- 172.21.15.76:0/3553323628 shutdown_connections 2024-03-08T06:01:01.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.475+0000 7f0844db6640 1 --2- 172.21.15.76:0/3553323628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f082006b5a0 0x7f082006da60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.475+0000 7f0844db6640 1 --2- 172.21.15.76:0/3553323628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0840161fe0 0x7f084014fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.475+0000 7f0844db6640 1 --2- 172.21.15.76:0/3553323628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08400cfb90 0x7f084015feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.475+0000 7f0844db6640 1 --2- 172.21.15.76:0/3553323628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f08400b27f0 0x7f084015f970 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.475+0000 7f0844db6640 1 -- 172.21.15.76:0/3553323628 >> 172.21.15.76:0/3553323628 conn(0x7f084006ac80 msgr2=0x7f084015e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:01.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.475+0000 7f0844db6640 1 -- 172.21.15.76:0/3553323628 shutdown_connections 2024-03-08T06:01:01.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.475+0000 7f0844db6640 1 -- 172.21.15.76:0/3553323628 wait complete. 2024-03-08T06:01:01.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.476+0000 7f560b59e640 1 -- 172.21.15.76:0/3113036432 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f55ec06b4d0 msgr2=0x7f55ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.476+0000 7f560b59e640 1 --2- 172.21.15.76:0/3113036432 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f55ec06b4d0 0x7f55ec06d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f55fc036000 tx=0x7f55fc008030 comp rx=0 tx=0).stop 2024-03-08T06:01:01.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.476+0000 7f560b59e640 1 -- 172.21.15.76:0/3113036432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56040535a0 msgr2=0x7f56040cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.476+0000 7f560b59e640 1 --2- 172.21.15.76:0/3113036432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56040535a0 0x7f56040cdaf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f560000c3c0 tx=0x7f560000c880 comp rx=0 tx=0).stop 2024-03-08T06:01:01.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.476+0000 7f560b59e640 1 -- 172.21.15.76:0/3113036432 shutdown_connections 2024-03-08T06:01:01.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.476+0000 7f560b59e640 1 --2- 172.21.15.76:0/3113036432 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f55ec06b4d0 0x7f55ec06d990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.476+0000 7f560b59e640 1 --2- 172.21.15.76:0/3113036432 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5604058b10 0x7f5604039e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.476+0000 7f560b59e640 1 --2- 172.21.15.76:0/3113036432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56040535a0 0x7f56040cdaf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.476+0000 7f560b59e640 1 --2- 172.21.15.76:0/3113036432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5604006210 0x7f560404f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.476+0000 7f560b59e640 1 -- 172.21.15.76:0/3113036432 >> 172.21.15.76:0/3113036432 conn(0x7f560403aa40 msgr2=0x7f56040e7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:01.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.476+0000 7f560b59e640 1 -- 172.21.15.76:0/3113036432 shutdown_connections 2024-03-08T06:01:01.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.477+0000 7f560b59e640 1 -- 172.21.15.76:0/3113036432 wait complete. 2024-03-08T06:01:01.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.477+0000 7f1659a3b640 1 -- 172.21.15.76:0/225287124 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f16540cbe40 con 0x7f16540b3830 2024-03-08T06:01:01.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.478+0000 7f163bfff640 1 -- 172.21.15.76:0/225287124 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1644058380 con 0x7f16540b3830 2024-03-08T06:01:01.483 INFO:teuthology.orchestra.run.smithi076.stdout:55834574858 2024-03-08T06:01:01.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.481+0000 7f1659a3b640 1 -- 172.21.15.76:0/225287124 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f164006b5a0 msgr2=0x7f164006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.481+0000 7f1659a3b640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f164006b5a0 0x7f164006da60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f16541fb6a0 tx=0x7f1648008040 comp rx=0 tx=0).stop 2024-03-08T06:01:01.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.481+0000 7f1659a3b640 1 -- 172.21.15.76:0/225287124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16540b3830 msgr2=0x7f165414d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.481+0000 7f1659a3b640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16540b3830 0x7f165414d900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f164400b4a0 tx=0x7f164400b960 comp rx=0 tx=0).stop 2024-03-08T06:01:01.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.481+0000 7f1659a3b640 1 -- 172.21.15.76:0/225287124 shutdown_connections 2024-03-08T06:01:01.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.481+0000 7f1659a3b640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f164006b5a0 0x7f164006da60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.481+0000 7f1659a3b640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1654166350 0x7f1654160930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.481+0000 7f1659a3b640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1654161cd0 0x7f165415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.481+0000 7f1659a3b640 1 --2- 172.21.15.76:0/225287124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16540b3830 0x7f165414d900 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.481+0000 7f907e872640 1 -- 172.21.15.76:0/2077858444 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f9060005580 con 0x7f9070007090 2024-03-08T06:01:01.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.481+0000 7f1659a3b640 1 -- 172.21.15.76:0/225287124 >> 172.21.15.76:0/225287124 conn(0x7f16540670c0 msgr2=0x7f16540cdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:01.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.482+0000 7f15bc9e3640 1 -- 172.21.15.76:0/1187115749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f15b003cc50 con 0x7f15b00e7750 2024-03-08T06:01:01.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.482+0000 7f1659a3b640 1 -- 172.21.15.76:0/225287124 shutdown_connections 2024-03-08T06:01:01.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.482+0000 7f1659a3b640 1 -- 172.21.15.76:0/225287124 wait complete. 2024-03-08T06:01:01.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.482+0000 7f906e7fc640 1 -- 172.21.15.76:0/2077858444 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9074058a70 con 0x7f9070007090 2024-03-08T06:01:01.485 INFO:teuthology.orchestra.run.smithi076.stdout:55834574858 2024-03-08T06:01:01.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.483+0000 7f159f7fe640 1 -- 172.21.15.76:0/1187115749 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f15ac058bd0 con 0x7f15b00e7750 2024-03-08T06:01:01.486 INFO:teuthology.orchestra.run.smithi076.stdout:55834574858 2024-03-08T06:01:01.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f15bc9e3640 1 -- 172.21.15.76:0/1187115749 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f15a406b5a0 msgr2=0x7f15a406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f907e872640 1 -- 172.21.15.76:0/2077858444 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f905c06b5a0 msgr2=0x7f905c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f15bc9e3640 1 --2- 172.21.15.76:0/1187115749 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f15a406b5a0 0x7f15a406da60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f15b8050230 tx=0x7f15b80742d0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f907e872640 1 --2- 172.21.15.76:0/2077858444 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f905c06b5a0 0x7f905c06da60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f907004a140 tx=0x7f9068008040 comp rx=0 tx=0).stop 2024-03-08T06:01:01.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f907e872640 1 -- 172.21.15.76:0/2077858444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9070007090 msgr2=0x7f907004dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f907e872640 1 --2- 172.21.15.76:0/2077858444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9070007090 0x7f907004dfb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f907400e870 tx=0x7f907400ed30 comp rx=0 tx=0).stop 2024-03-08T06:01:01.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f907e872640 1 -- 172.21.15.76:0/2077858444 shutdown_connections 2024-03-08T06:01:01.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f907e872640 1 --2- 172.21.15.76:0/2077858444 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f905c06b5a0 0x7f905c06da60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f907e872640 1 --2- 172.21.15.76:0/2077858444 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9070055540 0x7f9070186d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f907e872640 1 --2- 172.21.15.76:0/2077858444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9070050bf0 0x7f907004efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f907e872640 1 --2- 172.21.15.76:0/2077858444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9070007090 0x7f907004dfb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f907e872640 1 -- 172.21.15.76:0/2077858444 >> 172.21.15.76:0/2077858444 conn(0x7f9070055bc0 msgr2=0x7f9070008980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:01.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f15bc9e3640 1 -- 172.21.15.76:0/1187115749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b00e7750 msgr2=0x7f15b0173000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:01.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f907e872640 1 -- 172.21.15.76:0/2077858444 shutdown_connections 2024-03-08T06:01:01.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f907e872640 1 -- 172.21.15.76:0/2077858444 wait complete. 2024-03-08T06:01:01.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f15bc9e3640 1 --2- 172.21.15.76:0/1187115749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b00e7750 0x7f15b0173000 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00e840 tx=0x7f15ac00ed00 comp rx=0 tx=0).stop 2024-03-08T06:01:01.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f15bc9e3640 1 -- 172.21.15.76:0/1187115749 shutdown_connections 2024-03-08T06:01:01.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f15bc9e3640 1 --2- 172.21.15.76:0/1187115749 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f15a406b5a0 0x7f15a406da60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f15bc9e3640 1 --2- 172.21.15.76:0/1187115749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15b00e7b30 0x7f15b0180d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f15bc9e3640 1 --2- 172.21.15.76:0/1187115749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b00e7750 0x7f15b0173000 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00e840 tx=0x7f15ac00ed00 comp rx=0 tx=0).stop 2024-03-08T06:01:01.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f15bc9e3640 1 --2- 172.21.15.76:0/1187115749 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15b0039160 0x7f15b0172ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:01.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f15bc9e3640 1 -- 172.21.15.76:0/1187115749 >> 172.21.15.76:0/1187115749 conn(0x7f15b0053750 msgr2=0x7f15b0053b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:01.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f15bc9e3640 1 -- 172.21.15.76:0/1187115749 shutdown_connections 2024-03-08T06:01:01.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:01.487+0000 7f15bc9e3640 1 -- 172.21.15.76:0/1187115749 wait complete. 2024-03-08T06:01:01.493 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.6 2024-03-08T06:01:01.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2024-03-08T06:01:01.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.4 2024-03-08T06:01:01.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.1 2024-03-08T06:01:01.501 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.2 2024-03-08T06:01:01.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-03-08T06:01:01.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.0 2024-03-08T06:01:01.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.7 2024-03-08T06:01:02.494 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-08T06:01:02.498 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-08T06:01:02.499 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-08T06:01:02.500 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-08T06:01:02.501 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-08T06:01:02.506 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-08T06:01:02.506 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-08T06:01:02.508 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-08T06:01:02.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.649+0000 7f7b4759e640 1 Processor -- start 2024-03-08T06:01:02.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.649+0000 7f7b4759e640 1 -- start start 2024-03-08T06:01:02.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.651+0000 7f7b4759e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7b400e75d0 0x7f7b40050530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.651+0000 7f7b4759e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b40006990 0x7f7b400534d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.651+0000 7f7b4759e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b40034a70 con 0x7f7b400345c0 2024-03-08T06:01:02.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.651+0000 7f7b4759e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b40050c10 con 0x7f7b400e75d0 2024-03-08T06:01:02.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.651+0000 7f7b4759e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b40050ef0 con 0x7f7b40006990 2024-03-08T06:01:02.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.652+0000 7f7b46d9d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b40006990 0x7f7b400534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.652+0000 7f7b46d9d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b40006990 0x7f7b400534d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37364/0 (socket says 172.21.15.76:37364) 2024-03-08T06:01:02.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.652+0000 7f7b46d9d640 1 -- 172.21.15.76:0/2533217120 learned_addr learned my addr 172.21.15.76:0/2533217120 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:02.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.652+0000 7f7b46d9d640 1 -- 172.21.15.76:0/2533217120 >> v2:172.21.15.81:3300/0 conn(0x7f7b400e75d0 msgr2=0x7f7b40050530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-08T06:01:02.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.652+0000 7f7b46d9d640 1 --2- 172.21.15.76:0/2533217120 >> v2:172.21.15.81:3300/0 conn(0x7f7b400e75d0 0x7f7b40050530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.652+0000 7f7b46d9d640 1 -- 172.21.15.76:0/2533217120 >> v1:172.21.15.76:6789/0 conn(0x7f7b400345c0 legacy=0x7f7b40006280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:02.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.653+0000 7f7b46d9d640 1 -- 172.21.15.76:0/2533217120 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b400511d0 con 0x7f7b40006990 2024-03-08T06:01:02.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.654+0000 7f7b46d9d640 1 --2- 172.21.15.76:0/2533217120 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b40006990 0x7f7b400534d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7b4802d3c0 tx=0x7f7b480a68c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ede72c46107e6466 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.655+0000 7f7b4559a640 1 -- 172.21.15.76:0/2533217120 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b480b0020 con 0x7f7b40006990 2024-03-08T06:01:02.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.655+0000 7f7b4559a640 1 -- 172.21.15.76:0/2533217120 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b480ac8e0 con 0x7f7b40006990 2024-03-08T06:01:02.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.656+0000 7f7b4759e640 1 -- 172.21.15.76:0/2533217120 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b40006990 msgr2=0x7f7b400534d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.656+0000 7f7b4759e640 1 --2- 172.21.15.76:0/2533217120 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b40006990 0x7f7b400534d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7b4802d3c0 tx=0x7f7b480a68c0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.656+0000 7f7b4759e640 1 -- 172.21.15.76:0/2533217120 shutdown_connections 2024-03-08T06:01:02.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.656+0000 7f7b4759e640 1 --2- 172.21.15.76:0/2533217120 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b40006990 0x7f7b400534d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.656+0000 7f7b4759e640 1 --2- 172.21.15.76:0/2533217120 >> v2:172.21.15.81:3300/0 conn(0x7f7b400e75d0 0x7f7b40050530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.656+0000 7f7b4759e640 1 -- 172.21.15.76:0/2533217120 >> 172.21.15.76:0/2533217120 conn(0x7f7b40055340 msgr2=0x7f7b40055750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:02.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.657+0000 7f7b4759e640 1 -- 172.21.15.76:0/2533217120 shutdown_connections 2024-03-08T06:01:02.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.657+0000 7f7b4759e640 1 -- 172.21.15.76:0/2533217120 wait complete. 2024-03-08T06:01:02.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.659+0000 7f7b4759e640 1 Processor -- start 2024-03-08T06:01:02.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.659+0000 7f7b4759e640 1 -- start start 2024-03-08T06:01:02.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.661+0000 7f7b4759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b40006990 0x7f7b400d2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.661+0000 7f7b4759e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b400345c0 0x7f7b400d3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.661+0000 7f7b4759e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b400e75d0 0x7f7b400d3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.661+0000 7f7b4759e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b40053a10 con 0x7f7b40006990 2024-03-08T06:01:02.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.661+0000 7f7b4759e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b40052fb0 con 0x7f7b400e75d0 2024-03-08T06:01:02.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.661+0000 7f7b4759e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b40053b90 con 0x7f7b400345c0 2024-03-08T06:01:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.661+0000 7f7b46d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b400e75d0 0x7f7b400d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.661+0000 7f7b46d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b400e75d0 0x7f7b400d3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.76:57386/0 (socket says 172.21.15.76:57386) 2024-03-08T06:01:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.661+0000 7f7b46d9d640 1 -- 172.21.15.76:0/1891413806 learned_addr learned my addr 172.21.15.76:0/1891413806 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.661+0000 7f7b45d9b640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b400345c0 0x7f7b400d3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.662+0000 7f7b46d9d640 1 -- 172.21.15.76:0/1891413806 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b400345c0 msgr2=0x7f7b400d3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.662+0000 7f7b46d9d640 1 --2- 172.21.15.76:0/1891413806 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b400345c0 0x7f7b400d3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.662+0000 7f7b46d9d640 1 -- 172.21.15.76:0/1891413806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b40006990 msgr2=0x7f7b400d2830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T06:01:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.662+0000 7f7b4659c640 1 --2- 172.21.15.76:0/1891413806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b40006990 0x7f7b400d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.663+0000 7f7b46d9d640 1 --2- 172.21.15.76:0/1891413806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b40006990 0x7f7b400d2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.663+0000 7f7b46d9d640 1 -- 172.21.15.76:0/1891413806 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b400d6d10 con 0x7f7b400e75d0 2024-03-08T06:01:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.663+0000 7f7b4659c640 1 --2- 172.21.15.76:0/1891413806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b40006990 0x7f7b400d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.663+0000 7f7b46d9d640 1 --2- 172.21.15.76:0/1891413806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b400e75d0 0x7f7b400d3670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7b48000b60 tx=0x7f7b480689d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.664+0000 7f7b36ffd640 1 -- 172.21.15.76:0/1891413806 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b480b0030 con 0x7f7b400e75d0 2024-03-08T06:01:02.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.664+0000 7f7b4759e640 1 -- 172.21.15.76:0/1891413806 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b400d3f80 con 0x7f7b400e75d0 2024-03-08T06:01:02.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.664+0000 7f7b4759e640 1 -- 172.21.15.76:0/1891413806 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b400d1d70 con 0x7f7b400e75d0 2024-03-08T06:01:02.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.664+0000 7f7b36ffd640 1 -- 172.21.15.76:0/1891413806 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b480bf610 con 0x7f7b400e75d0 2024-03-08T06:01:02.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.665+0000 7f7b36ffd640 1 -- 172.21.15.76:0/1891413806 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b480bf9c0 con 0x7f7b400e75d0 2024-03-08T06:01:02.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.667+0000 7f7b4759e640 1 -- 172.21.15.76:0/1891413806 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b2c005290 con 0x7f7b400e75d0 2024-03-08T06:01:02.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.668+0000 7f7b36ffd640 1 -- 172.21.15.76:0/1891413806 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b480c6060 con 0x7f7b400e75d0 2024-03-08T06:01:02.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.668+0000 7f7b36ffd640 1 --2- 172.21.15.76:0/1891413806 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7b2806b5a0 0x7f7b2806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.669+0000 7f7b4659c640 1 --2- 172.21.15.76:0/1891413806 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7b2806b5a0 0x7f7b2806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.669+0000 7f7b36ffd640 1 -- 172.21.15.76:0/1891413806 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7b48127590 con 0x7f7b400e75d0 2024-03-08T06:01:02.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.670+0000 7f7b4659c640 1 --2- 172.21.15.76:0/1891413806 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7b2806b5a0 0x7f7b2806da60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7b40048ff0 tx=0x7f7b3c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.673+0000 7f7b36ffd640 1 -- 172.21.15.76:0/1891413806 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b480ba070 con 0x7f7b400e75d0 2024-03-08T06:01:02.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.704+0000 7fc6680ed640 1 Processor -- start 2024-03-08T06:01:02.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.705+0000 7fc6680ed640 1 -- start start 2024-03-08T06:01:02.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.706+0000 7fc6680ed640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc6601622f0 0x7fc66006bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.706+0000 7fc6680ed640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc6600cecf0 0x7fc6600cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.706+0000 7fc6680ed640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6601636c0 con 0x7fc660166700 2024-03-08T06:01:02.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.706+0000 7fc6680ed640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc660162b40 con 0x7fc6600cecf0 2024-03-08T06:01:02.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.706+0000 7fc6680ed640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc660162e20 con 0x7fc6601622f0 2024-03-08T06:01:02.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.706+0000 7fc665e89640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc6601622f0 0x7fc66006bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.706+0000 7fc665e89640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc6601622f0 0x7fc66006bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37388/0 (socket says 172.21.15.76:37388) 2024-03-08T06:01:02.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.706+0000 7fc665e89640 1 -- 172.21.15.76:0/4128723619 learned_addr learned my addr 172.21.15.76:0/4128723619 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:02.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.706+0000 7fc66668a640 1 --2- 172.21.15.76:0/4128723619 >> v2:172.21.15.76:3300/0 conn(0x7fc6600cecf0 0x7fc6600cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.707+0000 7fc665e89640 1 -- 172.21.15.76:0/4128723619 >> v2:172.21.15.76:3300/0 conn(0x7fc6600cecf0 msgr2=0x7fc6600cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:02.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.707+0000 7fc665e89640 1 --2- 172.21.15.76:0/4128723619 >> v2:172.21.15.76:3300/0 conn(0x7fc6600cecf0 0x7fc6600cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.707+0000 7fc664e87640 1 -- 172.21.15.76:0/4128723619 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625433461 0 0) 0x7fc6601636c0 con 0x7fc660166700 2024-03-08T06:01:02.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.707+0000 7fc665e89640 1 -- 172.21.15.76:0/4128723619 >> v1:172.21.15.81:6789/0 conn(0x7fc660166700 legacy=0x7fc6600ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.707+0000 7fc665e89640 1 -- 172.21.15.76:0/4128723619 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc660163100 con 0x7fc6601622f0 2024-03-08T06:01:02.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.708+0000 7f030e0d0640 1 Processor -- start 2024-03-08T06:01:02.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.708+0000 7f3b70b80640 1 Processor -- start 2024-03-08T06:01:02.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.708+0000 7f3b70b80640 1 -- start start 2024-03-08T06:01:02.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.708+0000 7f030e0d0640 1 -- start start 2024-03-08T06:01:02.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.709+0000 7fd8572c3640 1 Processor -- start 2024-03-08T06:01:02.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.709+0000 7fd8572c3640 1 -- start start 2024-03-08T06:01:02.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.709+0000 7f030e0d0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f03080b39e0 0x7f030806ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.709+0000 7f030e0d0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f030806f130 0x7f0308071800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.709+0000 7fd677fff640 1 Processor -- start 2024-03-08T06:01:02.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.709+0000 7f030e0d0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f030806f720 con 0x7f0308067c50 2024-03-08T06:01:02.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.709+0000 7f030e0d0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03080b2c20 con 0x7f030806f130 2024-03-08T06:01:02.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.709+0000 7f030e0d0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03080b2f00 con 0x7f03080b39e0 2024-03-08T06:01:02.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7f3b70b80640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b64055d00 0x7f3b6404fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7fc665e89640 1 --2- 172.21.15.76:0/4128723619 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc6601622f0 0x7fc66006bfa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc658009e50 tx=0x7fc65802bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bc05a7ad3e86cdb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7f0306ffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f03080b39e0 0x7f030806ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7f3b70b80640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b64056140 con 0x7f3b6403a090 2024-03-08T06:01:02.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7f3b70b80640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b64006c70 con 0x7f3b640df7f0 2024-03-08T06:01:02.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7f3b70b80640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b64006f50 con 0x7f3b64055d00 2024-03-08T06:01:02.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7f0306ffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f03080b39e0 0x7f030806ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.76:57400/0 (socket says 172.21.15.76:57400) 2024-03-08T06:01:02.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7f0306ffd640 1 -- 172.21.15.76:0/84195293 learned_addr learned my addr 172.21.15.76:0/84195293 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:02.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7fc664e87640 1 -- 172.21.15.76:0/4128723619 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65802cb90 con 0x7fc6601622f0 2024-03-08T06:01:02.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7fc664e87640 1 -- 172.21.15.76:0/4128723619 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65802cd30 con 0x7fc6601622f0 2024-03-08T06:01:02.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.709+0000 7fd677fff640 1 -- start start 2024-03-08T06:01:02.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fd677fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd678159d10 0x7fd6780cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fd677fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd6780b2d30 0x7fd67815e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fd677fff640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67815a0f0 con 0x7fd6780712e0 2024-03-08T06:01:02.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fd677fff640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67815e770 con 0x7fd678159d10 2024-03-08T06:01:02.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fd677fff640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67806dc30 con 0x7fd6780b2d30 2024-03-08T06:01:02.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7f3b6affd640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b64055d00 0x7f3b6404fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7f3b6affd640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b64055d00 0x7f3b6404fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37400/0 (socket says 172.21.15.76:37400) 2024-03-08T06:01:02.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7f3b6affd640 1 -- 172.21.15.76:0/4049229041 learned_addr learned my addr 172.21.15.76:0/4049229041 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:02.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7f3b6affd640 1 -- 172.21.15.76:0/4049229041 >> v1:172.21.15.81:6789/0 conn(0x7f3b640df7f0 legacy=0x7f3b64036690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:02.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7f3b6affd640 1 -- 172.21.15.76:0/4049229041 >> v1:172.21.15.76:6789/0 conn(0x7f3b6403a090 legacy=0x7f3b64007a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:02.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7f3b6affd640 1 -- 172.21.15.76:0/4049229041 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b64007230 con 0x7f3b64055d00 2024-03-08T06:01:02.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7fd8572c3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd8500b2710 0x7fd85006f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7fd8572c3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8500d0140 con 0x7fd85006f600 2024-03-08T06:01:02.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7fd8572c3640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85015c3a0 con 0x7fd8500cfd60 2024-03-08T06:01:02.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7fd8572c3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85015c680 con 0x7fd8500b2710 2024-03-08T06:01:02.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7fd85505f640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd8500cfd60 0x7fd85006d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:40178/0 (socket says 172.21.15.76:40178) 2024-03-08T06:01:02.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fd85505f640 1 -- 172.21.15.76:0/1514618089 learned_addr learned my addr 172.21.15.76:0/1514618089 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:02.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fd85485e640 1 --2- 172.21.15.76:0/1514618089 >> v2:172.21.15.76:3300/0 conn(0x7fd8500b2710 0x7fd85006f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fd847fff640 1 -- 172.21.15.76:0/1514618089 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590789868 0 0) 0x7fd85015c3a0 con 0x7fd8500cfd60 2024-03-08T06:01:02.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fd847fff640 1 -- 172.21.15.76:0/1514618089 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd83c003520 con 0x7fd8500cfd60 2024-03-08T06:01:02.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7f0306ffd640 1 -- 172.21.15.76:0/84195293 >> v2:172.21.15.76:3300/0 conn(0x7f030806f130 msgr2=0x7f0308071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:02.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7f0306ffd640 1 --2- 172.21.15.76:0/84195293 >> v2:172.21.15.76:3300/0 conn(0x7f030806f130 0x7f0308071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7f0306ffd640 1 -- 172.21.15.76:0/84195293 >> v1:172.21.15.81:6789/0 conn(0x7f0308067c50 legacy=0x7f03081435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.710+0000 7f0306ffd640 1 -- 172.21.15.76:0/84195293 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03080b31e0 con 0x7f03080b39e0 2024-03-08T06:01:02.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fc6680ed640 1 -- 172.21.15.76:0/4128723619 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc6601622f0 msgr2=0x7fc66006bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fc6680ed640 1 --2- 172.21.15.76:0/4128723619 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc6601622f0 0x7fc66006bfa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc658009e50 tx=0x7fc65802bff0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fd6777fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd6780712e0 0x7fd6780cd210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:40186/0 (socket says 172.21.15.76:40186) 2024-03-08T06:01:02.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fc6680ed640 1 -- 172.21.15.76:0/4128723619 shutdown_connections 2024-03-08T06:01:02.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fc6680ed640 1 --2- 172.21.15.76:0/4128723619 >> v2:172.21.15.76:3300/0 conn(0x7fc6600cecf0 0x7fc6600cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fc6680ed640 1 --2- 172.21.15.76:0/4128723619 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc6601622f0 0x7fc66006bfa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fc6680ed640 1 -- 172.21.15.76:0/4128723619 >> 172.21.15.76:0/4128723619 conn(0x7fc66006ae60 msgr2=0x7fc66006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:02.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7f0306ffd640 1 --2- 172.21.15.76:0/84195293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03080b39e0 0x7f030806ebf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f03080b4730 tx=0x7f02f802b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0bc52e454de5a90 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fd85485e640 1 -- 172.21.15.76:0/1514618089 >> v1:172.21.15.81:6789/0 conn(0x7fd8500cfd60 legacy=0x7fd85006d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fd847fff640 1 -- 172.21.15.76:0/1514618089 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3152396898 0 0) 0x7fd83c003520 con 0x7fd8500cfd60 2024-03-08T06:01:02.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fd85485e640 1 -- 172.21.15.76:0/1514618089 >> v1:172.21.15.76:6789/0 conn(0x7fd85006f600 legacy=0x7fd850071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:02.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fd85485e640 1 -- 172.21.15.76:0/1514618089 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85015c960 con 0x7fd8500b2710 2024-03-08T06:01:02.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fd6777fe640 1 -- 172.21.15.76:0/60667962 learned_addr learned my addr 172.21.15.76:0/60667962 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:02.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.711+0000 7fd6767fc640 1 --2- 172.21.15.76:0/60667962 >> v2:172.21.15.81:3300/0 conn(0x7fd6780b2d30 0x7fd67815e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7f03067fc640 1 -- 172.21.15.76:0/84195293 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8034070 con 0x7f03080b39e0 2024-03-08T06:01:02.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7fd676ffd640 1 --2- 172.21.15.76:0/60667962 >> v2:172.21.15.76:3300/0 conn(0x7fd678159d10 0x7fd6780cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7fc85d380640 1 Processor -- start 2024-03-08T06:01:02.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7f03067fc640 1 -- 172.21.15.76:0/84195293 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f8030880 con 0x7f03080b39e0 2024-03-08T06:01:02.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7f03067fc640 1 -- 172.21.15.76:0/84195293 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8034070 con 0x7f03080b39e0 2024-03-08T06:01:02.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7fd675ffb640 1 -- 172.21.15.76:0/60667962 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75036056 0 0) 0x7fd67815a0f0 con 0x7fd6780712e0 2024-03-08T06:01:02.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7f3b6affd640 1 --2- 172.21.15.76:0/4049229041 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b64055d00 0x7f3b6404fa30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c02d3c0 tx=0x7f3b6c0a68b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4777d8467c51e7c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7fc6680ed640 1 -- 172.21.15.76:0/4128723619 shutdown_connections 2024-03-08T06:01:02.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7fd675ffb640 1 -- 172.21.15.76:0/60667962 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd660003520 con 0x7fd6780712e0 2024-03-08T06:01:02.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7fd676ffd640 1 -- 172.21.15.76:0/60667962 >> v2:172.21.15.81:3300/0 conn(0x7fd6780b2d30 msgr2=0x7fd67815e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:02.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7fd676ffd640 1 --2- 172.21.15.76:0/60667962 >> v2:172.21.15.81:3300/0 conn(0x7fd6780b2d30 0x7fd67815e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7fd676ffd640 1 -- 172.21.15.76:0/60667962 >> v1:172.21.15.81:6789/0 conn(0x7fd6780712e0 legacy=0x7fd6780cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7fd676ffd640 1 -- 172.21.15.76:0/60667962 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67806d320 con 0x7fd678159d10 2024-03-08T06:01:02.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7f3b6a7fc640 1 -- 172.21.15.76:0/4049229041 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0b0020 con 0x7f3b64055d00 2024-03-08T06:01:02.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7f030e0d0640 1 -- 172.21.15.76:0/84195293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03080b39e0 msgr2=0x7f030806ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7f030e0d0640 1 --2- 172.21.15.76:0/84195293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03080b39e0 0x7f030806ebf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f03080b4730 tx=0x7f02f802b9c0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7f030e0d0640 1 -- 172.21.15.76:0/84195293 shutdown_connections 2024-03-08T06:01:02.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7f030e0d0640 1 --2- 172.21.15.76:0/84195293 >> v2:172.21.15.76:3300/0 conn(0x7f030806f130 0x7f0308071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.713+0000 7f030e0d0640 1 --2- 172.21.15.76:0/84195293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03080b39e0 0x7f030806ebf0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f03080b4730 tx=0x7f02f802b9c0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.713+0000 7f030e0d0640 1 -- 172.21.15.76:0/84195293 >> 172.21.15.76:0/84195293 conn(0x7f03080b1b00 msgr2=0x7f03080b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:02.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.713+0000 7f030e0d0640 1 -- 172.21.15.76:0/84195293 shutdown_connections 2024-03-08T06:01:02.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.713+0000 7f030e0d0640 1 -- 172.21.15.76:0/84195293 wait complete. 2024-03-08T06:01:02.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7fc6680ed640 1 -- 172.21.15.76:0/4128723619 wait complete. 2024-03-08T06:01:02.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.712+0000 7fc85d380640 1 -- start start 2024-03-08T06:01:02.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.713+0000 7fc85d380640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc850007a50 0x7fc850058a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.713+0000 7fc85d380640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8500e35f0 con 0x7fc8500585f0 2024-03-08T06:01:02.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.713+0000 7fc85d380640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc850006580 con 0x7fc8500e3210 2024-03-08T06:01:02.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.713+0000 7fc85d380640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc850058f90 con 0x7fc850007a50 2024-03-08T06:01:02.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.713+0000 7fc856ffd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc8500e3210 0x7fc85003c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:40196/0 (socket says 172.21.15.76:40196) 2024-03-08T06:01:02.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.713+0000 7fc856ffd640 1 -- 172.21.15.76:0/3111112651 learned_addr learned my addr 172.21.15.76:0/3111112651 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:02.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.713+0000 7fd676ffd640 1 --2- 172.21.15.76:0/60667962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd678159d10 0x7fd6780cec60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd6780cd1c0 tx=0x7fd66c02cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20954cf6518ac5d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.714+0000 7fd675ffb640 1 -- 172.21.15.76:0/60667962 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c03a070 con 0x7fd678159d10 2024-03-08T06:01:02.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.714+0000 7fc857fff640 1 --2- 172.21.15.76:0/3111112651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc850007a50 0x7fc850058a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.714+0000 7fd675ffb640 1 -- 172.21.15.76:0/60667962 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66c02d9b0 con 0x7fd678159d10 2024-03-08T06:01:02.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.714+0000 7fc6680ed640 1 Processor -- start 2024-03-08T06:01:02.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.713+0000 7f3b6a7fc640 1 -- 172.21.15.76:0/4049229041 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c0ac8d0 con 0x7f3b64055d00 2024-03-08T06:01:02.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.713+0000 7f3b6a7fc640 1 -- 172.21.15.76:0/4049229041 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0acd40 con 0x7f3b64055d00 2024-03-08T06:01:02.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.714+0000 7f3b70b80640 1 -- 172.21.15.76:0/4049229041 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b64055d00 msgr2=0x7f3b6404fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.714+0000 7f3b70b80640 1 --2- 172.21.15.76:0/4049229041 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b64055d00 0x7f3b6404fa30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c02d3c0 tx=0x7f3b6c0a68b0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.714+0000 7f3b70b80640 1 -- 172.21.15.76:0/4049229041 shutdown_connections 2024-03-08T06:01:02.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.714+0000 7f3b70b80640 1 --2- 172.21.15.76:0/4049229041 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b64055d00 0x7f3b6404fa30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.714+0000 7f3b70b80640 1 -- 172.21.15.76:0/4049229041 >> 172.21.15.76:0/4049229041 conn(0x7f3b6401e6b0 msgr2=0x7f3b640553e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:02.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.714+0000 7fc855ffb640 1 -- 172.21.15.76:0/3111112651 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3798825542 0 0) 0x7fc850006580 con 0x7fc8500e3210 2024-03-08T06:01:02.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.714+0000 7fc855ffb640 1 -- 172.21.15.76:0/3111112651 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc840003520 con 0x7fc8500e3210 2024-03-08T06:01:02.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.714+0000 7f030e0d0640 1 Processor -- start 2024-03-08T06:01:02.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.714+0000 7fc855ffb640 1 -- 172.21.15.76:0/3111112651 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766558672 0 0) 0x7fc8500e35f0 con 0x7fc8500585f0 2024-03-08T06:01:02.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.714+0000 7fc855ffb640 1 -- 172.21.15.76:0/3111112651 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc850006580 con 0x7fc8500585f0 2024-03-08T06:01:02.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.714+0000 7f030e0d0640 1 -- start start 2024-03-08T06:01:02.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.714+0000 7fc6680ed640 1 -- start start 2024-03-08T06:01:02.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc855ffb640 1 -- 172.21.15.76:0/3111112651 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2438566863 0 0) 0x7fc840003520 con 0x7fc8500e3210 2024-03-08T06:01:02.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fd677fff640 1 -- 172.21.15.76:0/60667962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd678159d10 msgr2=0x7fd6780cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fd677fff640 1 --2- 172.21.15.76:0/60667962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd678159d10 0x7fd6780cec60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd6780cd1c0 tx=0x7fd66c02cc00 comp rx=0 tx=0).stop 2024-03-08T06:01:02.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc855ffb640 1 -- 172.21.15.76:0/3111112651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc850007a50 msgr2=0x7fc850058a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:02.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc855ffb640 1 --2- 172.21.15.76:0/3111112651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc850007a50 0x7fc850058a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc855ffb640 1 -- 172.21.15.76:0/3111112651 >> v1:172.21.15.76:6789/0 conn(0x7fc8500585f0 legacy=0x7fc850007340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fd677fff640 1 -- 172.21.15.76:0/60667962 shutdown_connections 2024-03-08T06:01:02.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fd677fff640 1 --2- 172.21.15.76:0/60667962 >> v2:172.21.15.81:3300/0 conn(0x7fd6780b2d30 0x7fd67815e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fd677fff640 1 --2- 172.21.15.76:0/60667962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd678159d10 0x7fd6780cec60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fd677fff640 1 -- 172.21.15.76:0/60667962 >> 172.21.15.76:0/60667962 conn(0x7fd6780cf640 msgr2=0x7fd6780cfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:02.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc855ffb640 1 -- 172.21.15.76:0/3111112651 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc850006860 con 0x7fc8500e3210 2024-03-08T06:01:02.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc855ffb640 1 -- 172.21.15.76:0/3111112651 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fc85804f810 con 0x7fc8500e3210 2024-03-08T06:01:02.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc855ffb640 1 -- 172.21.15.76:0/3111112651 >> v1:172.21.15.81:6789/0 conn(0x7fc8500e3210 legacy=0x7fc85003c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc855ffb640 1 --2- 172.21.15.76:0/3111112651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc840003b70 0x7fc840003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc855ffb640 1 --2- 172.21.15.76:0/3111112651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc840004800 0x7fc840004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc855ffb640 1 --2- 172.21.15.76:0/3111112651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc8400051c0 0x7fc84000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc855ffb640 1 -- 172.21.15.76:0/3111112651 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84000bf00 con 0x7fc840003b70 2024-03-08T06:01:02.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc855ffb640 1 -- 172.21.15.76:0/3111112651 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84000c080 con 0x7fc840004800 2024-03-08T06:01:02.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc855ffb640 1 -- 172.21.15.76:0/3111112651 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84000c230 con 0x7fc8400051c0 2024-03-08T06:01:02.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc6680ed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6600cecf0 0x7fc6600b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fce823e5640 1 Processor -- start 2024-03-08T06:01:02.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc6680ed640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6601622f0 0x7fc6600b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc6680ed640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc660166700 0x7fc6601fbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc6680ed640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6600c4e80 con 0x7fc6600cecf0 2024-03-08T06:01:02.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc6680ed640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc660166b40 con 0x7fc6601622f0 2024-03-08T06:01:02.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc6680ed640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6600c5000 con 0x7fc660166700 2024-03-08T06:01:02.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fd85485e640 1 --2- 172.21.15.76:0/1514618089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8500b2710 0x7fd85006f0c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd848009d90 tx=0x7fd84802bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3f258c107a5c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc665e89640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6600cecf0 0x7fc6600b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc665e89640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6600cecf0 0x7fc6600b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51576/0 (socket says 172.21.15.76:51576) 2024-03-08T06:01:02.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc665e89640 1 -- 172.21.15.76:0/2313011834 learned_addr learned my addr 172.21.15.76:0/2313011834 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:02.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc665688640 1 --2- 172.21.15.76:0/2313011834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6601622f0 0x7fc6600b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc66668a640 1 --2- 172.21.15.76:0/2313011834 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc660166700 0x7fc6601fbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fc856ffd640 1 --2- 172.21.15.76:0/3111112651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc840003b70 0x7fc840003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7fce823e5640 1 -- start start 2024-03-08T06:01:02.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7f030e0d0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0308067c50 0x7f030815b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc665688640 1 -- 172.21.15.76:0/2313011834 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc660166700 msgr2=0x7fc6601fbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc665688640 1 --2- 172.21.15.76:0/2313011834 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc660166700 0x7fc6601fbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc665688640 1 -- 172.21.15.76:0/2313011834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6600cecf0 msgr2=0x7fc6600b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc665688640 1 --2- 172.21.15.76:0/2313011834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6600cecf0 0x7fc6600b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc665688640 1 -- 172.21.15.76:0/2313011834 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6600c7260 con 0x7fc6601622f0 2024-03-08T06:01:02.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7f030e0d0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f030806f130 0x7f0308155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc857fff640 1 --2- 172.21.15.76:0/3111112651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc8400051c0 0x7fc84000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc665688640 1 --2- 172.21.15.76:0/2313011834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6601622f0 0x7fc6600b24a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc65000c3e0 tx=0x7fc65000c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7f030e0d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f030813eb20 0x7f03080b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7f030e0d0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0308071d40 con 0x7f030813eb20 2024-03-08T06:01:02.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7f030e0d0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030806e900 con 0x7f0308067c50 2024-03-08T06:01:02.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7f030e0d0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0308071ec0 con 0x7f030806f130 2024-03-08T06:01:02.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fd847fff640 1 -- 172.21.15.76:0/1514618089 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848039070 con 0x7fd8500b2710 2024-03-08T06:01:02.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fd847fff640 1 -- 172.21.15.76:0/1514618089 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84802c820 con 0x7fd8500b2710 2024-03-08T06:01:02.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fd847fff640 1 -- 172.21.15.76:0/1514618089 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840001670 con 0x7fd8500b2710 2024-03-08T06:01:02.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7f3b70b80640 1 -- 172.21.15.76:0/4049229041 shutdown_connections 2024-03-08T06:01:02.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc6567fc640 1 -- 172.21.15.76:0/2313011834 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc650017070 con 0x7fc6601622f0 2024-03-08T06:01:02.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.715+0000 7f3b70b80640 1 -- 172.21.15.76:0/4049229041 wait complete. 2024-03-08T06:01:02.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc6567fc640 1 -- 172.21.15.76:0/2313011834 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6500035f0 con 0x7fc6601622f0 2024-03-08T06:01:02.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc6680ed640 1 -- 172.21.15.76:0/2313011834 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6600c7540 con 0x7fc6601622f0 2024-03-08T06:01:02.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc6680ed640 1 -- 172.21.15.76:0/2313011834 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6600c7b00 con 0x7fc6601622f0 2024-03-08T06:01:02.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc6567fc640 1 -- 172.21.15.76:0/2313011834 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6500039e0 con 0x7fc6601622f0 2024-03-08T06:01:02.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fd677fff640 1 -- 172.21.15.76:0/60667962 shutdown_connections 2024-03-08T06:01:02.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc856ffd640 1 -- 172.21.15.76:0/3111112651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc8400051c0 msgr2=0x7fc84000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:02.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7f0307fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f030813eb20 0x7f03080b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7f0307fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f030813eb20 0x7f03080b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51588/0 (socket says 172.21.15.76:51588) 2024-03-08T06:01:02.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7f0307fff640 1 -- 172.21.15.76:0/3686249357 learned_addr learned my addr 172.21.15.76:0/3686249357 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:02.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fd677fff640 1 -- 172.21.15.76:0/60667962 wait complete. 2024-03-08T06:01:02.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc856ffd640 1 --2- 172.21.15.76:0/3111112651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc8400051c0 0x7fc84000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc856ffd640 1 -- 172.21.15.76:0/3111112651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc840004800 msgr2=0x7fc840004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-08T06:01:02.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc856ffd640 1 --2- 172.21.15.76:0/3111112651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc840004800 0x7fc840004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.716+0000 7fc856ffd640 1 -- 172.21.15.76:0/3111112651 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc850006860 con 0x7fc840003b70 2024-03-08T06:01:02.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7fd8572c3640 1 -- 172.21.15.76:0/1514618089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8500b2710 msgr2=0x7fd85006f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7fd8572c3640 1 --2- 172.21.15.76:0/1514618089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8500b2710 0x7fd85006f0c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd848009d90 tx=0x7fd84802bcf0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7fd8572c3640 1 -- 172.21.15.76:0/1514618089 shutdown_connections 2024-03-08T06:01:02.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7fce823e5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fce740df0f0 0x7fce740e00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7fce823e5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce740df4d0 con 0x7fce74053090 2024-03-08T06:01:02.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7fce823e5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce740535e0 con 0x7fce74050170 2024-03-08T06:01:02.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7fce823e5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce740538c0 con 0x7fce740df0f0 2024-03-08T06:01:02.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7fce813e3640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fce740df0f0 0x7fce740e00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7fce813e3640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fce740df0f0 0x7fce740e00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.76:57432/0 (socket says 172.21.15.76:57432) 2024-03-08T06:01:02.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7fce813e3640 1 -- 172.21.15.76:0/4180965814 learned_addr learned my addr 172.21.15.76:0/4180965814 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:02.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7fd8572c3640 1 --2- 172.21.15.76:0/1514618089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8500b2710 0x7fd85006f0c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7fd8572c3640 1 -- 172.21.15.76:0/1514618089 >> 172.21.15.76:0/1514618089 conn(0x7fd8500b1410 msgr2=0x7fd8500b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:02.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7fce813e3640 1 -- 172.21.15.76:0/4180965814 >> v1:172.21.15.81:6789/0 conn(0x7fce74050170 legacy=0x7fce74005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:02.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7fce813e3640 1 -- 172.21.15.76:0/4180965814 >> v1:172.21.15.76:6789/0 conn(0x7fce74053090 legacy=0x7fce7404fa60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-08T06:01:02.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7fce813e3640 1 -- 172.21.15.76:0/4180965814 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce74053ba0 con 0x7fce740df0f0 2024-03-08T06:01:02.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7fd8572c3640 1 -- 172.21.15.76:0/1514618089 shutdown_connections 2024-03-08T06:01:02.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.718+0000 7fc856ffd640 1 --2- 172.21.15.76:0/3111112651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc840003b70 0x7fc840003f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc858075290 tx=0x7fc840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdf5edb641d167a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.718+0000 7fd8572c3640 1 -- 172.21.15.76:0/1514618089 wait complete. 2024-03-08T06:01:02.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.718+0000 7f03077fe640 1 --2- 172.21.15.76:0/3686249357 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0308067c50 0x7f030815b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.718+0000 7f0306ffd640 1 --2- 172.21.15.76:0/3686249357 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f030806f130 0x7f0308155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.718+0000 7fce813e3640 1 --2- 172.21.15.76:0/4180965814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce740df0f0 0x7fce740e00f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0091e0 tx=0x7fce6c02dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7b3482577c168ac server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7f3b70b80640 1 Processor -- start 2024-03-08T06:01:02.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.718+0000 7fd677fff640 1 Processor -- start 2024-03-08T06:01:02.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.717+0000 7f3b70b80640 1 -- start start 2024-03-08T06:01:02.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.718+0000 7f3b70b80640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b6403a090 0x7f3b640d2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.718+0000 7f3b70b80640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b64055d00 0x7f3b640d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.719+0000 7fce73fff640 1 -- 172.21.15.76:0/4180965814 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c03b070 con 0x7fce740df0f0 2024-03-08T06:01:02.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.719+0000 7fce73fff640 1 -- 172.21.15.76:0/4180965814 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c02eaf0 con 0x7fce740df0f0 2024-03-08T06:01:02.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.719+0000 7fce73fff640 1 -- 172.21.15.76:0/4180965814 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c036440 con 0x7fce740df0f0 2024-03-08T06:01:02.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.718+0000 7fc855ffb640 1 -- 172.21.15.76:0/3111112651 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858067c50 con 0x7fc840003b70 2024-03-08T06:01:02.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.718+0000 7fc855ffb640 1 -- 172.21.15.76:0/3111112651 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85806c1b0 con 0x7fc840003b70 2024-03-08T06:01:02.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.718+0000 7fc855ffb640 1 -- 172.21.15.76:0/3111112651 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85806c680 con 0x7fc840003b70 2024-03-08T06:01:02.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.719+0000 7fc85d380640 1 -- 172.21.15.76:0/3111112651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc840003b70 msgr2=0x7fc840003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.719+0000 7fc85d380640 1 --2- 172.21.15.76:0/3111112651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc840003b70 0x7fc840003f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc858075290 tx=0x7fc840005830 comp rx=0 tx=0).stop 2024-03-08T06:01:02.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.719+0000 7fd8572c3640 1 Processor -- start 2024-03-08T06:01:02.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.719+0000 7fc857fff640 1 -- 172.21.15.76:0/3111112651 reap_dead start 2024-03-08T06:01:02.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.719+0000 7fd677fff640 1 -- start start 2024-03-08T06:01:02.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.719+0000 7fc85d380640 1 -- 172.21.15.76:0/3111112651 shutdown_connections 2024-03-08T06:01:02.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.719+0000 7fc85d380640 1 -- 172.21.15.76:0/3111112651 >> 172.21.15.76:0/3111112651 conn(0x7fc85003ac10 msgr2=0x7fc85003b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:02.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.719+0000 7f3b6b7fe640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b6403a090 0x7f3b640d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.719+0000 7fc85d380640 1 -- 172.21.15.76:0/3111112651 shutdown_connections 2024-03-08T06:01:02.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7f3b6b7fe640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b6403a090 0x7f3b640d2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37440/0 (socket says 172.21.15.76:37440) 2024-03-08T06:01:02.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7f3b70b80640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b640df7f0 0x7f3b64033870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fd677fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6780712e0 0x7fd6780ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7f3b70b80640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6404ff70 con 0x7f3b64055d00 2024-03-08T06:01:02.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7f3b70b80640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640dfbd0 con 0x7f3b640df7f0 2024-03-08T06:01:02.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7f3b70b80640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640500f0 con 0x7f3b6403a090 2024-03-08T06:01:02.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7f3b6b7fe640 1 -- 172.21.15.76:0/1931606667 learned_addr learned my addr 172.21.15.76:0/1931606667 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:02.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.719+0000 7fc6680ed640 1 -- 172.21.15.76:0/2313011834 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66004f030 con 0x7fc6601622f0 2024-03-08T06:01:02.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7f3b6affd640 1 --2- 172.21.15.76:0/1931606667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b64055d00 0x7f3b640d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fd677fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6780b2d30 0x7fd67815b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fce823e5640 1 -- 172.21.15.76:0/4180965814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce740df0f0 msgr2=0x7fce740e00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fce823e5640 1 --2- 172.21.15.76:0/4180965814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce740df0f0 0x7fce740e00f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0091e0 tx=0x7fce6c02dd40 comp rx=0 tx=0).stop 2024-03-08T06:01:02.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fce823e5640 1 -- 172.21.15.76:0/4180965814 shutdown_connections 2024-03-08T06:01:02.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fce823e5640 1 --2- 172.21.15.76:0/4180965814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce740df0f0 0x7fce740e00f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fce823e5640 1 -- 172.21.15.76:0/4180965814 >> 172.21.15.76:0/4180965814 conn(0x7fce74009fa0 msgr2=0x7fce7400a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:02.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fd677fff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd6781f01c0 0x7fd6781ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fd677fff640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67806d8c0 con 0x7fd6780b2d30 2024-03-08T06:01:02.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7f3b6bfff640 1 --2- 172.21.15.76:0/1931606667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b640df7f0 0x7f3b64033870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7f3b6b7fe640 1 -- 172.21.15.76:0/1931606667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b640df7f0 msgr2=0x7f3b64033870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7f3b6b7fe640 1 --2- 172.21.15.76:0/1931606667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b640df7f0 0x7f3b64033870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7f3b6b7fe640 1 -- 172.21.15.76:0/1931606667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b64055d00 msgr2=0x7f3b640d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fd6767fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6780b2d30 0x7fd67815b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fd6767fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6780b2d30 0x7fd67815b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51612/0 (socket says 172.21.15.76:51612) 2024-03-08T06:01:02.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fd6767fc640 1 -- 172.21.15.76:0/3044858689 learned_addr learned my addr 172.21.15.76:0/3044858689 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:02.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7f03077fe640 1 -- 172.21.15.76:0/3686249357 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f030806f130 msgr2=0x7f0308155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7f03077fe640 1 --2- 172.21.15.76:0/3686249357 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f030806f130 0x7f0308155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7f03077fe640 1 -- 172.21.15.76:0/3686249357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f030813eb20 msgr2=0x7f03080b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fd677fff640 1 -- 172.21.15.76:0/3044858689 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67806d5e0 con 0x7fd6780712e0 2024-03-08T06:01:02.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fd677fff640 1 -- 172.21.15.76:0/3044858689 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780d0130 con 0x7fd6781f01c0 2024-03-08T06:01:02.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fd6777fe640 1 --2- 172.21.15.76:0/3044858689 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd6781f01c0 0x7fd6781ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fce823e5640 1 -- 172.21.15.76:0/4180965814 shutdown_connections 2024-03-08T06:01:02.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fc6567fc640 1 -- 172.21.15.76:0/2313011834 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc65001a070 con 0x7fc6601622f0 2024-03-08T06:01:02.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.719+0000 7fd8572c3640 1 -- start start 2024-03-08T06:01:02.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd8572c3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85006f600 0x7fd850157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd8572c3640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8500b2710 0x7fd8500ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd8572c3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8500cfd60 0x7fd85015e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd855860640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8500cfd60 0x7fd85015e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd85485e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8500b2710 0x7fd8500ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd85485e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8500b2710 0x7fd8500ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37458/0 (socket says 172.21.15.76:37458) 2024-03-08T06:01:02.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd85485e640 1 -- 172.21.15.76:0/780060352 learned_addr learned my addr 172.21.15.76:0/780060352 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:02.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd8572c3640 1 -- 172.21.15.76:0/780060352 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85015c050 con 0x7fd8500cfd60 2024-03-08T06:01:02.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd8572c3640 1 -- 172.21.15.76:0/780060352 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85015cf20 con 0x7fd85006f600 2024-03-08T06:01:02.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd6767fc640 1 -- 172.21.15.76:0/3044858689 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd6781f01c0 msgr2=0x7fd6781ef4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd6767fc640 1 --2- 172.21.15.76:0/3044858689 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd6781f01c0 0x7fd6781ef4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd6767fc640 1 -- 172.21.15.76:0/3044858689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6780712e0 msgr2=0x7fd6780ae900 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-08T06:01:02.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd8572c3640 1 -- 172.21.15.76:0/780060352 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85006fa40 con 0x7fd8500b2710 2024-03-08T06:01:02.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd6767fc640 1 --2- 172.21.15.76:0/3044858689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6780712e0 0x7fd6780ae900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd6767fc640 1 -- 172.21.15.76:0/3044858689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67815bd10 con 0x7fd6780b2d30 2024-03-08T06:01:02.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fce823e5640 1 -- 172.21.15.76:0/4180965814 wait complete. 2024-03-08T06:01:02.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fc6567fc640 1 --2- 172.21.15.76:0/2313011834 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc64406b4d0 0x7fc64406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fc665e89640 1 --2- 172.21.15.76:0/2313011834 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc64406b4d0 0x7fc64406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd85505f640 1 --2- 172.21.15.76:0/780060352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85006f600 0x7fd850157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7f3b6b7fe640 1 --2- 172.21.15.76:0/1931606667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b64055d00 0x7f3b640d1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7f3b6b7fe640 1 -- 172.21.15.76:0/1931606667 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b640d2f60 con 0x7f3b6403a090 2024-03-08T06:01:02.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7f3b6affd640 1 --2- 172.21.15.76:0/1931606667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b64055d00 0x7f3b640d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:02.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fc6567fc640 1 -- 172.21.15.76:0/2313011834 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc6500898b0 con 0x7fc6601622f0 2024-03-08T06:01:02.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.720+0000 7fc85d380640 1 -- 172.21.15.76:0/3111112651 wait complete. 2024-03-08T06:01:02.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fc85d380640 1 Processor -- start 2024-03-08T06:01:02.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7f03077fe640 1 --2- 172.21.15.76:0/3686249357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f030813eb20 0x7f03080b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7f03077fe640 1 -- 172.21.15.76:0/3686249357 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030815bdc0 con 0x7f0308067c50 2024-03-08T06:01:02.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7f0307fff640 1 --2- 172.21.15.76:0/3686249357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f030813eb20 0x7f03080b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:02.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7f3b6b7fe640 1 --2- 172.21.15.76:0/1931606667 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b6403a090 0x7f3b640d2660 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800b3f0 tx=0x7f3b5800b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd85485e640 1 -- 172.21.15.76:0/780060352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85006f600 msgr2=0x7fd850157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd85485e640 1 --2- 172.21.15.76:0/780060352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85006f600 0x7fd850157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd85485e640 1 -- 172.21.15.76:0/780060352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8500cfd60 msgr2=0x7fd85015e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd85485e640 1 --2- 172.21.15.76:0/780060352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8500cfd60 0x7fd85015e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.721+0000 7fd85485e640 1 -- 172.21.15.76:0/780060352 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8500cdd00 con 0x7fd8500b2710 2024-03-08T06:01:02.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7fc85d380640 1 -- start start 2024-03-08T06:01:02.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7fd6767fc640 1 --2- 172.21.15.76:0/3044858689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6780b2d30 0x7fd67815b350 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd66800e870 tx=0x7fd66800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7f3b57fff640 1 -- 172.21.15.76:0/1931606667 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b58015020 con 0x7f3b6403a090 2024-03-08T06:01:02.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7f3b57fff640 1 -- 172.21.15.76:0/1931606667 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b58011830 con 0x7f3b6403a090 2024-03-08T06:01:02.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7f3b57fff640 1 -- 172.21.15.76:0/1931606667 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b58011c20 con 0x7f3b6403a090 2024-03-08T06:01:02.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7f3b70b80640 1 -- 172.21.15.76:0/1931606667 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b640d1280 con 0x7f3b6403a090 2024-03-08T06:01:02.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7fd85485e640 1 --2- 172.21.15.76:0/780060352 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8500b2710 0x7fd8500ccb40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd848009eb0 tx=0x7fd848004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7f3b70b80640 1 -- 172.21.15.76:0/1931606667 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b640d1840 con 0x7f3b6403a090 2024-03-08T06:01:02.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7fce823e5640 1 Processor -- start 2024-03-08T06:01:02.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7fce823e5640 1 -- start start 2024-03-08T06:01:02.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fce823e5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce74050170 0x7fce740d9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fce823e5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce74053090 0x7fce7404da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fce823e5640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fce740df0f0 0x7fce740dbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fce823e5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7400aac0 con 0x7fce74050170 2024-03-08T06:01:02.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fce823e5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce74054160 con 0x7fce74053090 2024-03-08T06:01:02.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fce823e5640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7400ac40 con 0x7fce740df0f0 2024-03-08T06:01:02.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fce80be2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce74053090 0x7fce7404da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fce80be2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce74053090 0x7fce7404da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.76:57466/0 (socket says 172.21.15.76:57466) 2024-03-08T06:01:02.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fce80be2640 1 -- 172.21.15.76:0/1373980519 learned_addr learned my addr 172.21.15.76:0/1373980519 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:02.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fce81be4640 1 --2- 172.21.15.76:0/1373980519 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fce740df0f0 0x7fce740dbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7fc665e89640 1 --2- 172.21.15.76:0/2313011834 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc64406b4d0 0x7fc64406d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc6580093e0 tx=0x7fc658030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7fd65f7fe640 1 -- 172.21.15.76:0/3044858689 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd668017070 con 0x7fd6780b2d30 2024-03-08T06:01:02.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7fd677fff640 1 -- 172.21.15.76:0/3044858689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd67815bff0 con 0x7fd6780b2d30 2024-03-08T06:01:02.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7fd677fff640 1 -- 172.21.15.76:0/3044858689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67815c5b0 con 0x7fd6780b2d30 2024-03-08T06:01:02.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7f3b70b80640 1 -- 172.21.15.76:0/1931606667 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4c005290 con 0x7f3b6403a090 2024-03-08T06:01:02.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7f03077fe640 1 --2- 172.21.15.76:0/3686249357 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0308067c50 0x7f030815b7e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00c340 tx=0x7f02fc00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fd65f7fe640 1 -- 172.21.15.76:0/3044858689 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66800ee60 con 0x7fd6780b2d30 2024-03-08T06:01:02.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7fd845ffb640 1 -- 172.21.15.76:0/780060352 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848044030 con 0x7fd8500b2710 2024-03-08T06:01:02.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7fd845ffb640 1 -- 172.21.15.76:0/780060352 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84803a040 con 0x7fd8500b2710 2024-03-08T06:01:02.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7fd845ffb640 1 -- 172.21.15.76:0/780060352 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848039070 con 0x7fd8500b2710 2024-03-08T06:01:02.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.722+0000 7fd8572c3640 1 -- 172.21.15.76:0/780060352 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8500cdfe0 con 0x7fd8500b2710 2024-03-08T06:01:02.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fd8572c3640 1 -- 172.21.15.76:0/780060352 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8500ce5a0 con 0x7fd8500b2710 2024-03-08T06:01:02.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fc85d380640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc85000dbe0 0x7fc850178bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fc85d380640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85000dfc0 0x7fc8501772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fc85d380640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8500e1320 0x7fc850052ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fc85d380640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85003c8c0 con 0x7fc85000dbe0 2024-03-08T06:01:02.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fc85d380640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc850006e20 con 0x7fc8500e1320 2024-03-08T06:01:02.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fc85d380640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc850058a50 con 0x7fc85000dfc0 2024-03-08T06:01:02.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fc857fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8500e1320 0x7fc850052ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fc8567fc640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85000dfc0 0x7fc8501772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fc8567fc640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85000dfc0 0x7fc8501772e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37466/0 (socket says 172.21.15.76:37466) 2024-03-08T06:01:02.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.723+0000 7fc8567fc640 1 -- 172.21.15.76:0/2145457260 learned_addr learned my addr 172.21.15.76:0/2145457260 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:02.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fd65f7fe640 1 -- 172.21.15.76:0/3044858689 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66800cb80 con 0x7fd6780b2d30 2024-03-08T06:01:02.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fd8572c3640 1 -- 172.21.15.76:0/780060352 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85004f010 con 0x7fd8500b2710 2024-03-08T06:01:02.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fce813e3640 1 --2- 172.21.15.76:0/1373980519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce74050170 0x7fce740d9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fc856ffd640 1 --2- 172.21.15.76:0/2145457260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc85000dbe0 0x7fc850178bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fce80be2640 1 -- 172.21.15.76:0/1373980519 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fce740df0f0 msgr2=0x7fce740dbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fce80be2640 1 --2- 172.21.15.76:0/1373980519 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fce740df0f0 0x7fce740dbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fce80be2640 1 -- 172.21.15.76:0/1373980519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce74050170 msgr2=0x7fce740d9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fce80be2640 1 --2- 172.21.15.76:0/1373980519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce74050170 0x7fce740d9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fce80be2640 1 -- 172.21.15.76:0/1373980519 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce74168f20 con 0x7fce74053090 2024-03-08T06:01:02.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fc857fff640 1 -- 172.21.15.76:0/2145457260 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85000dfc0 msgr2=0x7fc8501772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fc857fff640 1 --2- 172.21.15.76:0/2145457260 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85000dfc0 0x7fc8501772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fc857fff640 1 -- 172.21.15.76:0/2145457260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc85000dbe0 msgr2=0x7fc850178bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:02.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fc857fff640 1 --2- 172.21.15.76:0/2145457260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc85000dbe0 0x7fc850178bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:02.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7fc857fff640 1 -- 172.21.15.76:0/2145457260 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8501790f0 con 0x7fc8500e1320 2024-03-08T06:01:02.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.725+0000 7fce813e3640 1 --2- 172.21.15.76:0/1373980519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce74050170 0x7fce740d9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:02.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.724+0000 7f3b57fff640 1 -- 172.21.15.76:0/1931606667 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b58018020 con 0x7f3b6403a090 2024-03-08T06:01:02.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.725+0000 7fce80be2640 1 --2- 172.21.15.76:0/1373980519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce74053090 0x7fce7404da60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fce7800b450 tx=0x7fce7800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.725+0000 7fc857fff640 1 --2- 172.21.15.76:0/2145457260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8500e1320 0x7fc850052ca0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc850055b70 tx=0x7fc84c011c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.725+0000 7f3b57fff640 1 --2- 172.21.15.76:0/1931606667 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3b5006b5a0 0x7f3b5006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.725+0000 7fd677fff640 1 -- 172.21.15.76:0/3044858689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd67806a1d0 con 0x7fd6780b2d30 2024-03-08T06:01:02.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.725+0000 7f3b57fff640 1 -- 172.21.15.76:0/1931606667 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3b5808a0d0 con 0x7f3b6403a090 2024-03-08T06:01:02.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.725+0000 7f3b6affd640 1 --2- 172.21.15.76:0/1931606667 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3b5006b5a0 0x7f3b5006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.725+0000 7fd845ffb640 1 -- 172.21.15.76:0/780060352 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd848047070 con 0x7fd8500b2710 2024-03-08T06:01:02.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.726+0000 7fc83f7fe640 1 -- 172.21.15.76:0/2145457260 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c005bf0 con 0x7fc8500e1320 2024-03-08T06:01:02.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.726+0000 7fc85d380640 1 -- 172.21.15.76:0/2145457260 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc850177b10 con 0x7fc8500e1320 2024-03-08T06:01:02.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.726+0000 7fc85d380640 1 -- 172.21.15.76:0/2145457260 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8501780d0 con 0x7fc8500e1320 2024-03-08T06:01:02.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.726+0000 7fc6567fc640 1 -- 172.21.15.76:0/2313011834 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc650057d00 con 0x7fc6601622f0 2024-03-08T06:01:02.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.726+0000 7fce71ffb640 1 -- 172.21.15.76:0/1373980519 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78015020 con 0x7fce74053090 2024-03-08T06:01:02.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.726+0000 7fce823e5640 1 -- 172.21.15.76:0/1373980519 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce74169200 con 0x7fce74053090 2024-03-08T06:01:02.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.726+0000 7fce823e5640 1 -- 172.21.15.76:0/1373980519 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce741697c0 con 0x7fce74053090 2024-03-08T06:01:02.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.726+0000 7f3b6affd640 1 --2- 172.21.15.76:0/1931606667 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3b5006b5a0 0x7f3b5006da60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400e540 tx=0x7f3b6c0b7040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.726+0000 7fd65f7fe640 1 -- 172.21.15.76:0/3044858689 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd66801a070 con 0x7fd6780b2d30 2024-03-08T06:01:02.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.726+0000 7fce71ffb640 1 -- 172.21.15.76:0/1373980519 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce780118b0 con 0x7fce74053090 2024-03-08T06:01:02.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.726+0000 7fce71ffb640 1 -- 172.21.15.76:0/1373980519 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78011ca0 con 0x7fce74053090 2024-03-08T06:01:02.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.726+0000 7fd65f7fe640 1 --2- 172.21.15.76:0/3044858689 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd66406b5d0 0x7fd66406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.727+0000 7fd65f7fe640 1 -- 172.21.15.76:0/3044858689 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd66808aba0 con 0x7fd6780b2d30 2024-03-08T06:01:02.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.727+0000 7fc83f7fe640 1 -- 172.21.15.76:0/2145457260 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c01a070 con 0x7fc8500e1320 2024-03-08T06:01:02.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.727+0000 7fc83f7fe640 1 -- 172.21.15.76:0/2145457260 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c004030 con 0x7fc8500e1320 2024-03-08T06:01:02.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.727+0000 7fd676ffd640 1 --2- 172.21.15.76:0/3044858689 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd66406b5d0 0x7fd66406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.727+0000 7fd676ffd640 1 --2- 172.21.15.76:0/3044858689 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd66406b5d0 0x7fd66406da90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd66c037000 tx=0x7fd66c031000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.727+0000 7f02effff640 1 -- 172.21.15.76:0/3686249357 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0052d0 con 0x7f0308067c50 2024-03-08T06:01:02.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.727+0000 7f02effff640 1 -- 172.21.15.76:0/3686249357 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc005490 con 0x7f0308067c50 2024-03-08T06:01:02.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.727+0000 7f02effff640 1 -- 172.21.15.76:0/3686249357 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc007260 con 0x7f0308067c50 2024-03-08T06:01:02.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.728+0000 7f030e0d0640 1 -- 172.21.15.76:0/3686249357 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030815c0a0 con 0x7f0308067c50 2024-03-08T06:01:02.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.728+0000 7f030e0d0640 1 -- 172.21.15.76:0/3686249357 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030815c660 con 0x7f0308067c50 2024-03-08T06:01:02.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.728+0000 7fc85d380640 1 -- 172.21.15.76:0/2145457260 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc838005290 con 0x7fc8500e1320 2024-03-08T06:01:02.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.726+0000 7fd845ffb640 1 --2- 172.21.15.76:0/780060352 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd83406b520 0x7fd83406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.726+0000 7fd845ffb640 1 -- 172.21.15.76:0/780060352 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd8480aca00 con 0x7fd8500b2710 2024-03-08T06:01:02.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.728+0000 7fce71ffb640 1 -- 172.21.15.76:0/1373980519 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce78018020 con 0x7fce74053090 2024-03-08T06:01:02.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.728+0000 7fce823e5640 1 -- 172.21.15.76:0/1373980519 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5c005290 con 0x7fce74053090 2024-03-08T06:01:02.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.729+0000 7fd85505f640 1 --2- 172.21.15.76:0/780060352 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd83406b520 0x7fd83406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.729+0000 7fce71ffb640 1 --2- 172.21.15.76:0/1373980519 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fce6006b4d0 0x7fce6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.729+0000 7fce71ffb640 1 -- 172.21.15.76:0/1373980519 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fce78056bc0 con 0x7fce74053090 2024-03-08T06:01:02.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.729+0000 7fce813e3640 1 --2- 172.21.15.76:0/1373980519 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fce6006b4d0 0x7fce6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.729+0000 7f030e0d0640 1 -- 172.21.15.76:0/3686249357 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030804f030 con 0x7f0308067c50 2024-03-08T06:01:02.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.729+0000 7fc83f7fe640 1 -- 172.21.15.76:0/2145457260 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc84c012040 con 0x7fc8500e1320 2024-03-08T06:01:02.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.730+0000 7fd85505f640 1 --2- 172.21.15.76:0/780060352 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd83406b520 0x7fd83406d9e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd8400011e0 tx=0x7fd840009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.730+0000 7fce813e3640 1 --2- 172.21.15.76:0/1373980519 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fce6006b4d0 0x7fce6006d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fce6c038000 tx=0x7fce6c032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.730+0000 7fc83f7fe640 1 --2- 172.21.15.76:0/2145457260 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc84406b4d0 0x7fc84406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.730+0000 7f3b57fff640 1 -- 172.21.15.76:0/1931606667 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b5801f030 con 0x7f3b6403a090 2024-03-08T06:01:02.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.730+0000 7fc83f7fe640 1 -- 172.21.15.76:0/2145457260 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc84c08ada0 con 0x7fc8500e1320 2024-03-08T06:01:02.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.730+0000 7fc856ffd640 1 --2- 172.21.15.76:0/2145457260 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc84406b4d0 0x7fc84406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.730+0000 7f02effff640 1 -- 172.21.15.76:0/3686249357 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02fc007400 con 0x7f0308067c50 2024-03-08T06:01:02.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.731+0000 7fc856ffd640 1 --2- 172.21.15.76:0/2145457260 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc84406b4d0 0x7fc84406d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc840005780 tx=0x7fc85806e030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.731+0000 7f02effff640 1 --2- 172.21.15.76:0/3686249357 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f02f406b520 0x7f02f406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:02.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.731+0000 7f02effff640 1 -- 172.21.15.76:0/3686249357 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f02fc0898a0 con 0x7f0308067c50 2024-03-08T06:01:02.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.731+0000 7f0306ffd640 1 --2- 172.21.15.76:0/3686249357 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f02f406b520 0x7f02f406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:02.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.732+0000 7f0306ffd640 1 --2- 172.21.15.76:0/3686249357 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f02f406b520 0x7f02f406d9e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f02f8036000 tx=0x7f02f803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:02.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.733+0000 7fd65f7fe640 1 -- 172.21.15.76:0/3044858689 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6680596c0 con 0x7fd6780b2d30 2024-03-08T06:01:02.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.734+0000 7fc83f7fe640 1 -- 172.21.15.76:0/2145457260 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc84c03d030 con 0x7fc8500e1320 2024-03-08T06:01:02.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.735+0000 7fd845ffb640 1 -- 172.21.15.76:0/780060352 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd848032340 con 0x7fd8500b2710 2024-03-08T06:01:02.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.738+0000 7fce71ffb640 1 -- 172.21.15.76:0/1373980519 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce7801f020 con 0x7fce74053090 2024-03-08T06:01:02.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:02.743+0000 7f02effff640 1 -- 172.21.15.76:0/3686249357 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f02fc057e00 con 0x7f0308067c50 2024-03-08T06:01:03.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.072+0000 7f7b4759e640 1 -- 172.21.15.76:0/1891413806 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7b2c005580 con 0x7f7b400e75d0 2024-03-08T06:01:03.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.073+0000 7f7b36ffd640 1 -- 172.21.15.76:0/1891413806 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b480a9070 con 0x7f7b400e75d0 2024-03-08T06:01:03.074 INFO:teuthology.orchestra.run.smithi076.stdout:55834574860 2024-03-08T06:01:03.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.076+0000 7f7b34ff9640 1 -- 172.21.15.76:0/1891413806 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7b2806b5a0 msgr2=0x7f7b2806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.076+0000 7f7b34ff9640 1 --2- 172.21.15.76:0/1891413806 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7b2806b5a0 0x7f7b2806da60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7b40048ff0 tx=0x7f7b3c008040 comp rx=0 tx=0).stop 2024-03-08T06:01:03.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.076+0000 7f7b34ff9640 1 -- 172.21.15.76:0/1891413806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b400e75d0 msgr2=0x7f7b400d3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.076+0000 7f7b34ff9640 1 --2- 172.21.15.76:0/1891413806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b400e75d0 0x7f7b400d3670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7b48000b60 tx=0x7f7b480689d0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.076+0000 7f7b34ff9640 1 -- 172.21.15.76:0/1891413806 shutdown_connections 2024-03-08T06:01:03.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.077+0000 7f7b34ff9640 1 --2- 172.21.15.76:0/1891413806 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7b2806b5a0 0x7f7b2806da60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.077+0000 7f7b34ff9640 1 --2- 172.21.15.76:0/1891413806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b400e75d0 0x7f7b400d3670 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.077+0000 7f7b34ff9640 1 --2- 172.21.15.76:0/1891413806 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b400345c0 0x7f7b400d3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.077+0000 7f7b34ff9640 1 --2- 172.21.15.76:0/1891413806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b40006990 0x7f7b400d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.077+0000 7f7b34ff9640 1 -- 172.21.15.76:0/1891413806 >> 172.21.15.76:0/1891413806 conn(0x7f7b40055340 msgr2=0x7f7b40050000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:03.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.077+0000 7f7b34ff9640 1 -- 172.21.15.76:0/1891413806 shutdown_connections 2024-03-08T06:01:03.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.077+0000 7f7b34ff9640 1 -- 172.21.15.76:0/1891413806 wait complete. 2024-03-08T06:01:03.094 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.6 2024-03-08T06:01:03.094 DEBUG:teuthology.parallel:result is None 2024-03-08T06:01:03.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.111+0000 7f3b70b80640 1 -- 172.21.15.76:0/1931606667 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3b4c005580 con 0x7f3b6403a090 2024-03-08T06:01:03.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.111+0000 7f3b57fff640 1 -- 172.21.15.76:0/1931606667 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3b58058c80 con 0x7f3b6403a090 2024-03-08T06:01:03.112 INFO:teuthology.orchestra.run.smithi076.stdout:55834574859 2024-03-08T06:01:03.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.114+0000 7f3b70b80640 1 -- 172.21.15.76:0/1931606667 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3b5006b5a0 msgr2=0x7f3b5006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.114+0000 7f3b70b80640 1 --2- 172.21.15.76:0/1931606667 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3b5006b5a0 0x7f3b5006da60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400e540 tx=0x7f3b6c0b7040 comp rx=0 tx=0).stop 2024-03-08T06:01:03.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.114+0000 7f3b70b80640 1 -- 172.21.15.76:0/1931606667 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b6403a090 msgr2=0x7f3b640d2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.114+0000 7f3b70b80640 1 --2- 172.21.15.76:0/1931606667 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b6403a090 0x7f3b640d2660 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800b3f0 tx=0x7f3b5800b8b0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.114+0000 7f3b70b80640 1 -- 172.21.15.76:0/1931606667 shutdown_connections 2024-03-08T06:01:03.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.114+0000 7f3b70b80640 1 --2- 172.21.15.76:0/1931606667 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3b5006b5a0 0x7f3b5006da60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.114+0000 7f3b70b80640 1 --2- 172.21.15.76:0/1931606667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b640df7f0 0x7f3b64033870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.114+0000 7f3b70b80640 1 --2- 172.21.15.76:0/1931606667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b64055d00 0x7f3b640d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.114+0000 7f3b70b80640 1 --2- 172.21.15.76:0/1931606667 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b6403a090 0x7f3b640d2660 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.114+0000 7f3b70b80640 1 -- 172.21.15.76:0/1931606667 >> 172.21.15.76:0/1931606667 conn(0x7f3b6401e6b0 msgr2=0x7f3b64050f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:03.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.114+0000 7f3b70b80640 1 -- 172.21.15.76:0/1931606667 shutdown_connections 2024-03-08T06:01:03.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.114+0000 7f3b70b80640 1 -- 172.21.15.76:0/1931606667 wait complete. 2024-03-08T06:01:03.126 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-03-08T06:01:03.126 DEBUG:teuthology.parallel:result is None 2024-03-08T06:01:03.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.132+0000 7fd677fff640 1 -- 172.21.15.76:0/3044858689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd6780723d0 con 0x7fd6780b2d30 2024-03-08T06:01:03.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.132+0000 7fd65f7fe640 1 -- 172.21.15.76:0/3044858689 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd668058fc0 con 0x7fd6780b2d30 2024-03-08T06:01:03.133 INFO:teuthology.orchestra.run.smithi076.stdout:55834574859 2024-03-08T06:01:03.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.133+0000 7fc6680ed640 1 -- 172.21.15.76:0/2313011834 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc6600b4330 con 0x7fc6601622f0 2024-03-08T06:01:03.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.133+0000 7fc6567fc640 1 -- 172.21.15.76:0/2313011834 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc650087e20 con 0x7fc6601622f0 2024-03-08T06:01:03.134 INFO:teuthology.orchestra.run.smithi076.stdout:55834574859 2024-03-08T06:01:03.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.135+0000 7fc85d380640 1 -- 172.21.15.76:0/2145457260 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc838005850 con 0x7fc8500e1320 2024-03-08T06:01:03.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.135+0000 7fc83f7fe640 1 -- 172.21.15.76:0/2145457260 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc84c059940 con 0x7fc8500e1320 2024-03-08T06:01:03.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.135+0000 7fd677fff640 1 -- 172.21.15.76:0/3044858689 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd66406b5d0 msgr2=0x7fd66406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.136 INFO:teuthology.orchestra.run.smithi076.stdout:55834574859 2024-03-08T06:01:03.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.135+0000 7fd677fff640 1 --2- 172.21.15.76:0/3044858689 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd66406b5d0 0x7fd66406da90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd66c037000 tx=0x7fd66c031000 comp rx=0 tx=0).stop 2024-03-08T06:01:03.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fd677fff640 1 -- 172.21.15.76:0/3044858689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6780b2d30 msgr2=0x7fd67815b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fd677fff640 1 --2- 172.21.15.76:0/3044858689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6780b2d30 0x7fd67815b350 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd66800e870 tx=0x7fd66800ed30 comp rx=0 tx=0).stop 2024-03-08T06:01:03.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fd677fff640 1 -- 172.21.15.76:0/3044858689 shutdown_connections 2024-03-08T06:01:03.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fd677fff640 1 --2- 172.21.15.76:0/3044858689 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd66406b5d0 0x7fd66406da90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fd677fff640 1 --2- 172.21.15.76:0/3044858689 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd6781f01c0 0x7fd6781ef4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fd677fff640 1 --2- 172.21.15.76:0/3044858689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6780b2d30 0x7fd67815b350 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fd677fff640 1 --2- 172.21.15.76:0/3044858689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6780712e0 0x7fd6780ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fd677fff640 1 -- 172.21.15.76:0/3044858689 >> 172.21.15.76:0/3044858689 conn(0x7fd6780cf640 msgr2=0x7fd678155620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:03.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fc6680ed640 1 -- 172.21.15.76:0/2313011834 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc64406b4d0 msgr2=0x7fc64406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fc6680ed640 1 --2- 172.21.15.76:0/2313011834 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc64406b4d0 0x7fc64406d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc6580093e0 tx=0x7fc658030000 comp rx=0 tx=0).stop 2024-03-08T06:01:03.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fc6680ed640 1 -- 172.21.15.76:0/2313011834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6601622f0 msgr2=0x7fc6600b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fc6680ed640 1 --2- 172.21.15.76:0/2313011834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6601622f0 0x7fc6600b24a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc65000c3e0 tx=0x7fc65000c8a0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fd677fff640 1 -- 172.21.15.76:0/3044858689 shutdown_connections 2024-03-08T06:01:03.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fd677fff640 1 -- 172.21.15.76:0/3044858689 wait complete. 2024-03-08T06:01:03.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fc6680ed640 1 -- 172.21.15.76:0/2313011834 shutdown_connections 2024-03-08T06:01:03.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fc6680ed640 1 --2- 172.21.15.76:0/2313011834 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc64406b4d0 0x7fc64406d990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fc6680ed640 1 --2- 172.21.15.76:0/2313011834 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc660166700 0x7fc6601fbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fc6680ed640 1 --2- 172.21.15.76:0/2313011834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6601622f0 0x7fc6600b24a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fc6680ed640 1 --2- 172.21.15.76:0/2313011834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6600cecf0 0x7fc6600b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fc6680ed640 1 -- 172.21.15.76:0/2313011834 >> 172.21.15.76:0/2313011834 conn(0x7fc66006ae60 msgr2=0x7fc6600b2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:03.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fc6680ed640 1 -- 172.21.15.76:0/2313011834 shutdown_connections 2024-03-08T06:01:03.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.136+0000 7fc6680ed640 1 -- 172.21.15.76:0/2313011834 wait complete. 2024-03-08T06:01:03.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.139+0000 7fc85d380640 1 -- 172.21.15.76:0/2145457260 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc84406b4d0 msgr2=0x7fc84406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.139+0000 7fc85d380640 1 --2- 172.21.15.76:0/2145457260 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc84406b4d0 0x7fc84406d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc840005780 tx=0x7fc85806e030 comp rx=0 tx=0).stop 2024-03-08T06:01:03.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.139+0000 7fc85d380640 1 -- 172.21.15.76:0/2145457260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8500e1320 msgr2=0x7fc850052ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.139+0000 7fc85d380640 1 --2- 172.21.15.76:0/2145457260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8500e1320 0x7fc850052ca0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc850055b70 tx=0x7fc84c011c30 comp rx=0 tx=0).stop 2024-03-08T06:01:03.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.139+0000 7fc85d380640 1 -- 172.21.15.76:0/2145457260 shutdown_connections 2024-03-08T06:01:03.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.139+0000 7fc85d380640 1 --2- 172.21.15.76:0/2145457260 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc84406b4d0 0x7fc84406d990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.139+0000 7fc85d380640 1 --2- 172.21.15.76:0/2145457260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8500e1320 0x7fc850052ca0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.139+0000 7fc85d380640 1 --2- 172.21.15.76:0/2145457260 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc85000dfc0 0x7fc8501772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.139+0000 7fc85d380640 1 --2- 172.21.15.76:0/2145457260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc85000dbe0 0x7fc850178bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.139+0000 7fc85d380640 1 -- 172.21.15.76:0/2145457260 >> 172.21.15.76:0/2145457260 conn(0x7fc85003ac10 msgr2=0x7fc850038b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:03.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.139+0000 7fc85d380640 1 -- 172.21.15.76:0/2145457260 shutdown_connections 2024-03-08T06:01:03.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.139+0000 7fc85d380640 1 -- 172.21.15.76:0/2145457260 wait complete. 2024-03-08T06:01:03.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.142+0000 7fd8572c3640 1 -- 172.21.15.76:0/780060352 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd8500b06a0 con 0x7fd8500b2710 2024-03-08T06:01:03.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.143+0000 7fd845ffb640 1 -- 172.21.15.76:0/780060352 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd84807b520 con 0x7fd8500b2710 2024-03-08T06:01:03.143 INFO:teuthology.orchestra.run.smithi076.stdout:55834574859 2024-03-08T06:01:03.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.143+0000 7fce823e5640 1 -- 172.21.15.76:0/1373980519 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fce5c005850 con 0x7fce74053090 2024-03-08T06:01:03.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.144+0000 7fce71ffb640 1 -- 172.21.15.76:0/1373980519 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fce7800d040 con 0x7fce74053090 2024-03-08T06:01:03.145 INFO:teuthology.orchestra.run.smithi076.stdout:55834574859 2024-03-08T06:01:03.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.145+0000 7fd8572c3640 1 -- 172.21.15.76:0/780060352 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd83406b520 msgr2=0x7fd83406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.145+0000 7fd8572c3640 1 --2- 172.21.15.76:0/780060352 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd83406b520 0x7fd83406d9e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd8400011e0 tx=0x7fd840009040 comp rx=0 tx=0).stop 2024-03-08T06:01:03.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.146+0000 7fd8572c3640 1 -- 172.21.15.76:0/780060352 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8500b2710 msgr2=0x7fd8500ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.146+0000 7fd8572c3640 1 --2- 172.21.15.76:0/780060352 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8500b2710 0x7fd8500ccb40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd848009eb0 tx=0x7fd848004a40 comp rx=0 tx=0).stop 2024-03-08T06:01:03.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.146+0000 7fd8572c3640 1 -- 172.21.15.76:0/780060352 shutdown_connections 2024-03-08T06:01:03.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.146+0000 7fd8572c3640 1 --2- 172.21.15.76:0/780060352 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd83406b520 0x7fd83406d9e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.146+0000 7fd8572c3640 1 --2- 172.21.15.76:0/780060352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8500cfd60 0x7fd85015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.146+0000 7fd8572c3640 1 --2- 172.21.15.76:0/780060352 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8500b2710 0x7fd8500ccb40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.146+0000 7fd8572c3640 1 --2- 172.21.15.76:0/780060352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85006f600 0x7fd850157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.146+0000 7fd8572c3640 1 -- 172.21.15.76:0/780060352 >> 172.21.15.76:0/780060352 conn(0x7fd8500b1410 msgr2=0x7fd8500afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:03.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.147+0000 7fd8572c3640 1 -- 172.21.15.76:0/780060352 shutdown_connections 2024-03-08T06:01:03.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.147+0000 7fd8572c3640 1 -- 172.21.15.76:0/780060352 wait complete. 2024-03-08T06:01:03.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.148+0000 7fce823e5640 1 -- 172.21.15.76:0/1373980519 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fce6006b4d0 msgr2=0x7fce6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.148+0000 7fce823e5640 1 --2- 172.21.15.76:0/1373980519 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fce6006b4d0 0x7fce6006d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fce6c038000 tx=0x7fce6c032000 comp rx=0 tx=0).stop 2024-03-08T06:01:03.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.148+0000 7fce823e5640 1 -- 172.21.15.76:0/1373980519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce74053090 msgr2=0x7fce7404da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.148+0000 7fce823e5640 1 --2- 172.21.15.76:0/1373980519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce74053090 0x7fce7404da60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fce7800b450 tx=0x7fce7800b910 comp rx=0 tx=0).stop 2024-03-08T06:01:03.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.148+0000 7fce823e5640 1 -- 172.21.15.76:0/1373980519 shutdown_connections 2024-03-08T06:01:03.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.148+0000 7fce823e5640 1 --2- 172.21.15.76:0/1373980519 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fce6006b4d0 0x7fce6006d990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.148+0000 7fce823e5640 1 --2- 172.21.15.76:0/1373980519 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fce740df0f0 0x7fce740dbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.148+0000 7fce823e5640 1 --2- 172.21.15.76:0/1373980519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce74053090 0x7fce7404da60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.148+0000 7fce823e5640 1 --2- 172.21.15.76:0/1373980519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce74050170 0x7fce740d9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.148+0000 7fce823e5640 1 -- 172.21.15.76:0/1373980519 >> 172.21.15.76:0/1373980519 conn(0x7fce74009fa0 msgr2=0x7fce74004b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:03.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.148+0000 7fce823e5640 1 -- 172.21.15.76:0/1373980519 shutdown_connections 2024-03-08T06:01:03.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.148+0000 7fce823e5640 1 -- 172.21.15.76:0/1373980519 wait complete. 2024-03-08T06:01:03.153 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-03-08T06:01:03.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.152+0000 7f030e0d0640 1 -- 172.21.15.76:0/3686249357 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f03080b1ee0 con 0x7f0308067c50 2024-03-08T06:01:03.153 DEBUG:teuthology.parallel:result is None 2024-03-08T06:01:03.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.153+0000 7f02effff640 1 -- 172.21.15.76:0/3686249357 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f02fc05bea0 con 0x7f0308067c50 2024-03-08T06:01:03.154 INFO:teuthology.orchestra.run.smithi076.stdout:55834574859 2024-03-08T06:01:03.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2024-03-08T06:01:03.155 DEBUG:teuthology.parallel:result is None 2024-03-08T06:01:03.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.155+0000 7f030e0d0640 1 -- 172.21.15.76:0/3686249357 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f02f406b520 msgr2=0x7f02f406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.155+0000 7f030e0d0640 1 --2- 172.21.15.76:0/3686249357 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f02f406b520 0x7f02f406d9e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f02f8036000 tx=0x7f02f803c040 comp rx=0 tx=0).stop 2024-03-08T06:01:03.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.155+0000 7f030e0d0640 1 -- 172.21.15.76:0/3686249357 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0308067c50 msgr2=0x7f030815b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.155+0000 7f030e0d0640 1 --2- 172.21.15.76:0/3686249357 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0308067c50 0x7f030815b7e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00c340 tx=0x7f02fc00c800 comp rx=0 tx=0).stop 2024-03-08T06:01:03.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.155+0000 7f030e0d0640 1 -- 172.21.15.76:0/3686249357 shutdown_connections 2024-03-08T06:01:03.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.155+0000 7f030e0d0640 1 --2- 172.21.15.76:0/3686249357 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f02f406b520 0x7f02f406d9e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.155+0000 7f030e0d0640 1 --2- 172.21.15.76:0/3686249357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f030813eb20 0x7f03080b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.155+0000 7f030e0d0640 1 --2- 172.21.15.76:0/3686249357 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f030806f130 0x7f0308155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.155+0000 7f030e0d0640 1 --2- 172.21.15.76:0/3686249357 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0308067c50 0x7f030815b7e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.155+0000 7f030e0d0640 1 -- 172.21.15.76:0/3686249357 >> 172.21.15.76:0/3686249357 conn(0x7f03080b1b00 msgr2=0x7f03081435d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:03.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.155+0000 7f030e0d0640 1 -- 172.21.15.76:0/3686249357 shutdown_connections 2024-03-08T06:01:03.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.155+0000 7f030e0d0640 1 -- 172.21.15.76:0/3686249357 wait complete. 2024-03-08T06:01:03.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2024-03-08T06:01:03.159 DEBUG:teuthology.parallel:result is None 2024-03-08T06:01:03.161 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.0 2024-03-08T06:01:03.161 DEBUG:teuthology.parallel:result is None 2024-03-08T06:01:03.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-03-08T06:01:03.165 DEBUG:teuthology.parallel:result is None 2024-03-08T06:01:03.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.4 2024-03-08T06:01:03.168 DEBUG:teuthology.parallel:result is None 2024-03-08T06:01:03.168 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-08T06:01:03.168 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:01:03.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.309+0000 7fa6de476640 1 Processor -- start 2024-03-08T06:01:03.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.309+0000 7fa6de476640 1 -- start start 2024-03-08T06:01:03.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.310+0000 7fa6de476640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa6d809a250 0x7fa6d80cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:03.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.310+0000 7fa6de476640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa6d80cf730 0x7fa6d80cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:03.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.310+0000 7fa6de476640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d80ccbb0 con 0x7fa6d815bcd0 2024-03-08T06:01:03.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.310+0000 7fa6de476640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d80cce90 con 0x7fa6d80cf730 2024-03-08T06:01:03.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.310+0000 7fa6de476640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d80cd170 con 0x7fa6d809a250 2024-03-08T06:01:03.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.310+0000 7fa6d7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa6d815bcd0 0x7fa6d8099b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57612/0 (socket says 172.21.15.76:57612) 2024-03-08T06:01:03.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.310+0000 7fa6d7fff640 1 -- 172.21.15.76:0/3956185371 learned_addr learned my addr 172.21.15.76:0/3956185371 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:03.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.310+0000 7fa6dca13640 1 --2- 172.21.15.76:0/3956185371 >> v2:172.21.15.76:3300/0 conn(0x7fa6d80cf730 0x7fa6d80cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:03.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.310+0000 7fa6d77fe640 1 --2- 172.21.15.76:0/3956185371 >> v2:172.21.15.81:3300/0 conn(0x7fa6d809a250 0x7fa6d80cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:03.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.310+0000 7fa6d6ffd640 1 -- 172.21.15.76:0/3956185371 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266097183 0 0) 0x7fa6d80ccbb0 con 0x7fa6d815bcd0 2024-03-08T06:01:03.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.310+0000 7fa6d6ffd640 1 -- 172.21.15.76:0/3956185371 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6c0003520 con 0x7fa6d815bcd0 2024-03-08T06:01:03.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.310+0000 7fa6dca13640 1 -- 172.21.15.76:0/3956185371 >> v2:172.21.15.81:3300/0 conn(0x7fa6d809a250 msgr2=0x7fa6d80cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:03.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.310+0000 7fa6dca13640 1 --2- 172.21.15.76:0/3956185371 >> v2:172.21.15.81:3300/0 conn(0x7fa6d809a250 0x7fa6d80cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.310+0000 7fa6dca13640 1 -- 172.21.15.76:0/3956185371 >> v1:172.21.15.76:6789/0 conn(0x7fa6d815bcd0 legacy=0x7fa6d8099b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.311+0000 7fa6dca13640 1 -- 172.21.15.76:0/3956185371 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d80cd450 con 0x7fa6d80cf730 2024-03-08T06:01:03.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.311+0000 7fa6dca13640 1 --2- 172.21.15.76:0/3956185371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6d80cf730 0x7fa6d80cdc50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8072820 tx=0x7fa6d002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a909302c10512b4c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:03.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.311+0000 7fa6d6ffd640 1 -- 172.21.15.76:0/3956185371 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0039070 con 0x7fa6d80cf730 2024-03-08T06:01:03.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.311+0000 7fa6d6ffd640 1 -- 172.21.15.76:0/3956185371 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d002cd00 con 0x7fa6d80cf730 2024-03-08T06:01:03.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.311+0000 7fa6d6ffd640 1 -- 172.21.15.76:0/3956185371 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0034660 con 0x7fa6d80cf730 2024-03-08T06:01:03.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.312+0000 7fa6de476640 1 -- 172.21.15.76:0/3956185371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6d80cf730 msgr2=0x7fa6d80cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.312+0000 7fa6de476640 1 --2- 172.21.15.76:0/3956185371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6d80cf730 0x7fa6d80cdc50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8072820 tx=0x7fa6d002bf50 comp rx=0 tx=0).stop 2024-03-08T06:01:03.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.312+0000 7fa6de476640 1 -- 172.21.15.76:0/3956185371 shutdown_connections 2024-03-08T06:01:03.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.312+0000 7fa6de476640 1 --2- 172.21.15.76:0/3956185371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6d80cf730 0x7fa6d80cdc50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.312+0000 7fa6de476640 1 --2- 172.21.15.76:0/3956185371 >> v2:172.21.15.81:3300/0 conn(0x7fa6d809a250 0x7fa6d80cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.312+0000 7fa6de476640 1 -- 172.21.15.76:0/3956185371 >> 172.21.15.76:0/3956185371 conn(0x7fa6d80b3440 msgr2=0x7fa6d80b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:03.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.312+0000 7fa6de476640 1 -- 172.21.15.76:0/3956185371 shutdown_connections 2024-03-08T06:01:03.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.312+0000 7fa6de476640 1 -- 172.21.15.76:0/3956185371 wait complete. 2024-03-08T06:01:03.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.313+0000 7fa6de476640 1 Processor -- start 2024-03-08T06:01:03.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.313+0000 7fa6de476640 1 -- start start 2024-03-08T06:01:03.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6de476640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6d809a250 0x7fa6d80c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:03.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6de476640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6d80cf730 0x7fa6d81579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:03.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6de476640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa6d815bcd0 0x7fa6d8155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:03.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6de476640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d809b160 con 0x7fa6d80cf730 2024-03-08T06:01:03.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6de476640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d815c0b0 con 0x7fa6d809a250 2024-03-08T06:01:03.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6de476640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d809b2e0 con 0x7fa6d815bcd0 2024-03-08T06:01:03.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6d77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6d80cf730 0x7fa6d81579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:03.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6d77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6d80cf730 0x7fa6d81579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51650/0 (socket says 172.21.15.76:51650) 2024-03-08T06:01:03.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6d77fe640 1 -- 172.21.15.76:0/1253383628 learned_addr learned my addr 172.21.15.76:0/1253383628 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:03.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6d7fff640 1 --2- 172.21.15.76:0/1253383628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6d809a250 0x7fa6d80c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:03.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6dca13640 1 --2- 172.21.15.76:0/1253383628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa6d815bcd0 0x7fa6d8155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:03.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6d77fe640 1 -- 172.21.15.76:0/1253383628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa6d815bcd0 msgr2=0x7fa6d8155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6d77fe640 1 --2- 172.21.15.76:0/1253383628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa6d815bcd0 0x7fa6d8155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6d77fe640 1 -- 172.21.15.76:0/1253383628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6d809a250 msgr2=0x7fa6d80c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6d77fe640 1 --2- 172.21.15.76:0/1253383628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6d809a250 0x7fa6d80c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6d77fe640 1 -- 172.21.15.76:0/1253383628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d815efa0 con 0x7fa6d80cf730 2024-03-08T06:01:03.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.314+0000 7fa6d7fff640 1 --2- 172.21.15.76:0/1253383628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6d809a250 0x7fa6d80c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:03.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.315+0000 7fa6d77fe640 1 --2- 172.21.15.76:0/1253383628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6d80cf730 0x7fa6d81579c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00e870 tx=0x7fa6cc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:03.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.315+0000 7fa6d4ff9640 1 -- 172.21.15.76:0/1253383628 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc017070 con 0x7fa6d80cf730 2024-03-08T06:01:03.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.315+0000 7fa6d4ff9640 1 -- 172.21.15.76:0/1253383628 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc00c940 con 0x7fa6d80cf730 2024-03-08T06:01:03.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.315+0000 7fa6de476640 1 -- 172.21.15.76:0/1253383628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d80ca0d0 con 0x7fa6d80cf730 2024-03-08T06:01:03.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.315+0000 7fa6d4ff9640 1 -- 172.21.15.76:0/1253383628 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00cd30 con 0x7fa6d80cf730 2024-03-08T06:01:03.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.315+0000 7fa6de476640 1 -- 172.21.15.76:0/1253383628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d80c90e0 con 0x7fa6d80cf730 2024-03-08T06:01:03.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.316+0000 7fa6de476640 1 -- 172.21.15.76:0/1253383628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d809a670 con 0x7fa6d80cf730 2024-03-08T06:01:03.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.316+0000 7fa6d4ff9640 1 -- 172.21.15.76:0/1253383628 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6cc01a070 con 0x7fa6d80cf730 2024-03-08T06:01:03.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.316+0000 7fa6d4ff9640 1 --2- 172.21.15.76:0/1253383628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa6c406b4d0 0x7fa6c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:03.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.317+0000 7fa6d4ff9640 1 -- 172.21.15.76:0/1253383628 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa6cc08a320 con 0x7fa6d80cf730 2024-03-08T06:01:03.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.317+0000 7fa6d7fff640 1 --2- 172.21.15.76:0/1253383628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa6c406b4d0 0x7fa6c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:03.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.317+0000 7fa6d7fff640 1 --2- 172.21.15.76:0/1253383628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa6c406b4d0 0x7fa6c406d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8005880 tx=0x7fa6c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:03.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.321+0000 7fa6d4ff9640 1 -- 172.21.15.76:0/1253383628 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6cc058ef0 con 0x7fa6d80cf730 2024-03-08T06:01:03.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.489+0000 7fa6de476640 1 -- 172.21.15.76:0/1253383628 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6d804f010 con 0x7fa6c406b4d0 2024-03-08T06:01:03.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.496+0000 7fa6d4ff9640 1 -- 172.21.15.76:0/1253383628 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332533 (secure 0 0 0) 0x7fa6d804f010 con 0x7fa6c406b4d0 2024-03-08T06:01:03.501 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:03.503 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-08T06:01:03.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.500+0000 7fa6de476640 1 -- 172.21.15.76:0/1253383628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa6c406b4d0 msgr2=0x7fa6c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.500+0000 7fa6de476640 1 --2- 172.21.15.76:0/1253383628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa6c406b4d0 0x7fa6c406d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8005880 tx=0x7fa6c8009040 comp rx=0 tx=0).stop 2024-03-08T06:01:03.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.500+0000 7fa6de476640 1 -- 172.21.15.76:0/1253383628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6d80cf730 msgr2=0x7fa6d81579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.500+0000 7fa6de476640 1 --2- 172.21.15.76:0/1253383628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6d80cf730 0x7fa6d81579c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00e870 tx=0x7fa6cc00ed30 comp rx=0 tx=0).stop 2024-03-08T06:01:03.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.500+0000 7fa6de476640 1 -- 172.21.15.76:0/1253383628 shutdown_connections 2024-03-08T06:01:03.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.500+0000 7fa6de476640 1 --2- 172.21.15.76:0/1253383628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa6c406b4d0 0x7fa6c406d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.500+0000 7fa6de476640 1 --2- 172.21.15.76:0/1253383628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa6d815bcd0 0x7fa6d8155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.500+0000 7fa6de476640 1 --2- 172.21.15.76:0/1253383628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6d80cf730 0x7fa6d81579c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.500+0000 7fa6de476640 1 --2- 172.21.15.76:0/1253383628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6d809a250 0x7fa6d80c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.500+0000 7fa6de476640 1 -- 172.21.15.76:0/1253383628 >> 172.21.15.76:0/1253383628 conn(0x7fa6d80b3440 msgr2=0x7fa6d80cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:03.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.500+0000 7fa6de476640 1 -- 172.21.15.76:0/1253383628 shutdown_connections 2024-03-08T06:01:03.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.500+0000 7fa6de476640 1 -- 172.21.15.76:0/1253383628 wait complete. 2024-03-08T06:01:03.511 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-03-08T06:01:01.925942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219560,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755160,"statfs":{"total":773094113280,"available":772869283840,"internally_reserved":0,"allocated":3481600,"data_stored":2089616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003162"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.250953+0000","last_change":"2024-03-08T06:00:35.165081+0000","last_active":"2024-03-08T06:00:43.250953+0000","last_peered":"2024-03-08T06:00:43.250953+0000","last_clean":"2024-03-08T06:00:43.250953+0000","last_became_active":"2024-03-08T06:00:35.164818+0000","last_became_peered":"2024-03-08T06:00:35.164818+0000","last_unstale":"2024-03-08T06:00:43.250953+0000","last_undegraded":"2024-03-08T06:00:43.250953+0000","last_fullsized":"2024-03-08T06:00:43.250953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:59:03.457310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205755+0000","last_change":"2024-03-08T06:00:36.182716+0000","last_active":"2024-03-08T06:00:42.205755+0000","last_peered":"2024-03-08T06:00:42.205755+0000","last_clean":"2024-03-08T06:00:42.205755+0000","last_became_active":"2024-03-08T06:00:36.182377+0000","last_became_peered":"2024-03-08T06:00:36.182377+0000","last_unstale":"2024-03-08T06:00:42.205755+0000","last_undegraded":"2024-03-08T06:00:42.205755+0000","last_fullsized":"2024-03-08T06:00:42.205755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:59:39.518589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.204982+0000","last_change":"2024-03-08T06:00:40.773175+0000","last_active":"2024-03-08T06:00:42.204982+0000","last_peered":"2024-03-08T06:00:42.204982+0000","last_clean":"2024-03-08T06:00:42.204982+0000","last_became_active":"2024-03-08T06:00:40.772777+0000","last_became_peered":"2024-03-08T06:00:40.772777+0000","last_unstale":"2024-03-08T06:00:42.204982+0000","last_undegraded":"2024-03-08T06:00:42.204982+0000","last_fullsized":"2024-03-08T06:00:42.204982+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:51:44.068190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205870+0000","last_change":"2024-03-08T06:00:35.177115+0000","last_active":"2024-03-08T06:00:42.205870+0000","last_peered":"2024-03-08T06:00:42.205870+0000","last_clean":"2024-03-08T06:00:42.205870+0000","last_became_active":"2024-03-08T06:00:35.176827+0000","last_became_peered":"2024-03-08T06:00:35.176827+0000","last_unstale":"2024-03-08T06:00:42.205870+0000","last_undegraded":"2024-03-08T06:00:42.205870+0000","last_fullsized":"2024-03-08T06:00:42.205870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:57:55.324372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251714+0000","last_change":"2024-03-08T06:00:36.191516+0000","last_active":"2024-03-08T06:00:43.251714+0000","last_peered":"2024-03-08T06:00:43.251714+0000","last_clean":"2024-03-08T06:00:43.251714+0000","last_became_active":"2024-03-08T06:00:36.191258+0000","last_became_peered":"2024-03-08T06:00:36.191258+0000","last_unstale":"2024-03-08T06:00:43.251714+0000","last_undegraded":"2024-03-08T06:00:43.251714+0000","last_fullsized":"2024-03-08T06:00:43.251714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206166+0000","last_change":"2024-03-08T06:00:40.219312+0000","last_active":"2024-03-08T06:00:42.206166+0000","last_peered":"2024-03-08T06:00:42.206166+0000","last_clean":"2024-03-08T06:00:42.206166+0000","last_became_active":"2024-03-08T06:00:40.218891+0000","last_became_peered":"2024-03-08T06:00:40.218891+0000","last_unstale":"2024-03-08T06:00:42.206166+0000","last_undegraded":"2024-03-08T06:00:42.206166+0000","last_fullsized":"2024-03-08T06:00:42.206166+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:19:30.099147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752093+0000","last_change":"2024-03-08T06:00:35.163380+0000","last_active":"2024-03-08T06:00:42.752093+0000","last_peered":"2024-03-08T06:00:42.752093+0000","last_clean":"2024-03-08T06:00:42.752093+0000","last_became_active":"2024-03-08T06:00:35.163123+0000","last_became_peered":"2024-03-08T06:00:35.163123+0000","last_unstale":"2024-03-08T06:00:42.752093+0000","last_undegraded":"2024-03-08T06:00:42.752093+0000","last_fullsized":"2024-03-08T06:00:42.752093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252109+0000","last_change":"2024-03-08T06:00:36.194135+0000","last_active":"2024-03-08T06:00:43.252109+0000","last_peered":"2024-03-08T06:00:43.252109+0000","last_clean":"2024-03-08T06:00:43.252109+0000","last_became_active":"2024-03-08T06:00:36.193870+0000","last_became_peered":"2024-03-08T06:00:36.193870+0000","last_unstale":"2024-03-08T06:00:43.252109+0000","last_undegraded":"2024-03-08T06:00:43.252109+0000","last_fullsized":"2024-03-08T06:00:43.252109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:02:15.458747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206553+0000","last_change":"2024-03-08T06:00:40.221366+0000","last_active":"2024-03-08T06:00:42.206553+0000","last_peered":"2024-03-08T06:00:42.206553+0000","last_clean":"2024-03-08T06:00:42.206553+0000","last_became_active":"2024-03-08T06:00:40.220967+0000","last_became_peered":"2024-03-08T06:00:40.220967+0000","last_unstale":"2024-03-08T06:00:42.206553+0000","last_undegraded":"2024-03-08T06:00:42.206553+0000","last_fullsized":"2024-03-08T06:00:42.206553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:16:51.281060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751868+0000","last_change":"2024-03-08T06:00:35.173354+0000","last_active":"2024-03-08T06:00:42.751868+0000","last_peered":"2024-03-08T06:00:42.751868+0000","last_clean":"2024-03-08T06:00:42.751868+0000","last_became_active":"2024-03-08T06:00:35.173105+0000","last_became_peered":"2024-03-08T06:00:35.173105+0000","last_unstale":"2024-03-08T06:00:42.751868+0000","last_undegraded":"2024-03-08T06:00:42.751868+0000","last_fullsized":"2024-03-08T06:00:42.751868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252178+0000","last_change":"2024-03-08T06:00:36.193424+0000","last_active":"2024-03-08T06:00:43.252178+0000","last_peered":"2024-03-08T06:00:43.252178+0000","last_clean":"2024-03-08T06:00:43.252178+0000","last_became_active":"2024-03-08T06:00:36.191212+0000","last_became_peered":"2024-03-08T06:00:36.191212+0000","last_unstale":"2024-03-08T06:00:43.252178+0000","last_undegraded":"2024-03-08T06:00:43.252178+0000","last_fullsized":"2024-03-08T06:00:43.252178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:02.331109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206122+0000","last_change":"2024-03-08T06:00:40.221344+0000","last_active":"2024-03-08T06:00:42.206122+0000","last_peered":"2024-03-08T06:00:42.206122+0000","last_clean":"2024-03-08T06:00:42.206122+0000","last_became_active":"2024-03-08T06:00:40.220925+0000","last_became_peered":"2024-03-08T06:00:40.220925+0000","last_unstale":"2024-03-08T06:00:42.206122+0000","last_undegraded":"2024-03-08T06:00:42.206122+0000","last_fullsized":"2024-03-08T06:00:42.206122+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:06.372126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254602+0000","last_change":"2024-03-08T06:00:36.186043+0000","last_active":"2024-03-08T06:00:43.254602+0000","last_peered":"2024-03-08T06:00:43.254602+0000","last_clean":"2024-03-08T06:00:43.254602+0000","last_became_active":"2024-03-08T06:00:36.185793+0000","last_became_peered":"2024-03-08T06:00:36.185793+0000","last_unstale":"2024-03-08T06:00:43.254602+0000","last_undegraded":"2024-03-08T06:00:43.254602+0000","last_fullsized":"2024-03-08T06:00:43.254602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:01:37.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751980+0000","last_change":"2024-03-08T06:00:35.163406+0000","last_active":"2024-03-08T06:00:42.751980+0000","last_peered":"2024-03-08T06:00:42.751980+0000","last_clean":"2024-03-08T06:00:42.751980+0000","last_became_active":"2024-03-08T06:00:35.163138+0000","last_became_peered":"2024-03-08T06:00:35.163138+0000","last_unstale":"2024-03-08T06:00:42.751980+0000","last_undegraded":"2024-03-08T06:00:42.751980+0000","last_fullsized":"2024-03-08T06:00:42.751980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:38.366092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206808+0000","last_change":"2024-03-08T06:00:40.219654+0000","last_active":"2024-03-08T06:00:42.206808+0000","last_peered":"2024-03-08T06:00:42.206808+0000","last_clean":"2024-03-08T06:00:42.206808+0000","last_became_active":"2024-03-08T06:00:40.219225+0000","last_became_peered":"2024-03-08T06:00:40.219225+0000","last_unstale":"2024-03-08T06:00:42.206808+0000","last_undegraded":"2024-03-08T06:00:42.206808+0000","last_fullsized":"2024-03-08T06:00:42.206808+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:34:22.301392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205304+0000","last_change":"2024-03-08T06:00:36.184886+0000","last_active":"2024-03-08T06:00:42.205304+0000","last_peered":"2024-03-08T06:00:42.205304+0000","last_clean":"2024-03-08T06:00:42.205304+0000","last_became_active":"2024-03-08T06:00:36.184582+0000","last_became_peered":"2024-03-08T06:00:36.184582+0000","last_unstale":"2024-03-08T06:00:42.205304+0000","last_undegraded":"2024-03-08T06:00:42.205304+0000","last_fullsized":"2024-03-08T06:00:42.205304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251830+0000","last_change":"2024-03-08T06:00:35.171210+0000","last_active":"2024-03-08T06:00:43.251830+0000","last_peered":"2024-03-08T06:00:43.251830+0000","last_clean":"2024-03-08T06:00:43.251830+0000","last_became_active":"2024-03-08T06:00:35.170784+0000","last_became_peered":"2024-03-08T06:00:35.170784+0000","last_unstale":"2024-03-08T06:00:43.251830+0000","last_undegraded":"2024-03-08T06:00:43.251830+0000","last_fullsized":"2024-03-08T06:00:43.251830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206300+0000","last_change":"2024-03-08T06:00:40.221460+0000","last_active":"2024-03-08T06:00:42.206300+0000","last_peered":"2024-03-08T06:00:42.206300+0000","last_clean":"2024-03-08T06:00:42.206300+0000","last_became_active":"2024-03-08T06:00:40.221046+0000","last_became_peered":"2024-03-08T06:00:40.221046+0000","last_unstale":"2024-03-08T06:00:42.206300+0000","last_undegraded":"2024-03-08T06:00:42.206300+0000","last_fullsized":"2024-03-08T06:00:42.206300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:35.686997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205125+0000","last_change":"2024-03-08T06:00:36.181765+0000","last_active":"2024-03-08T06:00:42.205125+0000","last_peered":"2024-03-08T06:00:42.205125+0000","last_clean":"2024-03-08T06:00:42.205125+0000","last_became_active":"2024-03-08T06:00:36.181503+0000","last_became_peered":"2024-03-08T06:00:36.181503+0000","last_unstale":"2024-03-08T06:00:42.205125+0000","last_undegraded":"2024-03-08T06:00:42.205125+0000","last_fullsized":"2024-03-08T06:00:42.205125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:01:37.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.250999+0000","last_change":"2024-03-08T06:00:35.164505+0000","last_active":"2024-03-08T06:00:43.250999+0000","last_peered":"2024-03-08T06:00:43.250999+0000","last_clean":"2024-03-08T06:00:43.250999+0000","last_became_active":"2024-03-08T06:00:35.164203+0000","last_became_peered":"2024-03-08T06:00:35.164203+0000","last_unstale":"2024-03-08T06:00:43.250999+0000","last_undegraded":"2024-03-08T06:00:43.250999+0000","last_fullsized":"2024-03-08T06:00:43.250999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:38.366092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253073+0000","last_change":"2024-03-08T06:00:40.772590+0000","last_active":"2024-03-08T06:00:43.253073+0000","last_peered":"2024-03-08T06:00:43.253073+0000","last_clean":"2024-03-08T06:00:43.253073+0000","last_became_active":"2024-03-08T06:00:40.772198+0000","last_became_peered":"2024-03-08T06:00:40.772198+0000","last_unstale":"2024-03-08T06:00:43.253073+0000","last_undegraded":"2024-03-08T06:00:43.253073+0000","last_fullsized":"2024-03-08T06:00:43.253073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:24:20.856936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206949+0000","last_change":"2024-03-08T06:00:36.180323+0000","last_active":"2024-03-08T06:00:42.206949+0000","last_peered":"2024-03-08T06:00:42.206949+0000","last_clean":"2024-03-08T06:00:42.206949+0000","last_became_active":"2024-03-08T06:00:36.180087+0000","last_became_peered":"2024-03-08T06:00:36.180087+0000","last_unstale":"2024-03-08T06:00:42.206949+0000","last_undegraded":"2024-03-08T06:00:42.206949+0000","last_fullsized":"2024-03-08T06:00:42.206949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:30:07.063230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751552+0000","last_change":"2024-03-08T06:00:35.179401+0000","last_active":"2024-03-08T06:00:42.751552+0000","last_peered":"2024-03-08T06:00:42.751552+0000","last_clean":"2024-03-08T06:00:42.751552+0000","last_became_active":"2024-03-08T06:00:35.179121+0000","last_became_peered":"2024-03-08T06:00:35.179121+0000","last_unstale":"2024-03-08T06:00:42.751552+0000","last_undegraded":"2024-03-08T06:00:42.751552+0000","last_fullsized":"2024-03-08T06:00:42.751552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:30:06.058910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.207092+0000","last_change":"2024-03-08T06:00:40.222278+0000","last_active":"2024-03-08T06:00:42.207092+0000","last_peered":"2024-03-08T06:00:42.207092+0000","last_clean":"2024-03-08T06:00:42.207092+0000","last_became_active":"2024-03-08T06:00:40.221874+0000","last_became_peered":"2024-03-08T06:00:40.221874+0000","last_unstale":"2024-03-08T06:00:42.207092+0000","last_undegraded":"2024-03-08T06:00:42.207092+0000","last_fullsized":"2024-03-08T06:00:42.207092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:33.029056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251991+0000","last_change":"2024-03-08T06:00:36.180994+0000","last_active":"2024-03-08T06:00:43.251991+0000","last_peered":"2024-03-08T06:00:43.251991+0000","last_clean":"2024-03-08T06:00:43.251991+0000","last_became_active":"2024-03-08T06:00:36.178891+0000","last_became_peered":"2024-03-08T06:00:36.178891+0000","last_unstale":"2024-03-08T06:00:43.251991+0000","last_undegraded":"2024-03-08T06:00:43.251991+0000","last_fullsized":"2024-03-08T06:00:43.251991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:40:16.053974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253772+0000","last_change":"2024-03-08T06:00:35.166077+0000","last_active":"2024-03-08T06:00:43.253772+0000","last_peered":"2024-03-08T06:00:43.253772+0000","last_clean":"2024-03-08T06:00:43.253772+0000","last_became_active":"2024-03-08T06:00:35.165721+0000","last_became_peered":"2024-03-08T06:00:35.165721+0000","last_unstale":"2024-03-08T06:00:43.253772+0000","last_undegraded":"2024-03-08T06:00:43.253772+0000","last_fullsized":"2024-03-08T06:00:43.253772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206253+0000","last_change":"2024-03-08T06:00:40.772000+0000","last_active":"2024-03-08T06:00:42.206253+0000","last_peered":"2024-03-08T06:00:42.206253+0000","last_clean":"2024-03-08T06:00:42.206253+0000","last_became_active":"2024-03-08T06:00:40.771449+0000","last_became_peered":"2024-03-08T06:00:40.771449+0000","last_unstale":"2024-03-08T06:00:42.206253+0000","last_undegraded":"2024-03-08T06:00:42.206253+0000","last_fullsized":"2024-03-08T06:00:42.206253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:44:48.992193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205931+0000","last_change":"2024-03-08T06:00:36.185234+0000","last_active":"2024-03-08T06:00:42.205931+0000","last_peered":"2024-03-08T06:00:42.205931+0000","last_clean":"2024-03-08T06:00:42.205931+0000","last_became_active":"2024-03-08T06:00:36.184020+0000","last_became_peered":"2024-03-08T06:00:36.184020+0000","last_unstale":"2024-03-08T06:00:42.205931+0000","last_undegraded":"2024-03-08T06:00:42.205931+0000","last_fullsized":"2024-03-08T06:00:42.205931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:39:59.986300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252725+0000","last_change":"2024-03-08T06:00:35.166222+0000","last_active":"2024-03-08T06:00:43.252725+0000","last_peered":"2024-03-08T06:00:43.252725+0000","last_clean":"2024-03-08T06:00:43.252725+0000","last_became_active":"2024-03-08T06:00:35.165815+0000","last_became_peered":"2024-03-08T06:00:35.165815+0000","last_unstale":"2024-03-08T06:00:43.252725+0000","last_undegraded":"2024-03-08T06:00:43.252725+0000","last_fullsized":"2024-03-08T06:00:43.252725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.204928+0000","last_change":"2024-03-08T06:00:40.771309+0000","last_active":"2024-03-08T06:00:42.204928+0000","last_peered":"2024-03-08T06:00:42.204928+0000","last_clean":"2024-03-08T06:00:42.204928+0000","last_became_active":"2024-03-08T06:00:40.770916+0000","last_became_peered":"2024-03-08T06:00:40.770916+0000","last_unstale":"2024-03-08T06:00:42.204928+0000","last_undegraded":"2024-03-08T06:00:42.204928+0000","last_fullsized":"2024-03-08T06:00:42.204928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:26:51.436673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252117+0000","last_change":"2024-03-08T06:00:36.177665+0000","last_active":"2024-03-08T06:00:43.252117+0000","last_peered":"2024-03-08T06:00:43.252117+0000","last_clean":"2024-03-08T06:00:43.252117+0000","last_became_active":"2024-03-08T06:00:36.177409+0000","last_became_peered":"2024-03-08T06:00:36.177409+0000","last_unstale":"2024-03-08T06:00:43.252117+0000","last_undegraded":"2024-03-08T06:00:43.252117+0000","last_fullsized":"2024-03-08T06:00:43.252117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752203+0000","last_change":"2024-03-08T06:00:35.179418+0000","last_active":"2024-03-08T06:00:42.752203+0000","last_peered":"2024-03-08T06:00:42.752203+0000","last_clean":"2024-03-08T06:00:42.752203+0000","last_became_active":"2024-03-08T06:00:35.179132+0000","last_became_peered":"2024-03-08T06:00:35.179132+0000","last_unstale":"2024-03-08T06:00:42.752203+0000","last_undegraded":"2024-03-08T06:00:42.752203+0000","last_fullsized":"2024-03-08T06:00:42.752203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:11.262668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251653+0000","last_change":"2024-03-08T06:00:40.771662+0000","last_active":"2024-03-08T06:00:43.251653+0000","last_peered":"2024-03-08T06:00:43.251653+0000","last_clean":"2024-03-08T06:00:43.251653+0000","last_became_active":"2024-03-08T06:00:40.771168+0000","last_became_peered":"2024-03-08T06:00:40.771168+0000","last_unstale":"2024-03-08T06:00:43.251653+0000","last_undegraded":"2024-03-08T06:00:43.251653+0000","last_fullsized":"2024-03-08T06:00:43.251653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:16:51.281060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252306+0000","last_change":"2024-03-08T06:00:36.193542+0000","last_active":"2024-03-08T06:00:43.252306+0000","last_peered":"2024-03-08T06:00:43.252306+0000","last_clean":"2024-03-08T06:00:43.252306+0000","last_became_active":"2024-03-08T06:00:36.191695+0000","last_became_peered":"2024-03-08T06:00:36.191695+0000","last_unstale":"2024-03-08T06:00:43.252306+0000","last_undegraded":"2024-03-08T06:00:43.252306+0000","last_fullsized":"2024-03-08T06:00:43.252306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:16:14.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252207+0000","last_change":"2024-03-08T06:00:35.177137+0000","last_active":"2024-03-08T06:00:43.252207+0000","last_peered":"2024-03-08T06:00:43.252207+0000","last_clean":"2024-03-08T06:00:43.252207+0000","last_became_active":"2024-03-08T06:00:35.176790+0000","last_became_peered":"2024-03-08T06:00:35.176790+0000","last_unstale":"2024-03-08T06:00:43.252207+0000","last_undegraded":"2024-03-08T06:00:43.252207+0000","last_fullsized":"2024-03-08T06:00:43.252207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:11.262668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252112+0000","last_change":"2024-03-08T06:00:40.220329+0000","last_active":"2024-03-08T06:00:43.252112+0000","last_peered":"2024-03-08T06:00:43.252112+0000","last_clean":"2024-03-08T06:00:43.252112+0000","last_became_active":"2024-03-08T06:00:40.219921+0000","last_became_peered":"2024-03-08T06:00:40.219921+0000","last_unstale":"2024-03-08T06:00:43.252112+0000","last_undegraded":"2024-03-08T06:00:43.252112+0000","last_fullsized":"2024-03-08T06:00:43.252112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:08:55.908866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.207142+0000","last_change":"2024-03-08T06:00:36.187190+0000","last_active":"2024-03-08T06:00:42.207142+0000","last_peered":"2024-03-08T06:00:42.207142+0000","last_clean":"2024-03-08T06:00:42.207142+0000","last_became_active":"2024-03-08T06:00:36.186916+0000","last_became_peered":"2024-03-08T06:00:36.186916+0000","last_unstale":"2024-03-08T06:00:42.207142+0000","last_undegraded":"2024-03-08T06:00:42.207142+0000","last_fullsized":"2024-03-08T06:00:42.207142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:39:00.058426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251977+0000","last_change":"2024-03-08T06:00:35.179558+0000","last_active":"2024-03-08T06:00:43.251977+0000","last_peered":"2024-03-08T06:00:43.251977+0000","last_clean":"2024-03-08T06:00:43.251977+0000","last_became_active":"2024-03-08T06:00:35.179266+0000","last_became_peered":"2024-03-08T06:00:35.179266+0000","last_unstale":"2024-03-08T06:00:43.251977+0000","last_undegraded":"2024-03-08T06:00:43.251977+0000","last_fullsized":"2024-03-08T06:00:43.251977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:38.366092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252067+0000","last_change":"2024-03-08T06:00:40.224570+0000","last_active":"2024-03-08T06:00:43.252067+0000","last_peered":"2024-03-08T06:00:43.252067+0000","last_clean":"2024-03-08T06:00:43.252067+0000","last_became_active":"2024-03-08T06:00:40.222949+0000","last_became_peered":"2024-03-08T06:00:40.222949+0000","last_unstale":"2024-03-08T06:00:43.252067+0000","last_undegraded":"2024-03-08T06:00:43.252067+0000","last_fullsized":"2024-03-08T06:00:43.252067+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:16:51.281060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251279+0000","last_change":"2024-03-08T06:00:36.178768+0000","last_active":"2024-03-08T06:00:43.251279+0000","last_peered":"2024-03-08T06:00:43.251279+0000","last_clean":"2024-03-08T06:00:43.251279+0000","last_became_active":"2024-03-08T06:00:36.177890+0000","last_became_peered":"2024-03-08T06:00:36.177890+0000","last_unstale":"2024-03-08T06:00:43.251279+0000","last_undegraded":"2024-03-08T06:00:43.251279+0000","last_fullsized":"2024-03-08T06:00:43.251279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:01:37.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251961+0000","last_change":"2024-03-08T06:00:35.177031+0000","last_active":"2024-03-08T06:00:43.251961+0000","last_peered":"2024-03-08T06:00:43.251961+0000","last_clean":"2024-03-08T06:00:43.251961+0000","last_became_active":"2024-03-08T06:00:35.176687+0000","last_became_peered":"2024-03-08T06:00:35.176687+0000","last_unstale":"2024-03-08T06:00:43.251961+0000","last_undegraded":"2024-03-08T06:00:43.251961+0000","last_fullsized":"2024-03-08T06:00:43.251961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253204+0000","last_change":"2024-03-08T06:00:40.219392+0000","last_active":"2024-03-08T06:00:43.253204+0000","last_peered":"2024-03-08T06:00:43.253204+0000","last_clean":"2024-03-08T06:00:43.253204+0000","last_became_active":"2024-03-08T06:00:40.219040+0000","last_became_peered":"2024-03-08T06:00:40.219040+0000","last_unstale":"2024-03-08T06:00:43.253204+0000","last_undegraded":"2024-03-08T06:00:43.253204+0000","last_fullsized":"2024-03-08T06:00:43.253204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:33.029056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251221+0000","last_change":"2024-03-08T06:00:36.185525+0000","last_active":"2024-03-08T06:00:43.251221+0000","last_peered":"2024-03-08T06:00:43.251221+0000","last_clean":"2024-03-08T06:00:43.251221+0000","last_became_active":"2024-03-08T06:00:36.185111+0000","last_became_peered":"2024-03-08T06:00:36.185111+0000","last_unstale":"2024-03-08T06:00:43.251221+0000","last_undegraded":"2024-03-08T06:00:43.251221+0000","last_fullsized":"2024-03-08T06:00:43.251221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:52.669024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751556+0000","last_change":"2024-03-08T06:00:35.166949+0000","last_active":"2024-03-08T06:00:42.751556+0000","last_peered":"2024-03-08T06:00:42.751556+0000","last_clean":"2024-03-08T06:00:42.751556+0000","last_became_active":"2024-03-08T06:00:35.166680+0000","last_became_peered":"2024-03-08T06:00:35.166680+0000","last_unstale":"2024-03-08T06:00:42.751556+0000","last_undegraded":"2024-03-08T06:00:42.751556+0000","last_fullsized":"2024-03-08T06:00:42.751556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:58:33.115927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251873+0000","last_change":"2024-03-08T06:00:40.771268+0000","last_active":"2024-03-08T06:00:43.251873+0000","last_peered":"2024-03-08T06:00:43.251873+0000","last_clean":"2024-03-08T06:00:43.251873+0000","last_became_active":"2024-03-08T06:00:40.770846+0000","last_became_peered":"2024-03-08T06:00:40.770846+0000","last_unstale":"2024-03-08T06:00:43.251873+0000","last_undegraded":"2024-03-08T06:00:43.251873+0000","last_fullsized":"2024-03-08T06:00:43.251873+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:34:22.301392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205219+0000","last_change":"2024-03-08T06:00:36.200350+0000","last_active":"2024-03-08T06:00:42.205219+0000","last_peered":"2024-03-08T06:00:42.205219+0000","last_clean":"2024-03-08T06:00:42.205219+0000","last_became_active":"2024-03-08T06:00:36.200036+0000","last_became_peered":"2024-03-08T06:00:36.200036+0000","last_unstale":"2024-03-08T06:00:42.205219+0000","last_undegraded":"2024-03-08T06:00:42.205219+0000","last_fullsized":"2024-03-08T06:00:42.205219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:10:17.180499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251960+0000","last_change":"2024-03-08T06:00:35.176857+0000","last_active":"2024-03-08T06:00:43.251960+0000","last_peered":"2024-03-08T06:00:43.251960+0000","last_clean":"2024-03-08T06:00:43.251960+0000","last_became_active":"2024-03-08T06:00:35.176593+0000","last_became_peered":"2024-03-08T06:00:35.176593+0000","last_unstale":"2024-03-08T06:00:43.251960+0000","last_undegraded":"2024-03-08T06:00:43.251960+0000","last_fullsized":"2024-03-08T06:00:43.251960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:41.059111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751648+0000","last_change":"2024-03-08T06:00:26.178419+0000","last_active":"2024-03-08T06:00:42.751648+0000","last_peered":"2024-03-08T06:00:42.751648+0000","last_clean":"2024-03-08T06:00:42.751648+0000","last_became_active":"2024-03-08T06:00:26.175826+0000","last_became_peered":"2024-03-08T06:00:26.175826+0000","last_unstale":"2024-03-08T06:00:42.751648+0000","last_undegraded":"2024-03-08T06:00:42.751648+0000","last_fullsized":"2024-03-08T06:00:42.751648+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_clean_scrub_stamp":"2024-03-08T06:00:24.029492+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:36:00.682517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254739+0000","last_change":"2024-03-08T06:00:40.773106+0000","last_active":"2024-03-08T06:00:43.254739+0000","last_peered":"2024-03-08T06:00:43.254739+0000","last_clean":"2024-03-08T06:00:43.254739+0000","last_became_active":"2024-03-08T06:00:40.772665+0000","last_became_peered":"2024-03-08T06:00:40.772665+0000","last_unstale":"2024-03-08T06:00:43.254739+0000","last_undegraded":"2024-03-08T06:00:43.254739+0000","last_fullsized":"2024-03-08T06:00:43.254739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:02:19.499764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752004+0000","last_change":"2024-03-08T06:00:36.204915+0000","last_active":"2024-03-08T06:00:42.752004+0000","last_peered":"2024-03-08T06:00:42.752004+0000","last_clean":"2024-03-08T06:00:42.752004+0000","last_became_active":"2024-03-08T06:00:36.204663+0000","last_became_peered":"2024-03-08T06:00:36.204663+0000","last_unstale":"2024-03-08T06:00:42.752004+0000","last_undegraded":"2024-03-08T06:00:42.752004+0000","last_fullsized":"2024-03-08T06:00:42.752004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:05.458469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252634+0000","last_change":"2024-03-08T06:00:35.173339+0000","last_active":"2024-03-08T06:00:43.252634+0000","last_peered":"2024-03-08T06:00:43.252634+0000","last_clean":"2024-03-08T06:00:43.252634+0000","last_became_active":"2024-03-08T06:00:35.173024+0000","last_became_peered":"2024-03-08T06:00:35.173024+0000","last_unstale":"2024-03-08T06:00:43.252634+0000","last_undegraded":"2024-03-08T06:00:43.252634+0000","last_fullsized":"2024-03-08T06:00:43.252634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:41.059111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254629+0000","last_change":"2024-03-08T06:00:40.771194+0000","last_active":"2024-03-08T06:00:43.254629+0000","last_peered":"2024-03-08T06:00:43.254629+0000","last_clean":"2024-03-08T06:00:43.254629+0000","last_became_active":"2024-03-08T06:00:40.770762+0000","last_became_peered":"2024-03-08T06:00:40.770762+0000","last_unstale":"2024-03-08T06:00:43.254629+0000","last_undegraded":"2024-03-08T06:00:43.254629+0000","last_fullsized":"2024-03-08T06:00:43.254629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:21:03.123682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252111+0000","last_change":"2024-03-08T06:00:36.186111+0000","last_active":"2024-03-08T06:00:43.252111+0000","last_peered":"2024-03-08T06:00:43.252111+0000","last_clean":"2024-03-08T06:00:43.252111+0000","last_became_active":"2024-03-08T06:00:36.185865+0000","last_became_peered":"2024-03-08T06:00:36.185865+0000","last_unstale":"2024-03-08T06:00:43.252111+0000","last_undegraded":"2024-03-08T06:00:43.252111+0000","last_fullsized":"2024-03-08T06:00:43.252111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:19:26.058130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251711+0000","last_change":"2024-03-08T06:00:35.166769+0000","last_active":"2024-03-08T06:00:43.251711+0000","last_peered":"2024-03-08T06:00:43.251711+0000","last_clean":"2024-03-08T06:00:43.251711+0000","last_became_active":"2024-03-08T06:00:35.166396+0000","last_became_peered":"2024-03-08T06:00:35.166396+0000","last_unstale":"2024-03-08T06:00:43.251711+0000","last_undegraded":"2024-03-08T06:00:43.251711+0000","last_fullsized":"2024-03-08T06:00:43.251711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:30:06.058910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251907+0000","last_change":"2024-03-08T06:00:40.219431+0000","last_active":"2024-03-08T06:00:43.251907+0000","last_peered":"2024-03-08T06:00:43.251907+0000","last_clean":"2024-03-08T06:00:43.251907+0000","last_became_active":"2024-03-08T06:00:40.219091+0000","last_became_peered":"2024-03-08T06:00:40.219091+0000","last_unstale":"2024-03-08T06:00:43.251907+0000","last_undegraded":"2024-03-08T06:00:43.251907+0000","last_fullsized":"2024-03-08T06:00:43.251907+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:24:20.856936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751480+0000","last_change":"2024-03-08T06:00:36.202917+0000","last_active":"2024-03-08T06:00:42.751480+0000","last_peered":"2024-03-08T06:00:42.751480+0000","last_clean":"2024-03-08T06:00:42.751480+0000","last_became_active":"2024-03-08T06:00:36.202100+0000","last_became_peered":"2024-03-08T06:00:36.202100+0000","last_unstale":"2024-03-08T06:00:42.751480+0000","last_undegraded":"2024-03-08T06:00:42.751480+0000","last_fullsized":"2024-03-08T06:00:42.751480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:31.645981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252205+0000","last_change":"2024-03-08T06:00:35.171650+0000","last_active":"2024-03-08T06:00:43.252205+0000","last_peered":"2024-03-08T06:00:43.252205+0000","last_clean":"2024-03-08T06:00:43.252205+0000","last_became_active":"2024-03-08T06:00:35.171310+0000","last_became_peered":"2024-03-08T06:00:35.171310+0000","last_unstale":"2024-03-08T06:00:43.252205+0000","last_undegraded":"2024-03-08T06:00:43.252205+0000","last_fullsized":"2024-03-08T06:00:43.252205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251524+0000","last_change":"2024-03-08T06:00:40.774574+0000","last_active":"2024-03-08T06:00:43.251524+0000","last_peered":"2024-03-08T06:00:43.251524+0000","last_clean":"2024-03-08T06:00:43.251524+0000","last_became_active":"2024-03-08T06:00:40.774172+0000","last_became_peered":"2024-03-08T06:00:40.774172+0000","last_unstale":"2024-03-08T06:00:43.251524+0000","last_undegraded":"2024-03-08T06:00:43.251524+0000","last_fullsized":"2024-03-08T06:00:43.251524+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:48:26.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752182+0000","last_change":"2024-03-08T06:00:36.195883+0000","last_active":"2024-03-08T06:00:42.752182+0000","last_peered":"2024-03-08T06:00:42.752182+0000","last_clean":"2024-03-08T06:00:42.752182+0000","last_became_active":"2024-03-08T06:00:36.195641+0000","last_became_peered":"2024-03-08T06:00:36.195641+0000","last_unstale":"2024-03-08T06:00:42.752182+0000","last_undegraded":"2024-03-08T06:00:42.752182+0000","last_fullsized":"2024-03-08T06:00:42.752182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:02:15.458747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206956+0000","last_change":"2024-03-08T06:00:35.165484+0000","last_active":"2024-03-08T06:00:42.206956+0000","last_peered":"2024-03-08T06:00:42.206956+0000","last_clean":"2024-03-08T06:00:42.206956+0000","last_became_active":"2024-03-08T06:00:35.165147+0000","last_became_peered":"2024-03-08T06:00:35.165147+0000","last_unstale":"2024-03-08T06:00:42.206956+0000","last_undegraded":"2024-03-08T06:00:42.206956+0000","last_fullsized":"2024-03-08T06:00:42.206956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:07:54.742119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206806+0000","last_change":"2024-03-08T06:00:40.221435+0000","last_active":"2024-03-08T06:00:42.206806+0000","last_peered":"2024-03-08T06:00:42.206806+0000","last_clean":"2024-03-08T06:00:42.206806+0000","last_became_active":"2024-03-08T06:00:40.220886+0000","last_became_peered":"2024-03-08T06:00:40.220886+0000","last_unstale":"2024-03-08T06:00:42.206806+0000","last_undegraded":"2024-03-08T06:00:42.206806+0000","last_fullsized":"2024-03-08T06:00:42.206806+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:24:20.856936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751752+0000","last_change":"2024-03-08T06:00:36.202146+0000","last_active":"2024-03-08T06:00:42.751752+0000","last_peered":"2024-03-08T06:00:42.751752+0000","last_clean":"2024-03-08T06:00:42.751752+0000","last_became_active":"2024-03-08T06:00:36.201880+0000","last_became_peered":"2024-03-08T06:00:36.201880+0000","last_unstale":"2024-03-08T06:00:42.751752+0000","last_undegraded":"2024-03-08T06:00:42.751752+0000","last_fullsized":"2024-03-08T06:00:42.751752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:45:22.429641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252369+0000","last_change":"2024-03-08T06:00:35.168305+0000","last_active":"2024-03-08T06:00:43.252369+0000","last_peered":"2024-03-08T06:00:43.252369+0000","last_clean":"2024-03-08T06:00:43.252369+0000","last_became_active":"2024-03-08T06:00:35.168007+0000","last_became_peered":"2024-03-08T06:00:35.168007+0000","last_unstale":"2024-03-08T06:00:43.252369+0000","last_undegraded":"2024-03-08T06:00:43.252369+0000","last_fullsized":"2024-03-08T06:00:43.252369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:57:55.324372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251733+0000","last_change":"2024-03-08T06:00:40.217477+0000","last_active":"2024-03-08T06:00:43.251733+0000","last_peered":"2024-03-08T06:00:43.251733+0000","last_clean":"2024-03-08T06:00:43.251733+0000","last_became_active":"2024-03-08T06:00:40.217140+0000","last_became_peered":"2024-03-08T06:00:40.217140+0000","last_unstale":"2024-03-08T06:00:43.251733+0000","last_undegraded":"2024-03-08T06:00:43.251733+0000","last_fullsized":"2024-03-08T06:00:43.251733+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:08:55.908866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254784+0000","last_change":"2024-03-08T06:00:36.188881+0000","last_active":"2024-03-08T06:00:43.254784+0000","last_peered":"2024-03-08T06:00:43.254784+0000","last_clean":"2024-03-08T06:00:43.254784+0000","last_became_active":"2024-03-08T06:00:36.188323+0000","last_became_peered":"2024-03-08T06:00:36.188323+0000","last_unstale":"2024-03-08T06:00:43.254784+0000","last_undegraded":"2024-03-08T06:00:43.254784+0000","last_fullsized":"2024-03-08T06:00:43.254784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:11:34.950063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253637+0000","last_change":"2024-03-08T06:00:35.163917+0000","last_active":"2024-03-08T06:00:43.253637+0000","last_peered":"2024-03-08T06:00:43.253637+0000","last_clean":"2024-03-08T06:00:43.253637+0000","last_became_active":"2024-03-08T06:00:35.163650+0000","last_became_peered":"2024-03-08T06:00:35.163650+0000","last_unstale":"2024-03-08T06:00:43.253637+0000","last_undegraded":"2024-03-08T06:00:43.253637+0000","last_fullsized":"2024-03-08T06:00:43.253637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:11.262668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205799+0000","last_change":"2024-03-08T06:00:40.773617+0000","last_active":"2024-03-08T06:00:42.205799+0000","last_peered":"2024-03-08T06:00:42.205799+0000","last_clean":"2024-03-08T06:00:42.205799+0000","last_became_active":"2024-03-08T06:00:40.773144+0000","last_became_peered":"2024-03-08T06:00:40.773144+0000","last_unstale":"2024-03-08T06:00:42.205799+0000","last_undegraded":"2024-03-08T06:00:42.205799+0000","last_fullsized":"2024-03-08T06:00:42.205799+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:32:08.762183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751941+0000","last_change":"2024-03-08T06:00:36.202751+0000","last_active":"2024-03-08T06:00:42.751941+0000","last_peered":"2024-03-08T06:00:42.751941+0000","last_clean":"2024-03-08T06:00:42.751941+0000","last_became_active":"2024-03-08T06:00:36.202228+0000","last_became_peered":"2024-03-08T06:00:36.202228+0000","last_unstale":"2024-03-08T06:00:42.751941+0000","last_undegraded":"2024-03-08T06:00:42.751941+0000","last_fullsized":"2024-03-08T06:00:42.751941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:16:14.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251319+0000","last_change":"2024-03-08T06:00:35.176896+0000","last_active":"2024-03-08T06:00:43.251319+0000","last_peered":"2024-03-08T06:00:43.251319+0000","last_clean":"2024-03-08T06:00:43.251319+0000","last_became_active":"2024-03-08T06:00:35.176665+0000","last_became_peered":"2024-03-08T06:00:35.176665+0000","last_unstale":"2024-03-08T06:00:43.251319+0000","last_undegraded":"2024-03-08T06:00:43.251319+0000","last_fullsized":"2024-03-08T06:00:43.251319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:11.262668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205674+0000","last_change":"2024-03-08T06:00:40.771254+0000","last_active":"2024-03-08T06:00:42.205674+0000","last_peered":"2024-03-08T06:00:42.205674+0000","last_clean":"2024-03-08T06:00:42.205674+0000","last_became_active":"2024-03-08T06:00:40.770828+0000","last_became_peered":"2024-03-08T06:00:40.770828+0000","last_unstale":"2024-03-08T06:00:42.205674+0000","last_undegraded":"2024-03-08T06:00:42.205674+0000","last_fullsized":"2024-03-08T06:00:42.205674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:06:36.831014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251667+0000","last_change":"2024-03-08T06:00:36.192057+0000","last_active":"2024-03-08T06:00:43.251667+0000","last_peered":"2024-03-08T06:00:43.251667+0000","last_clean":"2024-03-08T06:00:43.251667+0000","last_became_active":"2024-03-08T06:00:36.191786+0000","last_became_peered":"2024-03-08T06:00:36.191786+0000","last_unstale":"2024-03-08T06:00:43.251667+0000","last_undegraded":"2024-03-08T06:00:43.251667+0000","last_fullsized":"2024-03-08T06:00:43.251667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:10:17.180499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251277+0000","last_change":"2024-03-08T06:00:35.164201+0000","last_active":"2024-03-08T06:00:43.251277+0000","last_peered":"2024-03-08T06:00:43.251277+0000","last_clean":"2024-03-08T06:00:43.251277+0000","last_became_active":"2024-03-08T06:00:35.163882+0000","last_became_peered":"2024-03-08T06:00:35.163882+0000","last_unstale":"2024-03-08T06:00:43.251277+0000","last_undegraded":"2024-03-08T06:00:43.251277+0000","last_fullsized":"2024-03-08T06:00:43.251277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:43:03.634887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252307+0000","last_change":"2024-03-08T06:00:40.773272+0000","last_active":"2024-03-08T06:00:43.252307+0000","last_peered":"2024-03-08T06:00:43.252307+0000","last_clean":"2024-03-08T06:00:43.252307+0000","last_became_active":"2024-03-08T06:00:40.772833+0000","last_became_peered":"2024-03-08T06:00:40.772833+0000","last_unstale":"2024-03-08T06:00:43.252307+0000","last_undegraded":"2024-03-08T06:00:43.252307+0000","last_fullsized":"2024-03-08T06:00:43.252307+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:53:52.773407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751887+0000","last_change":"2024-03-08T06:00:36.191731+0000","last_active":"2024-03-08T06:00:42.751887+0000","last_peered":"2024-03-08T06:00:42.751887+0000","last_clean":"2024-03-08T06:00:42.751887+0000","last_became_active":"2024-03-08T06:00:36.191469+0000","last_became_peered":"2024-03-08T06:00:36.191469+0000","last_unstale":"2024-03-08T06:00:42.751887+0000","last_undegraded":"2024-03-08T06:00:42.751887+0000","last_fullsized":"2024-03-08T06:00:42.751887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:59.082666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:45.428919+0000","last_change":"2024-03-08T06:00:35.171245+0000","last_active":"2024-03-08T06:00:45.428919+0000","last_peered":"2024-03-08T06:00:45.428919+0000","last_clean":"2024-03-08T06:00:45.428919+0000","last_became_active":"2024-03-08T06:00:35.170999+0000","last_became_peered":"2024-03-08T06:00:35.170999+0000","last_unstale":"2024-03-08T06:00:45.428919+0000","last_undegraded":"2024-03-08T06:00:45.428919+0000","last_fullsized":"2024-03-08T06:00:45.428919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252315+0000","last_change":"2024-03-08T06:00:40.222311+0000","last_active":"2024-03-08T06:00:43.252315+0000","last_peered":"2024-03-08T06:00:43.252315+0000","last_clean":"2024-03-08T06:00:43.252315+0000","last_became_active":"2024-03-08T06:00:40.221923+0000","last_became_peered":"2024-03-08T06:00:40.221923+0000","last_unstale":"2024-03-08T06:00:43.252315+0000","last_undegraded":"2024-03-08T06:00:43.252315+0000","last_fullsized":"2024-03-08T06:00:43.252315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:34:22.301392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206554+0000","last_change":"2024-03-08T06:00:36.190623+0000","last_active":"2024-03-08T06:00:42.206554+0000","last_peered":"2024-03-08T06:00:42.206554+0000","last_clean":"2024-03-08T06:00:42.206554+0000","last_became_active":"2024-03-08T06:00:36.190343+0000","last_became_peered":"2024-03-08T06:00:36.190343+0000","last_unstale":"2024-03-08T06:00:42.206554+0000","last_undegraded":"2024-03-08T06:00:42.206554+0000","last_fullsized":"2024-03-08T06:00:42.206554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:39.370412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252587+0000","last_change":"2024-03-08T06:00:35.169188+0000","last_active":"2024-03-08T06:00:43.252587+0000","last_peered":"2024-03-08T06:00:43.252587+0000","last_clean":"2024-03-08T06:00:43.252587+0000","last_became_active":"2024-03-08T06:00:35.168945+0000","last_became_peered":"2024-03-08T06:00:35.168945+0000","last_unstale":"2024-03-08T06:00:43.252587+0000","last_undegraded":"2024-03-08T06:00:43.252587+0000","last_fullsized":"2024-03-08T06:00:43.252587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:58:33.115927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206843+0000","last_change":"2024-03-08T06:00:40.219289+0000","last_active":"2024-03-08T06:00:42.206843+0000","last_peered":"2024-03-08T06:00:42.206843+0000","last_clean":"2024-03-08T06:00:42.206843+0000","last_became_active":"2024-03-08T06:00:40.218831+0000","last_became_peered":"2024-03-08T06:00:40.218831+0000","last_unstale":"2024-03-08T06:00:42.206843+0000","last_undegraded":"2024-03-08T06:00:42.206843+0000","last_fullsized":"2024-03-08T06:00:42.206843+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:08:55.908866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251809+0000","last_change":"2024-03-08T06:00:36.191492+0000","last_active":"2024-03-08T06:00:43.251809+0000","last_peered":"2024-03-08T06:00:43.251809+0000","last_clean":"2024-03-08T06:00:43.251809+0000","last_became_active":"2024-03-08T06:00:36.190895+0000","last_became_peered":"2024-03-08T06:00:36.190895+0000","last_unstale":"2024-03-08T06:00:43.251809+0000","last_undegraded":"2024-03-08T06:00:43.251809+0000","last_fullsized":"2024-03-08T06:00:43.251809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252270+0000","last_change":"2024-03-08T06:00:35.174075+0000","last_active":"2024-03-08T06:00:43.252270+0000","last_peered":"2024-03-08T06:00:43.252270+0000","last_clean":"2024-03-08T06:00:43.252270+0000","last_became_active":"2024-03-08T06:00:35.173803+0000","last_became_peered":"2024-03-08T06:00:35.173803+0000","last_unstale":"2024-03-08T06:00:43.252270+0000","last_undegraded":"2024-03-08T06:00:43.252270+0000","last_fullsized":"2024-03-08T06:00:43.252270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205087+0000","last_change":"2024-03-08T06:00:40.222466+0000","last_active":"2024-03-08T06:00:42.205087+0000","last_peered":"2024-03-08T06:00:42.205087+0000","last_clean":"2024-03-08T06:00:42.205087+0000","last_became_active":"2024-03-08T06:00:40.222083+0000","last_became_peered":"2024-03-08T06:00:40.222083+0000","last_unstale":"2024-03-08T06:00:42.205087+0000","last_undegraded":"2024-03-08T06:00:42.205087+0000","last_fullsized":"2024-03-08T06:00:42.205087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:21:08.958898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252185+0000","last_change":"2024-03-08T06:00:36.191479+0000","last_active":"2024-03-08T06:00:43.252185+0000","last_peered":"2024-03-08T06:00:43.252185+0000","last_clean":"2024-03-08T06:00:43.252185+0000","last_became_active":"2024-03-08T06:00:36.189667+0000","last_became_peered":"2024-03-08T06:00:36.189667+0000","last_unstale":"2024-03-08T06:00:43.252185+0000","last_undegraded":"2024-03-08T06:00:43.252185+0000","last_fullsized":"2024-03-08T06:00:43.252185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:59.082666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253827+0000","last_change":"2024-03-08T06:00:35.173447+0000","last_active":"2024-03-08T06:00:43.253827+0000","last_peered":"2024-03-08T06:00:43.253827+0000","last_clean":"2024-03-08T06:00:43.253827+0000","last_became_active":"2024-03-08T06:00:35.173050+0000","last_became_peered":"2024-03-08T06:00:35.173050+0000","last_unstale":"2024-03-08T06:00:43.253827+0000","last_undegraded":"2024-03-08T06:00:43.253827+0000","last_fullsized":"2024-03-08T06:00:43.253827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:41.059111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205049+0000","last_change":"2024-03-08T06:00:40.772940+0000","last_active":"2024-03-08T06:00:42.205049+0000","last_peered":"2024-03-08T06:00:42.205049+0000","last_clean":"2024-03-08T06:00:42.205049+0000","last_became_active":"2024-03-08T06:00:40.772593+0000","last_became_peered":"2024-03-08T06:00:40.772593+0000","last_unstale":"2024-03-08T06:00:42.205049+0000","last_undegraded":"2024-03-08T06:00:42.205049+0000","last_fullsized":"2024-03-08T06:00:42.205049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:58:58.974013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252132+0000","last_change":"2024-03-08T06:00:40.773346+0000","last_active":"2024-03-08T06:00:43.252132+0000","last_peered":"2024-03-08T06:00:43.252132+0000","last_clean":"2024-03-08T06:00:43.252132+0000","last_became_active":"2024-03-08T06:00:40.772977+0000","last_became_peered":"2024-03-08T06:00:40.772977+0000","last_unstale":"2024-03-08T06:00:43.252132+0000","last_undegraded":"2024-03-08T06:00:43.252132+0000","last_fullsized":"2024-03-08T06:00:43.252132+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:33.029056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251192+0000","last_change":"2024-03-08T06:00:36.182265+0000","last_active":"2024-03-08T06:00:43.251192+0000","last_peered":"2024-03-08T06:00:43.251192+0000","last_clean":"2024-03-08T06:00:43.251192+0000","last_became_active":"2024-03-08T06:00:36.181732+0000","last_became_peered":"2024-03-08T06:00:36.181732+0000","last_unstale":"2024-03-08T06:00:43.251192+0000","last_undegraded":"2024-03-08T06:00:43.251192+0000","last_fullsized":"2024-03-08T06:00:43.251192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:40:16.053974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251308+0000","last_change":"2024-03-08T06:00:35.173204+0000","last_active":"2024-03-08T06:00:43.251308+0000","last_peered":"2024-03-08T06:00:43.251308+0000","last_clean":"2024-03-08T06:00:43.251308+0000","last_became_active":"2024-03-08T06:00:35.172937+0000","last_became_peered":"2024-03-08T06:00:35.172937+0000","last_unstale":"2024-03-08T06:00:43.251308+0000","last_undegraded":"2024-03-08T06:00:43.251308+0000","last_fullsized":"2024-03-08T06:00:43.251308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254582+0000","last_change":"2024-03-08T06:00:40.223795+0000","last_active":"2024-03-08T06:00:43.254582+0000","last_peered":"2024-03-08T06:00:43.254582+0000","last_clean":"2024-03-08T06:00:43.254582+0000","last_became_active":"2024-03-08T06:00:40.223283+0000","last_became_peered":"2024-03-08T06:00:40.223283+0000","last_unstale":"2024-03-08T06:00:43.254582+0000","last_undegraded":"2024-03-08T06:00:43.254582+0000","last_fullsized":"2024-03-08T06:00:43.254582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:09.499486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205033+0000","last_change":"2024-03-08T06:00:36.185555+0000","last_active":"2024-03-08T06:00:42.205033+0000","last_peered":"2024-03-08T06:00:42.205033+0000","last_clean":"2024-03-08T06:00:42.205033+0000","last_became_active":"2024-03-08T06:00:36.185294+0000","last_became_peered":"2024-03-08T06:00:36.185294+0000","last_unstale":"2024-03-08T06:00:42.205033+0000","last_undegraded":"2024-03-08T06:00:42.205033+0000","last_fullsized":"2024-03-08T06:00:42.205033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:13.003781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205103+0000","last_change":"2024-03-08T06:00:35.171146+0000","last_active":"2024-03-08T06:00:42.205103+0000","last_peered":"2024-03-08T06:00:42.205103+0000","last_clean":"2024-03-08T06:00:42.205103+0000","last_became_active":"2024-03-08T06:00:35.170885+0000","last_became_peered":"2024-03-08T06:00:35.170885+0000","last_unstale":"2024-03-08T06:00:42.205103+0000","last_undegraded":"2024-03-08T06:00:42.205103+0000","last_fullsized":"2024-03-08T06:00:42.205103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:59:03.457310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253633+0000","last_change":"2024-03-08T06:00:35.163289+0000","last_active":"2024-03-08T06:00:43.253633+0000","last_peered":"2024-03-08T06:00:43.253633+0000","last_clean":"2024-03-08T06:00:43.253633+0000","last_became_active":"2024-03-08T06:00:35.163005+0000","last_became_peered":"2024-03-08T06:00:35.163005+0000","last_unstale":"2024-03-08T06:00:43.253633+0000","last_undegraded":"2024-03-08T06:00:43.253633+0000","last_fullsized":"2024-03-08T06:00:43.253633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:38.366092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252056+0000","last_change":"2024-03-08T06:00:36.194840+0000","last_active":"2024-03-08T06:00:43.252056+0000","last_peered":"2024-03-08T06:00:43.252056+0000","last_clean":"2024-03-08T06:00:43.252056+0000","last_became_active":"2024-03-08T06:00:36.194547+0000","last_became_peered":"2024-03-08T06:00:36.194547+0000","last_unstale":"2024-03-08T06:00:43.252056+0000","last_undegraded":"2024-03-08T06:00:43.252056+0000","last_fullsized":"2024-03-08T06:00:43.252056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:11:34.950063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253014+0000","last_change":"2024-03-08T06:00:40.219835+0000","last_active":"2024-03-08T06:00:43.253014+0000","last_peered":"2024-03-08T06:00:43.253014+0000","last_clean":"2024-03-08T06:00:43.253014+0000","last_became_active":"2024-03-08T06:00:40.219483+0000","last_became_peered":"2024-03-08T06:00:40.219483+0000","last_unstale":"2024-03-08T06:00:43.253014+0000","last_undegraded":"2024-03-08T06:00:43.253014+0000","last_fullsized":"2024-03-08T06:00:43.253014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:41:06.159304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205357+0000","last_change":"2024-03-08T06:00:35.171304+0000","last_active":"2024-03-08T06:00:42.205357+0000","last_peered":"2024-03-08T06:00:42.205357+0000","last_clean":"2024-03-08T06:00:42.205357+0000","last_became_active":"2024-03-08T06:00:35.170979+0000","last_became_peered":"2024-03-08T06:00:35.170979+0000","last_unstale":"2024-03-08T06:00:42.205357+0000","last_undegraded":"2024-03-08T06:00:42.205357+0000","last_fullsized":"2024-03-08T06:00:42.205357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:41.059111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206410+0000","last_change":"2024-03-08T06:00:36.179352+0000","last_active":"2024-03-08T06:00:42.206410+0000","last_peered":"2024-03-08T06:00:42.206410+0000","last_clean":"2024-03-08T06:00:42.206410+0000","last_became_active":"2024-03-08T06:00:36.179123+0000","last_became_peered":"2024-03-08T06:00:36.179123+0000","last_unstale":"2024-03-08T06:00:42.206410+0000","last_undegraded":"2024-03-08T06:00:42.206410+0000","last_fullsized":"2024-03-08T06:00:42.206410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:12.266989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205576+0000","last_change":"2024-03-08T06:00:40.220569+0000","last_active":"2024-03-08T06:00:42.205576+0000","last_peered":"2024-03-08T06:00:42.205576+0000","last_clean":"2024-03-08T06:00:42.205576+0000","last_became_active":"2024-03-08T06:00:40.220153+0000","last_became_peered":"2024-03-08T06:00:40.220153+0000","last_unstale":"2024-03-08T06:00:42.205576+0000","last_undegraded":"2024-03-08T06:00:42.205576+0000","last_fullsized":"2024-03-08T06:00:42.205576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:44:28.955585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254222+0000","last_change":"2024-03-08T06:00:36.189112+0000","last_active":"2024-03-08T06:00:43.254222+0000","last_peered":"2024-03-08T06:00:43.254222+0000","last_clean":"2024-03-08T06:00:43.254222+0000","last_became_active":"2024-03-08T06:00:36.188650+0000","last_became_peered":"2024-03-08T06:00:36.188650+0000","last_unstale":"2024-03-08T06:00:43.254222+0000","last_undegraded":"2024-03-08T06:00:43.254222+0000","last_fullsized":"2024-03-08T06:00:43.254222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:13.003781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252434+0000","last_change":"2024-03-08T06:00:36.188109+0000","last_active":"2024-03-08T06:00:43.252434+0000","last_peered":"2024-03-08T06:00:43.252434+0000","last_clean":"2024-03-08T06:00:43.252434+0000","last_became_active":"2024-03-08T06:00:36.187798+0000","last_became_peered":"2024-03-08T06:00:36.187798+0000","last_unstale":"2024-03-08T06:00:43.252434+0000","last_undegraded":"2024-03-08T06:00:43.252434+0000","last_fullsized":"2024-03-08T06:00:43.252434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:30:07.063230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251880+0000","last_change":"2024-03-08T06:00:36.193357+0000","last_active":"2024-03-08T06:00:43.251880+0000","last_peered":"2024-03-08T06:00:43.251880+0000","last_clean":"2024-03-08T06:00:43.251880+0000","last_became_active":"2024-03-08T06:00:36.191586+0000","last_became_peered":"2024-03-08T06:00:36.191586+0000","last_unstale":"2024-03-08T06:00:43.251880+0000","last_undegraded":"2024-03-08T06:00:43.251880+0000","last_fullsized":"2024-03-08T06:00:43.251880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:44:44.951176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251957+0000","last_change":"2024-03-08T06:00:36.186266+0000","last_active":"2024-03-08T06:00:43.251957+0000","last_peered":"2024-03-08T06:00:43.251957+0000","last_clean":"2024-03-08T06:00:43.251957+0000","last_became_active":"2024-03-08T06:00:36.185663+0000","last_became_peered":"2024-03-08T06:00:36.185663+0000","last_unstale":"2024-03-08T06:00:43.251957+0000","last_undegraded":"2024-03-08T06:00:43.251957+0000","last_fullsized":"2024-03-08T06:00:43.251957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:53:20.034194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205842+0000","last_change":"2024-03-08T06:00:36.192645+0000","last_active":"2024-03-08T06:00:42.205842+0000","last_peered":"2024-03-08T06:00:42.205842+0000","last_clean":"2024-03-08T06:00:42.205842+0000","last_became_active":"2024-03-08T06:00:36.192391+0000","last_became_peered":"2024-03-08T06:00:36.192391+0000","last_unstale":"2024-03-08T06:00:42.205842+0000","last_undegraded":"2024-03-08T06:00:42.205842+0000","last_fullsized":"2024-03-08T06:00:42.205842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:45:22.429641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252413+0000","last_change":"2024-03-08T06:00:36.181871+0000","last_active":"2024-03-08T06:00:43.252413+0000","last_peered":"2024-03-08T06:00:43.252413+0000","last_clean":"2024-03-08T06:00:43.252413+0000","last_became_active":"2024-03-08T06:00:36.181557+0000","last_became_peered":"2024-03-08T06:00:36.181557+0000","last_unstale":"2024-03-08T06:00:43.252413+0000","last_undegraded":"2024-03-08T06:00:43.252413+0000","last_fullsized":"2024-03-08T06:00:43.252413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:10:17.180499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254922+0000","last_change":"2024-03-08T06:00:36.185046+0000","last_active":"2024-03-08T06:00:43.254922+0000","last_peered":"2024-03-08T06:00:43.254922+0000","last_clean":"2024-03-08T06:00:43.254922+0000","last_became_active":"2024-03-08T06:00:36.184066+0000","last_became_peered":"2024-03-08T06:00:36.184066+0000","last_unstale":"2024-03-08T06:00:43.254922+0000","last_undegraded":"2024-03-08T06:00:43.254922+0000","last_fullsized":"2024-03-08T06:00:43.254922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205846+0000","last_change":"2024-03-08T06:00:36.186595+0000","last_active":"2024-03-08T06:00:42.205846+0000","last_peered":"2024-03-08T06:00:42.205846+0000","last_clean":"2024-03-08T06:00:42.205846+0000","last_became_active":"2024-03-08T06:00:36.186261+0000","last_became_peered":"2024-03-08T06:00:36.186261+0000","last_unstale":"2024-03-08T06:00:42.205846+0000","last_undegraded":"2024-03-08T06:00:42.205846+0000","last_fullsized":"2024-03-08T06:00:42.205846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:16:14.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251609+0000","last_change":"2024-03-08T06:00:36.186618+0000","last_active":"2024-03-08T06:00:43.251609+0000","last_peered":"2024-03-08T06:00:43.251609+0000","last_clean":"2024-03-08T06:00:43.251609+0000","last_became_active":"2024-03-08T06:00:36.186068+0000","last_became_peered":"2024-03-08T06:00:36.186068+0000","last_unstale":"2024-03-08T06:00:43.251609+0000","last_undegraded":"2024-03-08T06:00:43.251609+0000","last_fullsized":"2024-03-08T06:00:43.251609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:39:59.986300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205158+0000","last_change":"2024-03-08T06:00:36.186759+0000","last_active":"2024-03-08T06:00:42.205158+0000","last_peered":"2024-03-08T06:00:42.205158+0000","last_clean":"2024-03-08T06:00:42.205158+0000","last_became_active":"2024-03-08T06:00:36.186338+0000","last_became_peered":"2024-03-08T06:00:36.186338+0000","last_unstale":"2024-03-08T06:00:42.205158+0000","last_undegraded":"2024-03-08T06:00:42.205158+0000","last_fullsized":"2024-03-08T06:00:42.205158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:40:16.053974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251835+0000","last_change":"2024-03-08T06:00:36.197002+0000","last_active":"2024-03-08T06:00:43.251835+0000","last_peered":"2024-03-08T06:00:43.251835+0000","last_clean":"2024-03-08T06:00:43.251835+0000","last_became_active":"2024-03-08T06:00:36.196750+0000","last_became_peered":"2024-03-08T06:00:36.196750+0000","last_unstale":"2024-03-08T06:00:43.251835+0000","last_undegraded":"2024-03-08T06:00:43.251835+0000","last_fullsized":"2024-03-08T06:00:43.251835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:45:22.429641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251708+0000","last_change":"2024-03-08T06:00:36.184956+0000","last_active":"2024-03-08T06:00:43.251708+0000","last_peered":"2024-03-08T06:00:43.251708+0000","last_clean":"2024-03-08T06:00:43.251708+0000","last_became_active":"2024-03-08T06:00:36.184694+0000","last_became_peered":"2024-03-08T06:00:36.184694+0000","last_unstale":"2024-03-08T06:00:43.251708+0000","last_undegraded":"2024-03-08T06:00:43.251708+0000","last_fullsized":"2024-03-08T06:00:43.251708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:41:02.118287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254526+0000","last_change":"2024-03-08T06:00:36.187667+0000","last_active":"2024-03-08T06:00:43.254526+0000","last_peered":"2024-03-08T06:00:43.254526+0000","last_clean":"2024-03-08T06:00:43.254526+0000","last_became_active":"2024-03-08T06:00:36.187407+0000","last_became_peered":"2024-03-08T06:00:36.187407+0000","last_unstale":"2024-03-08T06:00:43.254526+0000","last_undegraded":"2024-03-08T06:00:43.254526+0000","last_fullsized":"2024-03-08T06:00:43.254526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:10:17.180499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752121+0000","last_change":"2024-03-08T06:00:36.198182+0000","last_active":"2024-03-08T06:00:42.752121+0000","last_peered":"2024-03-08T06:00:42.752121+0000","last_clean":"2024-03-08T06:00:42.752121+0000","last_became_active":"2024-03-08T06:00:36.197894+0000","last_became_peered":"2024-03-08T06:00:36.197894+0000","last_unstale":"2024-03-08T06:00:42.752121+0000","last_undegraded":"2024-03-08T06:00:42.752121+0000","last_fullsized":"2024-03-08T06:00:42.752121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:02.331109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253403+0000","last_change":"2024-03-08T06:00:36.180664+0000","last_active":"2024-03-08T06:00:43.253403+0000","last_peered":"2024-03-08T06:00:43.253403+0000","last_clean":"2024-03-08T06:00:43.253403+0000","last_became_active":"2024-03-08T06:00:36.180389+0000","last_became_peered":"2024-03-08T06:00:36.180389+0000","last_unstale":"2024-03-08T06:00:43.253403+0000","last_undegraded":"2024-03-08T06:00:43.253403+0000","last_fullsized":"2024-03-08T06:00:43.253403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:58:34.120247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252430+0000","last_change":"2024-03-08T06:00:36.189412+0000","last_active":"2024-03-08T06:00:43.252430+0000","last_peered":"2024-03-08T06:00:43.252430+0000","last_clean":"2024-03-08T06:00:43.252430+0000","last_became_active":"2024-03-08T06:00:36.187130+0000","last_became_peered":"2024-03-08T06:00:36.187130+0000","last_unstale":"2024-03-08T06:00:43.252430+0000","last_undegraded":"2024-03-08T06:00:43.252430+0000","last_fullsized":"2024-03-08T06:00:43.252430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:05.458469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251511+0000","last_change":"2024-03-08T06:00:36.186949+0000","last_active":"2024-03-08T06:00:43.251511+0000","last_peered":"2024-03-08T06:00:43.251511+0000","last_clean":"2024-03-08T06:00:43.251511+0000","last_became_active":"2024-03-08T06:00:36.186386+0000","last_became_peered":"2024-03-08T06:00:36.186386+0000","last_unstale":"2024-03-08T06:00:43.251511+0000","last_undegraded":"2024-03-08T06:00:43.251511+0000","last_fullsized":"2024-03-08T06:00:43.251511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251280+0000","last_change":"2024-03-08T06:00:36.178773+0000","last_active":"2024-03-08T06:00:43.251280+0000","last_peered":"2024-03-08T06:00:43.251280+0000","last_clean":"2024-03-08T06:00:43.251280+0000","last_became_active":"2024-03-08T06:00:36.178515+0000","last_became_peered":"2024-03-08T06:00:36.178515+0000","last_unstale":"2024-03-08T06:00:43.251280+0000","last_undegraded":"2024-03-08T06:00:43.251280+0000","last_fullsized":"2024-03-08T06:00:43.251280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:11:34.950063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251635+0000","last_change":"2024-03-08T06:00:36.179334+0000","last_active":"2024-03-08T06:00:43.251635+0000","last_peered":"2024-03-08T06:00:43.251635+0000","last_clean":"2024-03-08T06:00:43.251635+0000","last_became_active":"2024-03-08T06:00:36.179056+0000","last_became_peered":"2024-03-08T06:00:36.179056+0000","last_unstale":"2024-03-08T06:00:43.251635+0000","last_undegraded":"2024-03-08T06:00:43.251635+0000","last_fullsized":"2024-03-08T06:00:43.251635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:13.003781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205268+0000","last_change":"2024-03-08T06:00:36.182864+0000","last_active":"2024-03-08T06:00:42.205268+0000","last_peered":"2024-03-08T06:00:42.205268+0000","last_clean":"2024-03-08T06:00:42.205268+0000","last_became_active":"2024-03-08T06:00:36.182290+0000","last_became_peered":"2024-03-08T06:00:36.182290+0000","last_unstale":"2024-03-08T06:00:42.205268+0000","last_undegraded":"2024-03-08T06:00:42.205268+0000","last_fullsized":"2024-03-08T06:00:42.205268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206446+0000","last_change":"2024-03-08T06:00:36.184044+0000","last_active":"2024-03-08T06:00:42.206446+0000","last_peered":"2024-03-08T06:00:42.206446+0000","last_clean":"2024-03-08T06:00:42.206446+0000","last_became_active":"2024-03-08T06:00:36.183745+0000","last_became_peered":"2024-03-08T06:00:36.183745+0000","last_unstale":"2024-03-08T06:00:42.206446+0000","last_undegraded":"2024-03-08T06:00:42.206446+0000","last_fullsized":"2024-03-08T06:00:42.206446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:58:34.120247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254652+0000","last_change":"2024-03-08T06:00:36.200379+0000","last_active":"2024-03-08T06:00:43.254652+0000","last_peered":"2024-03-08T06:00:43.254652+0000","last_clean":"2024-03-08T06:00:43.254652+0000","last_became_active":"2024-03-08T06:00:36.200023+0000","last_became_peered":"2024-03-08T06:00:36.200023+0000","last_unstale":"2024-03-08T06:00:43.254652+0000","last_undegraded":"2024-03-08T06:00:43.254652+0000","last_fullsized":"2024-03-08T06:00:43.254652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:40:16.053974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254844+0000","last_change":"2024-03-08T06:00:36.200329+0000","last_active":"2024-03-08T06:00:43.254844+0000","last_peered":"2024-03-08T06:00:43.254844+0000","last_clean":"2024-03-08T06:00:43.254844+0000","last_became_active":"2024-03-08T06:00:36.200069+0000","last_became_peered":"2024-03-08T06:00:36.200069+0000","last_unstale":"2024-03-08T06:00:43.254844+0000","last_undegraded":"2024-03-08T06:00:43.254844+0000","last_fullsized":"2024-03-08T06:00:43.254844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751472+0000","last_change":"2024-03-08T06:00:36.205202+0000","last_active":"2024-03-08T06:00:42.751472+0000","last_peered":"2024-03-08T06:00:42.751472+0000","last_clean":"2024-03-08T06:00:42.751472+0000","last_became_active":"2024-03-08T06:00:36.204866+0000","last_became_peered":"2024-03-08T06:00:36.204866+0000","last_unstale":"2024-03-08T06:00:42.751472+0000","last_undegraded":"2024-03-08T06:00:42.751472+0000","last_fullsized":"2024-03-08T06:00:42.751472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:44:44.951176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.204931+0000","last_change":"2024-03-08T06:00:36.188976+0000","last_active":"2024-03-08T06:00:42.204931+0000","last_peered":"2024-03-08T06:00:42.204931+0000","last_clean":"2024-03-08T06:00:42.204931+0000","last_became_active":"2024-03-08T06:00:36.188717+0000","last_became_peered":"2024-03-08T06:00:36.188717+0000","last_unstale":"2024-03-08T06:00:42.204931+0000","last_undegraded":"2024-03-08T06:00:42.204931+0000","last_fullsized":"2024-03-08T06:00:42.204931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:11:34.950063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751727+0000","last_change":"2024-03-08T06:00:36.197547+0000","last_active":"2024-03-08T06:00:42.751727+0000","last_peered":"2024-03-08T06:00:42.751727+0000","last_clean":"2024-03-08T06:00:42.751727+0000","last_became_active":"2024-03-08T06:00:36.197295+0000","last_became_peered":"2024-03-08T06:00:36.197295+0000","last_unstale":"2024-03-08T06:00:42.751727+0000","last_undegraded":"2024-03-08T06:00:42.751727+0000","last_fullsized":"2024-03-08T06:00:42.751727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:39:59.986300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251773+0000","last_change":"2024-03-08T06:00:36.191642+0000","last_active":"2024-03-08T06:00:43.251773+0000","last_peered":"2024-03-08T06:00:43.251773+0000","last_clean":"2024-03-08T06:00:43.251773+0000","last_became_active":"2024-03-08T06:00:36.191242+0000","last_became_peered":"2024-03-08T06:00:36.191242+0000","last_unstale":"2024-03-08T06:00:43.251773+0000","last_undegraded":"2024-03-08T06:00:43.251773+0000","last_fullsized":"2024-03-08T06:00:43.251773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751820+0000","last_change":"2024-03-08T06:00:36.197506+0000","last_active":"2024-03-08T06:00:42.751820+0000","last_peered":"2024-03-08T06:00:42.751820+0000","last_clean":"2024-03-08T06:00:42.751820+0000","last_became_active":"2024-03-08T06:00:36.197208+0000","last_became_peered":"2024-03-08T06:00:36.197208+0000","last_unstale":"2024-03-08T06:00:42.751820+0000","last_undegraded":"2024-03-08T06:00:42.751820+0000","last_fullsized":"2024-03-08T06:00:42.751820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:59:39.518589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252005+0000","last_change":"2024-03-08T06:00:36.186669+0000","last_active":"2024-03-08T06:00:43.252005+0000","last_peered":"2024-03-08T06:00:43.252005+0000","last_clean":"2024-03-08T06:00:43.252005+0000","last_became_active":"2024-03-08T06:00:36.186417+0000","last_became_peered":"2024-03-08T06:00:36.186417+0000","last_unstale":"2024-03-08T06:00:43.252005+0000","last_undegraded":"2024-03-08T06:00:43.252005+0000","last_fullsized":"2024-03-08T06:00:43.252005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:59:39.518589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251432+0000","last_change":"2024-03-08T06:00:36.186124+0000","last_active":"2024-03-08T06:00:43.251432+0000","last_peered":"2024-03-08T06:00:43.251432+0000","last_clean":"2024-03-08T06:00:43.251432+0000","last_became_active":"2024-03-08T06:00:36.185808+0000","last_became_peered":"2024-03-08T06:00:36.185808+0000","last_unstale":"2024-03-08T06:00:43.251432+0000","last_undegraded":"2024-03-08T06:00:43.251432+0000","last_fullsized":"2024-03-08T06:00:43.251432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:01:37.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252630+0000","last_change":"2024-03-08T06:00:36.183150+0000","last_active":"2024-03-08T06:00:43.252630+0000","last_peered":"2024-03-08T06:00:43.252630+0000","last_clean":"2024-03-08T06:00:43.252630+0000","last_became_active":"2024-03-08T06:00:36.182716+0000","last_became_peered":"2024-03-08T06:00:36.182716+0000","last_unstale":"2024-03-08T06:00:43.252630+0000","last_undegraded":"2024-03-08T06:00:43.252630+0000","last_fullsized":"2024-03-08T06:00:43.252630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752346+0000","last_change":"2024-03-08T06:00:36.198478+0000","last_active":"2024-03-08T06:00:42.752346+0000","last_peered":"2024-03-08T06:00:42.752346+0000","last_clean":"2024-03-08T06:00:42.752346+0000","last_became_active":"2024-03-08T06:00:36.198218+0000","last_became_peered":"2024-03-08T06:00:36.198218+0000","last_unstale":"2024-03-08T06:00:42.752346+0000","last_undegraded":"2024-03-08T06:00:42.752346+0000","last_fullsized":"2024-03-08T06:00:42.752346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:19:26.058130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":876544,"data_stored":703400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":116292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":112540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":112576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":116292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":112450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":112610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":703456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:01:03.516 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:01:03.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.651+0000 7f626c9bc640 1 Processor -- start 2024-03-08T06:01:03.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.652+0000 7f626c9bc640 1 -- start start 2024-03-08T06:01:03.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.652+0000 7f626c9bc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f626815dc60 0x7f62680d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:03.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.652+0000 7f626c9bc640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f626815e040 0x7f6268099b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:03.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.652+0000 7f626c9bc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62680cfcc0 con 0x7f626806bfa0 2024-03-08T06:01:03.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.652+0000 7f626c9bc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62680b4360 con 0x7f626815dc60 2024-03-08T06:01:03.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.652+0000 7f626c9bc640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62680cff60 con 0x7f626815e040 2024-03-08T06:01:03.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.653+0000 7f626659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f626815dc60 0x7f62680d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:03.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.653+0000 7f6266d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f626806bfa0 0x7f62680cbfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57624/0 (socket says 172.21.15.76:57624) 2024-03-08T06:01:03.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.653+0000 7f6266d9d640 1 -- 172.21.15.76:0/1191880800 learned_addr learned my addr 172.21.15.76:0/1191880800 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:03.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.653+0000 7f625ffff640 1 --2- 172.21.15.76:0/1191880800 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f626815e040 0x7f6268099b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:03.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.653+0000 7f626659c640 1 -- 172.21.15.76:0/1191880800 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f626815e040 msgr2=0x7f6268099b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:03.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.653+0000 7f626659c640 1 --2- 172.21.15.76:0/1191880800 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f626815e040 0x7f6268099b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.653+0000 7f626659c640 1 -- 172.21.15.76:0/1191880800 >> v1:172.21.15.76:6789/0 conn(0x7f626806bfa0 legacy=0x7f62680cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.653+0000 7f626659c640 1 -- 172.21.15.76:0/1191880800 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62680d0240 con 0x7f626815dc60 2024-03-08T06:01:03.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.653+0000 7f6265d9b640 1 -- 172.21.15.76:0/1191880800 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292059236 0 0) 0x7f62680cfcc0 con 0x7f626806bfa0 2024-03-08T06:01:03.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.653+0000 7f625ffff640 1 --2- 172.21.15.76:0/1191880800 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f626815e040 0x7f6268099b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:03.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.654+0000 7f626659c640 1 --2- 172.21.15.76:0/1191880800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f626815dc60 0x7f62680d1000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f62580097a0 tx=0x7f625802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330e805bade0a49e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:03.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.654+0000 7f6265d9b640 1 -- 172.21.15.76:0/1191880800 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258034070 con 0x7f626815dc60 2024-03-08T06:01:03.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.654+0000 7f6265d9b640 1 -- 172.21.15.76:0/1191880800 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6258030610 con 0x7f626815dc60 2024-03-08T06:01:03.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.654+0000 7f6265d9b640 1 -- 172.21.15.76:0/1191880800 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62580309e0 con 0x7f626815dc60 2024-03-08T06:01:03.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.654+0000 7f626c9bc640 1 -- 172.21.15.76:0/1191880800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f626815dc60 msgr2=0x7f62680d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.654+0000 7f626c9bc640 1 --2- 172.21.15.76:0/1191880800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f626815dc60 0x7f62680d1000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f62580097a0 tx=0x7f625802b960 comp rx=0 tx=0).stop 2024-03-08T06:01:03.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.654+0000 7f626c9bc640 1 -- 172.21.15.76:0/1191880800 shutdown_connections 2024-03-08T06:01:03.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.654+0000 7f626c9bc640 1 --2- 172.21.15.76:0/1191880800 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f626815e040 0x7f6268099b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.654+0000 7f626c9bc640 1 --2- 172.21.15.76:0/1191880800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f626815dc60 0x7f62680d1000 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.654+0000 7f626c9bc640 1 -- 172.21.15.76:0/1191880800 >> 172.21.15.76:0/1191880800 conn(0x7f62680b3440 msgr2=0x7f62680b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:03.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.654+0000 7f626c9bc640 1 -- 172.21.15.76:0/1191880800 shutdown_connections 2024-03-08T06:01:03.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.654+0000 7f626c9bc640 1 -- 172.21.15.76:0/1191880800 wait complete. 2024-03-08T06:01:03.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.656+0000 7f626c9bc640 1 Processor -- start 2024-03-08T06:01:03.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.656+0000 7f626c9bc640 1 -- start start 2024-03-08T06:01:03.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.656+0000 7f626c9bc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f626806bfa0 0x7f626814dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:03.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.656+0000 7f626c9bc640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f626815dc60 0x7f62680c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:03.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.656+0000 7f626c9bc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f626815e040 0x7f62680af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:03.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.656+0000 7f626c9bc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62680d0ae0 con 0x7f626806bfa0 2024-03-08T06:01:03.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.656+0000 7f626c9bc640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62680d0800 con 0x7f626815e040 2024-03-08T06:01:03.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.656+0000 7f626c9bc640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62680d1540 con 0x7f626815dc60 2024-03-08T06:01:03.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.656+0000 7f626659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f626806bfa0 0x7f626814dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:03.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.656+0000 7f626659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f626806bfa0 0x7f626814dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51672/0 (socket says 172.21.15.76:51672) 2024-03-08T06:01:03.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.656+0000 7f626659c640 1 -- 172.21.15.76:0/1364206151 learned_addr learned my addr 172.21.15.76:0/1364206151 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:03.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.657+0000 7f626659c640 1 -- 172.21.15.76:0/1364206151 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f626815dc60 msgr2=0x7f62680c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.657+0000 7f626659c640 1 --2- 172.21.15.76:0/1364206151 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f626815dc60 0x7f62680c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.657+0000 7f626659c640 1 -- 172.21.15.76:0/1364206151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f626815e040 msgr2=0x7f62680af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.657+0000 7f626659c640 1 --2- 172.21.15.76:0/1364206151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f626815e040 0x7f62680af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.657+0000 7f626659c640 1 -- 172.21.15.76:0/1364206151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626814e260 con 0x7f626806bfa0 2024-03-08T06:01:03.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.657+0000 7f626659c640 1 --2- 172.21.15.76:0/1364206151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f626806bfa0 0x7f626814dd20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6258009890 tx=0x7f62580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:03.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.657+0000 7f625effd640 1 -- 172.21.15.76:0/1364206151 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258034040 con 0x7f626806bfa0 2024-03-08T06:01:03.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.657+0000 7f625effd640 1 -- 172.21.15.76:0/1364206151 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6258039a10 con 0x7f626806bfa0 2024-03-08T06:01:03.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.657+0000 7f626c9bc640 1 -- 172.21.15.76:0/1364206151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62680cd550 con 0x7f626806bfa0 2024-03-08T06:01:03.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.657+0000 7f625effd640 1 -- 172.21.15.76:0/1364206151 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258039de0 con 0x7f626806bfa0 2024-03-08T06:01:03.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.657+0000 7f626c9bc640 1 -- 172.21.15.76:0/1364206151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62680cdb10 con 0x7f626806bfa0 2024-03-08T06:01:03.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.658+0000 7f626c9bc640 1 -- 172.21.15.76:0/1364206151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626809a3d0 con 0x7f626806bfa0 2024-03-08T06:01:03.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.658+0000 7f625effd640 1 -- 172.21.15.76:0/1364206151 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6258037070 con 0x7f626806bfa0 2024-03-08T06:01:03.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.659+0000 7f625effd640 1 --2- 172.21.15.76:0/1364206151 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f624806b4d0 0x7f624806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:03.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.659+0000 7f625effd640 1 -- 172.21.15.76:0/1364206151 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f62580ac5b0 con 0x7f626806bfa0 2024-03-08T06:01:03.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.659+0000 7f625ffff640 1 --2- 172.21.15.76:0/1364206151 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f624806b4d0 0x7f624806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:03.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.659+0000 7f625ffff640 1 --2- 172.21.15.76:0/1364206151 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f624806b4d0 0x7f624806d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6250006fd0 tx=0x7f6250008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:03.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.663+0000 7f625effd640 1 -- 172.21.15.76:0/1364206151 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f626809a3d0 con 0x7f626806bfa0 2024-03-08T06:01:03.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.831+0000 7f626c9bc640 1 -- 172.21.15.76:0/1364206151 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f62680ce5f0 con 0x7f624806b4d0 2024-03-08T06:01:03.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.838+0000 7f625effd640 1 -- 172.21.15.76:0/1364206151 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332533 (secure 0 0 0) 0x7f625000c7a0 con 0x7f624806b4d0 2024-03-08T06:01:03.842 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:03.844 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-08T06:01:03.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.841+0000 7f626c9bc640 1 -- 172.21.15.76:0/1364206151 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f624806b4d0 msgr2=0x7f624806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.841+0000 7f626c9bc640 1 --2- 172.21.15.76:0/1364206151 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f624806b4d0 0x7f624806d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6250006fd0 tx=0x7f6250008040 comp rx=0 tx=0).stop 2024-03-08T06:01:03.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.841+0000 7f626c9bc640 1 -- 172.21.15.76:0/1364206151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f626806bfa0 msgr2=0x7f626814dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.841+0000 7f626c9bc640 1 --2- 172.21.15.76:0/1364206151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f626806bfa0 0x7f626814dd20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6258009890 tx=0x7f62580046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.841+0000 7f626c9bc640 1 -- 172.21.15.76:0/1364206151 shutdown_connections 2024-03-08T06:01:03.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.841+0000 7f626c9bc640 1 --2- 172.21.15.76:0/1364206151 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f624806b4d0 0x7f624806d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.841+0000 7f626c9bc640 1 --2- 172.21.15.76:0/1364206151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f626815e040 0x7f62680af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.841+0000 7f626c9bc640 1 --2- 172.21.15.76:0/1364206151 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f626815dc60 0x7f62680c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.841+0000 7f626c9bc640 1 --2- 172.21.15.76:0/1364206151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f626806bfa0 0x7f626814dd20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.841+0000 7f626c9bc640 1 -- 172.21.15.76:0/1364206151 >> 172.21.15.76:0/1364206151 conn(0x7f62680b3440 msgr2=0x7f626806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:03.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.842+0000 7f626c9bc640 1 -- 172.21.15.76:0/1364206151 shutdown_connections 2024-03-08T06:01:03.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.842+0000 7f626c9bc640 1 -- 172.21.15.76:0/1364206151 wait complete. 2024-03-08T06:01:03.853 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-03-08T06:01:01.925942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219560,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755160,"statfs":{"total":773094113280,"available":772869283840,"internally_reserved":0,"allocated":3481600,"data_stored":2089616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003162"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.250953+0000","last_change":"2024-03-08T06:00:35.165081+0000","last_active":"2024-03-08T06:00:43.250953+0000","last_peered":"2024-03-08T06:00:43.250953+0000","last_clean":"2024-03-08T06:00:43.250953+0000","last_became_active":"2024-03-08T06:00:35.164818+0000","last_became_peered":"2024-03-08T06:00:35.164818+0000","last_unstale":"2024-03-08T06:00:43.250953+0000","last_undegraded":"2024-03-08T06:00:43.250953+0000","last_fullsized":"2024-03-08T06:00:43.250953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:59:03.457310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205755+0000","last_change":"2024-03-08T06:00:36.182716+0000","last_active":"2024-03-08T06:00:42.205755+0000","last_peered":"2024-03-08T06:00:42.205755+0000","last_clean":"2024-03-08T06:00:42.205755+0000","last_became_active":"2024-03-08T06:00:36.182377+0000","last_became_peered":"2024-03-08T06:00:36.182377+0000","last_unstale":"2024-03-08T06:00:42.205755+0000","last_undegraded":"2024-03-08T06:00:42.205755+0000","last_fullsized":"2024-03-08T06:00:42.205755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:59:39.518589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.204982+0000","last_change":"2024-03-08T06:00:40.773175+0000","last_active":"2024-03-08T06:00:42.204982+0000","last_peered":"2024-03-08T06:00:42.204982+0000","last_clean":"2024-03-08T06:00:42.204982+0000","last_became_active":"2024-03-08T06:00:40.772777+0000","last_became_peered":"2024-03-08T06:00:40.772777+0000","last_unstale":"2024-03-08T06:00:42.204982+0000","last_undegraded":"2024-03-08T06:00:42.204982+0000","last_fullsized":"2024-03-08T06:00:42.204982+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:51:44.068190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205870+0000","last_change":"2024-03-08T06:00:35.177115+0000","last_active":"2024-03-08T06:00:42.205870+0000","last_peered":"2024-03-08T06:00:42.205870+0000","last_clean":"2024-03-08T06:00:42.205870+0000","last_became_active":"2024-03-08T06:00:35.176827+0000","last_became_peered":"2024-03-08T06:00:35.176827+0000","last_unstale":"2024-03-08T06:00:42.205870+0000","last_undegraded":"2024-03-08T06:00:42.205870+0000","last_fullsized":"2024-03-08T06:00:42.205870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:57:55.324372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251714+0000","last_change":"2024-03-08T06:00:36.191516+0000","last_active":"2024-03-08T06:00:43.251714+0000","last_peered":"2024-03-08T06:00:43.251714+0000","last_clean":"2024-03-08T06:00:43.251714+0000","last_became_active":"2024-03-08T06:00:36.191258+0000","last_became_peered":"2024-03-08T06:00:36.191258+0000","last_unstale":"2024-03-08T06:00:43.251714+0000","last_undegraded":"2024-03-08T06:00:43.251714+0000","last_fullsized":"2024-03-08T06:00:43.251714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206166+0000","last_change":"2024-03-08T06:00:40.219312+0000","last_active":"2024-03-08T06:00:42.206166+0000","last_peered":"2024-03-08T06:00:42.206166+0000","last_clean":"2024-03-08T06:00:42.206166+0000","last_became_active":"2024-03-08T06:00:40.218891+0000","last_became_peered":"2024-03-08T06:00:40.218891+0000","last_unstale":"2024-03-08T06:00:42.206166+0000","last_undegraded":"2024-03-08T06:00:42.206166+0000","last_fullsized":"2024-03-08T06:00:42.206166+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:19:30.099147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752093+0000","last_change":"2024-03-08T06:00:35.163380+0000","last_active":"2024-03-08T06:00:42.752093+0000","last_peered":"2024-03-08T06:00:42.752093+0000","last_clean":"2024-03-08T06:00:42.752093+0000","last_became_active":"2024-03-08T06:00:35.163123+0000","last_became_peered":"2024-03-08T06:00:35.163123+0000","last_unstale":"2024-03-08T06:00:42.752093+0000","last_undegraded":"2024-03-08T06:00:42.752093+0000","last_fullsized":"2024-03-08T06:00:42.752093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252109+0000","last_change":"2024-03-08T06:00:36.194135+0000","last_active":"2024-03-08T06:00:43.252109+0000","last_peered":"2024-03-08T06:00:43.252109+0000","last_clean":"2024-03-08T06:00:43.252109+0000","last_became_active":"2024-03-08T06:00:36.193870+0000","last_became_peered":"2024-03-08T06:00:36.193870+0000","last_unstale":"2024-03-08T06:00:43.252109+0000","last_undegraded":"2024-03-08T06:00:43.252109+0000","last_fullsized":"2024-03-08T06:00:43.252109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:02:15.458747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206553+0000","last_change":"2024-03-08T06:00:40.221366+0000","last_active":"2024-03-08T06:00:42.206553+0000","last_peered":"2024-03-08T06:00:42.206553+0000","last_clean":"2024-03-08T06:00:42.206553+0000","last_became_active":"2024-03-08T06:00:40.220967+0000","last_became_peered":"2024-03-08T06:00:40.220967+0000","last_unstale":"2024-03-08T06:00:42.206553+0000","last_undegraded":"2024-03-08T06:00:42.206553+0000","last_fullsized":"2024-03-08T06:00:42.206553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:16:51.281060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751868+0000","last_change":"2024-03-08T06:00:35.173354+0000","last_active":"2024-03-08T06:00:42.751868+0000","last_peered":"2024-03-08T06:00:42.751868+0000","last_clean":"2024-03-08T06:00:42.751868+0000","last_became_active":"2024-03-08T06:00:35.173105+0000","last_became_peered":"2024-03-08T06:00:35.173105+0000","last_unstale":"2024-03-08T06:00:42.751868+0000","last_undegraded":"2024-03-08T06:00:42.751868+0000","last_fullsized":"2024-03-08T06:00:42.751868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252178+0000","last_change":"2024-03-08T06:00:36.193424+0000","last_active":"2024-03-08T06:00:43.252178+0000","last_peered":"2024-03-08T06:00:43.252178+0000","last_clean":"2024-03-08T06:00:43.252178+0000","last_became_active":"2024-03-08T06:00:36.191212+0000","last_became_peered":"2024-03-08T06:00:36.191212+0000","last_unstale":"2024-03-08T06:00:43.252178+0000","last_undegraded":"2024-03-08T06:00:43.252178+0000","last_fullsized":"2024-03-08T06:00:43.252178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:02.331109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206122+0000","last_change":"2024-03-08T06:00:40.221344+0000","last_active":"2024-03-08T06:00:42.206122+0000","last_peered":"2024-03-08T06:00:42.206122+0000","last_clean":"2024-03-08T06:00:42.206122+0000","last_became_active":"2024-03-08T06:00:40.220925+0000","last_became_peered":"2024-03-08T06:00:40.220925+0000","last_unstale":"2024-03-08T06:00:42.206122+0000","last_undegraded":"2024-03-08T06:00:42.206122+0000","last_fullsized":"2024-03-08T06:00:42.206122+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:06.372126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254602+0000","last_change":"2024-03-08T06:00:36.186043+0000","last_active":"2024-03-08T06:00:43.254602+0000","last_peered":"2024-03-08T06:00:43.254602+0000","last_clean":"2024-03-08T06:00:43.254602+0000","last_became_active":"2024-03-08T06:00:36.185793+0000","last_became_peered":"2024-03-08T06:00:36.185793+0000","last_unstale":"2024-03-08T06:00:43.254602+0000","last_undegraded":"2024-03-08T06:00:43.254602+0000","last_fullsized":"2024-03-08T06:00:43.254602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:01:37.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751980+0000","last_change":"2024-03-08T06:00:35.163406+0000","last_active":"2024-03-08T06:00:42.751980+0000","last_peered":"2024-03-08T06:00:42.751980+0000","last_clean":"2024-03-08T06:00:42.751980+0000","last_became_active":"2024-03-08T06:00:35.163138+0000","last_became_peered":"2024-03-08T06:00:35.163138+0000","last_unstale":"2024-03-08T06:00:42.751980+0000","last_undegraded":"2024-03-08T06:00:42.751980+0000","last_fullsized":"2024-03-08T06:00:42.751980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:38.366092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206808+0000","last_change":"2024-03-08T06:00:40.219654+0000","last_active":"2024-03-08T06:00:42.206808+0000","last_peered":"2024-03-08T06:00:42.206808+0000","last_clean":"2024-03-08T06:00:42.206808+0000","last_became_active":"2024-03-08T06:00:40.219225+0000","last_became_peered":"2024-03-08T06:00:40.219225+0000","last_unstale":"2024-03-08T06:00:42.206808+0000","last_undegraded":"2024-03-08T06:00:42.206808+0000","last_fullsized":"2024-03-08T06:00:42.206808+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:34:22.301392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205304+0000","last_change":"2024-03-08T06:00:36.184886+0000","last_active":"2024-03-08T06:00:42.205304+0000","last_peered":"2024-03-08T06:00:42.205304+0000","last_clean":"2024-03-08T06:00:42.205304+0000","last_became_active":"2024-03-08T06:00:36.184582+0000","last_became_peered":"2024-03-08T06:00:36.184582+0000","last_unstale":"2024-03-08T06:00:42.205304+0000","last_undegraded":"2024-03-08T06:00:42.205304+0000","last_fullsized":"2024-03-08T06:00:42.205304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251830+0000","last_change":"2024-03-08T06:00:35.171210+0000","last_active":"2024-03-08T06:00:43.251830+0000","last_peered":"2024-03-08T06:00:43.251830+0000","last_clean":"2024-03-08T06:00:43.251830+0000","last_became_active":"2024-03-08T06:00:35.170784+0000","last_became_peered":"2024-03-08T06:00:35.170784+0000","last_unstale":"2024-03-08T06:00:43.251830+0000","last_undegraded":"2024-03-08T06:00:43.251830+0000","last_fullsized":"2024-03-08T06:00:43.251830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206300+0000","last_change":"2024-03-08T06:00:40.221460+0000","last_active":"2024-03-08T06:00:42.206300+0000","last_peered":"2024-03-08T06:00:42.206300+0000","last_clean":"2024-03-08T06:00:42.206300+0000","last_became_active":"2024-03-08T06:00:40.221046+0000","last_became_peered":"2024-03-08T06:00:40.221046+0000","last_unstale":"2024-03-08T06:00:42.206300+0000","last_undegraded":"2024-03-08T06:00:42.206300+0000","last_fullsized":"2024-03-08T06:00:42.206300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:35.686997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205125+0000","last_change":"2024-03-08T06:00:36.181765+0000","last_active":"2024-03-08T06:00:42.205125+0000","last_peered":"2024-03-08T06:00:42.205125+0000","last_clean":"2024-03-08T06:00:42.205125+0000","last_became_active":"2024-03-08T06:00:36.181503+0000","last_became_peered":"2024-03-08T06:00:36.181503+0000","last_unstale":"2024-03-08T06:00:42.205125+0000","last_undegraded":"2024-03-08T06:00:42.205125+0000","last_fullsized":"2024-03-08T06:00:42.205125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:01:37.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.250999+0000","last_change":"2024-03-08T06:00:35.164505+0000","last_active":"2024-03-08T06:00:43.250999+0000","last_peered":"2024-03-08T06:00:43.250999+0000","last_clean":"2024-03-08T06:00:43.250999+0000","last_became_active":"2024-03-08T06:00:35.164203+0000","last_became_peered":"2024-03-08T06:00:35.164203+0000","last_unstale":"2024-03-08T06:00:43.250999+0000","last_undegraded":"2024-03-08T06:00:43.250999+0000","last_fullsized":"2024-03-08T06:00:43.250999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:38.366092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253073+0000","last_change":"2024-03-08T06:00:40.772590+0000","last_active":"2024-03-08T06:00:43.253073+0000","last_peered":"2024-03-08T06:00:43.253073+0000","last_clean":"2024-03-08T06:00:43.253073+0000","last_became_active":"2024-03-08T06:00:40.772198+0000","last_became_peered":"2024-03-08T06:00:40.772198+0000","last_unstale":"2024-03-08T06:00:43.253073+0000","last_undegraded":"2024-03-08T06:00:43.253073+0000","last_fullsized":"2024-03-08T06:00:43.253073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:24:20.856936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206949+0000","last_change":"2024-03-08T06:00:36.180323+0000","last_active":"2024-03-08T06:00:42.206949+0000","last_peered":"2024-03-08T06:00:42.206949+0000","last_clean":"2024-03-08T06:00:42.206949+0000","last_became_active":"2024-03-08T06:00:36.180087+0000","last_became_peered":"2024-03-08T06:00:36.180087+0000","last_unstale":"2024-03-08T06:00:42.206949+0000","last_undegraded":"2024-03-08T06:00:42.206949+0000","last_fullsized":"2024-03-08T06:00:42.206949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:30:07.063230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751552+0000","last_change":"2024-03-08T06:00:35.179401+0000","last_active":"2024-03-08T06:00:42.751552+0000","last_peered":"2024-03-08T06:00:42.751552+0000","last_clean":"2024-03-08T06:00:42.751552+0000","last_became_active":"2024-03-08T06:00:35.179121+0000","last_became_peered":"2024-03-08T06:00:35.179121+0000","last_unstale":"2024-03-08T06:00:42.751552+0000","last_undegraded":"2024-03-08T06:00:42.751552+0000","last_fullsized":"2024-03-08T06:00:42.751552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:30:06.058910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.207092+0000","last_change":"2024-03-08T06:00:40.222278+0000","last_active":"2024-03-08T06:00:42.207092+0000","last_peered":"2024-03-08T06:00:42.207092+0000","last_clean":"2024-03-08T06:00:42.207092+0000","last_became_active":"2024-03-08T06:00:40.221874+0000","last_became_peered":"2024-03-08T06:00:40.221874+0000","last_unstale":"2024-03-08T06:00:42.207092+0000","last_undegraded":"2024-03-08T06:00:42.207092+0000","last_fullsized":"2024-03-08T06:00:42.207092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:33.029056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251991+0000","last_change":"2024-03-08T06:00:36.180994+0000","last_active":"2024-03-08T06:00:43.251991+0000","last_peered":"2024-03-08T06:00:43.251991+0000","last_clean":"2024-03-08T06:00:43.251991+0000","last_became_active":"2024-03-08T06:00:36.178891+0000","last_became_peered":"2024-03-08T06:00:36.178891+0000","last_unstale":"2024-03-08T06:00:43.251991+0000","last_undegraded":"2024-03-08T06:00:43.251991+0000","last_fullsized":"2024-03-08T06:00:43.251991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:40:16.053974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253772+0000","last_change":"2024-03-08T06:00:35.166077+0000","last_active":"2024-03-08T06:00:43.253772+0000","last_peered":"2024-03-08T06:00:43.253772+0000","last_clean":"2024-03-08T06:00:43.253772+0000","last_became_active":"2024-03-08T06:00:35.165721+0000","last_became_peered":"2024-03-08T06:00:35.165721+0000","last_unstale":"2024-03-08T06:00:43.253772+0000","last_undegraded":"2024-03-08T06:00:43.253772+0000","last_fullsized":"2024-03-08T06:00:43.253772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206253+0000","last_change":"2024-03-08T06:00:40.772000+0000","last_active":"2024-03-08T06:00:42.206253+0000","last_peered":"2024-03-08T06:00:42.206253+0000","last_clean":"2024-03-08T06:00:42.206253+0000","last_became_active":"2024-03-08T06:00:40.771449+0000","last_became_peered":"2024-03-08T06:00:40.771449+0000","last_unstale":"2024-03-08T06:00:42.206253+0000","last_undegraded":"2024-03-08T06:00:42.206253+0000","last_fullsized":"2024-03-08T06:00:42.206253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:44:48.992193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205931+0000","last_change":"2024-03-08T06:00:36.185234+0000","last_active":"2024-03-08T06:00:42.205931+0000","last_peered":"2024-03-08T06:00:42.205931+0000","last_clean":"2024-03-08T06:00:42.205931+0000","last_became_active":"2024-03-08T06:00:36.184020+0000","last_became_peered":"2024-03-08T06:00:36.184020+0000","last_unstale":"2024-03-08T06:00:42.205931+0000","last_undegraded":"2024-03-08T06:00:42.205931+0000","last_fullsized":"2024-03-08T06:00:42.205931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:39:59.986300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252725+0000","last_change":"2024-03-08T06:00:35.166222+0000","last_active":"2024-03-08T06:00:43.252725+0000","last_peered":"2024-03-08T06:00:43.252725+0000","last_clean":"2024-03-08T06:00:43.252725+0000","last_became_active":"2024-03-08T06:00:35.165815+0000","last_became_peered":"2024-03-08T06:00:35.165815+0000","last_unstale":"2024-03-08T06:00:43.252725+0000","last_undegraded":"2024-03-08T06:00:43.252725+0000","last_fullsized":"2024-03-08T06:00:43.252725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.204928+0000","last_change":"2024-03-08T06:00:40.771309+0000","last_active":"2024-03-08T06:00:42.204928+0000","last_peered":"2024-03-08T06:00:42.204928+0000","last_clean":"2024-03-08T06:00:42.204928+0000","last_became_active":"2024-03-08T06:00:40.770916+0000","last_became_peered":"2024-03-08T06:00:40.770916+0000","last_unstale":"2024-03-08T06:00:42.204928+0000","last_undegraded":"2024-03-08T06:00:42.204928+0000","last_fullsized":"2024-03-08T06:00:42.204928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:26:51.436673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252117+0000","last_change":"2024-03-08T06:00:36.177665+0000","last_active":"2024-03-08T06:00:43.252117+0000","last_peered":"2024-03-08T06:00:43.252117+0000","last_clean":"2024-03-08T06:00:43.252117+0000","last_became_active":"2024-03-08T06:00:36.177409+0000","last_became_peered":"2024-03-08T06:00:36.177409+0000","last_unstale":"2024-03-08T06:00:43.252117+0000","last_undegraded":"2024-03-08T06:00:43.252117+0000","last_fullsized":"2024-03-08T06:00:43.252117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752203+0000","last_change":"2024-03-08T06:00:35.179418+0000","last_active":"2024-03-08T06:00:42.752203+0000","last_peered":"2024-03-08T06:00:42.752203+0000","last_clean":"2024-03-08T06:00:42.752203+0000","last_became_active":"2024-03-08T06:00:35.179132+0000","last_became_peered":"2024-03-08T06:00:35.179132+0000","last_unstale":"2024-03-08T06:00:42.752203+0000","last_undegraded":"2024-03-08T06:00:42.752203+0000","last_fullsized":"2024-03-08T06:00:42.752203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:11.262668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251653+0000","last_change":"2024-03-08T06:00:40.771662+0000","last_active":"2024-03-08T06:00:43.251653+0000","last_peered":"2024-03-08T06:00:43.251653+0000","last_clean":"2024-03-08T06:00:43.251653+0000","last_became_active":"2024-03-08T06:00:40.771168+0000","last_became_peered":"2024-03-08T06:00:40.771168+0000","last_unstale":"2024-03-08T06:00:43.251653+0000","last_undegraded":"2024-03-08T06:00:43.251653+0000","last_fullsized":"2024-03-08T06:00:43.251653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:16:51.281060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252306+0000","last_change":"2024-03-08T06:00:36.193542+0000","last_active":"2024-03-08T06:00:43.252306+0000","last_peered":"2024-03-08T06:00:43.252306+0000","last_clean":"2024-03-08T06:00:43.252306+0000","last_became_active":"2024-03-08T06:00:36.191695+0000","last_became_peered":"2024-03-08T06:00:36.191695+0000","last_unstale":"2024-03-08T06:00:43.252306+0000","last_undegraded":"2024-03-08T06:00:43.252306+0000","last_fullsized":"2024-03-08T06:00:43.252306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:16:14.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252207+0000","last_change":"2024-03-08T06:00:35.177137+0000","last_active":"2024-03-08T06:00:43.252207+0000","last_peered":"2024-03-08T06:00:43.252207+0000","last_clean":"2024-03-08T06:00:43.252207+0000","last_became_active":"2024-03-08T06:00:35.176790+0000","last_became_peered":"2024-03-08T06:00:35.176790+0000","last_unstale":"2024-03-08T06:00:43.252207+0000","last_undegraded":"2024-03-08T06:00:43.252207+0000","last_fullsized":"2024-03-08T06:00:43.252207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:11.262668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252112+0000","last_change":"2024-03-08T06:00:40.220329+0000","last_active":"2024-03-08T06:00:43.252112+0000","last_peered":"2024-03-08T06:00:43.252112+0000","last_clean":"2024-03-08T06:00:43.252112+0000","last_became_active":"2024-03-08T06:00:40.219921+0000","last_became_peered":"2024-03-08T06:00:40.219921+0000","last_unstale":"2024-03-08T06:00:43.252112+0000","last_undegraded":"2024-03-08T06:00:43.252112+0000","last_fullsized":"2024-03-08T06:00:43.252112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:08:55.908866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.207142+0000","last_change":"2024-03-08T06:00:36.187190+0000","last_active":"2024-03-08T06:00:42.207142+0000","last_peered":"2024-03-08T06:00:42.207142+0000","last_clean":"2024-03-08T06:00:42.207142+0000","last_became_active":"2024-03-08T06:00:36.186916+0000","last_became_peered":"2024-03-08T06:00:36.186916+0000","last_unstale":"2024-03-08T06:00:42.207142+0000","last_undegraded":"2024-03-08T06:00:42.207142+0000","last_fullsized":"2024-03-08T06:00:42.207142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:39:00.058426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251977+0000","last_change":"2024-03-08T06:00:35.179558+0000","last_active":"2024-03-08T06:00:43.251977+0000","last_peered":"2024-03-08T06:00:43.251977+0000","last_clean":"2024-03-08T06:00:43.251977+0000","last_became_active":"2024-03-08T06:00:35.179266+0000","last_became_peered":"2024-03-08T06:00:35.179266+0000","last_unstale":"2024-03-08T06:00:43.251977+0000","last_undegraded":"2024-03-08T06:00:43.251977+0000","last_fullsized":"2024-03-08T06:00:43.251977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:38.366092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252067+0000","last_change":"2024-03-08T06:00:40.224570+0000","last_active":"2024-03-08T06:00:43.252067+0000","last_peered":"2024-03-08T06:00:43.252067+0000","last_clean":"2024-03-08T06:00:43.252067+0000","last_became_active":"2024-03-08T06:00:40.222949+0000","last_became_peered":"2024-03-08T06:00:40.222949+0000","last_unstale":"2024-03-08T06:00:43.252067+0000","last_undegraded":"2024-03-08T06:00:43.252067+0000","last_fullsized":"2024-03-08T06:00:43.252067+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:16:51.281060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251279+0000","last_change":"2024-03-08T06:00:36.178768+0000","last_active":"2024-03-08T06:00:43.251279+0000","last_peered":"2024-03-08T06:00:43.251279+0000","last_clean":"2024-03-08T06:00:43.251279+0000","last_became_active":"2024-03-08T06:00:36.177890+0000","last_became_peered":"2024-03-08T06:00:36.177890+0000","last_unstale":"2024-03-08T06:00:43.251279+0000","last_undegraded":"2024-03-08T06:00:43.251279+0000","last_fullsized":"2024-03-08T06:00:43.251279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:01:37.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251961+0000","last_change":"2024-03-08T06:00:35.177031+0000","last_active":"2024-03-08T06:00:43.251961+0000","last_peered":"2024-03-08T06:00:43.251961+0000","last_clean":"2024-03-08T06:00:43.251961+0000","last_became_active":"2024-03-08T06:00:35.176687+0000","last_became_peered":"2024-03-08T06:00:35.176687+0000","last_unstale":"2024-03-08T06:00:43.251961+0000","last_undegraded":"2024-03-08T06:00:43.251961+0000","last_fullsized":"2024-03-08T06:00:43.251961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253204+0000","last_change":"2024-03-08T06:00:40.219392+0000","last_active":"2024-03-08T06:00:43.253204+0000","last_peered":"2024-03-08T06:00:43.253204+0000","last_clean":"2024-03-08T06:00:43.253204+0000","last_became_active":"2024-03-08T06:00:40.219040+0000","last_became_peered":"2024-03-08T06:00:40.219040+0000","last_unstale":"2024-03-08T06:00:43.253204+0000","last_undegraded":"2024-03-08T06:00:43.253204+0000","last_fullsized":"2024-03-08T06:00:43.253204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:33.029056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251221+0000","last_change":"2024-03-08T06:00:36.185525+0000","last_active":"2024-03-08T06:00:43.251221+0000","last_peered":"2024-03-08T06:00:43.251221+0000","last_clean":"2024-03-08T06:00:43.251221+0000","last_became_active":"2024-03-08T06:00:36.185111+0000","last_became_peered":"2024-03-08T06:00:36.185111+0000","last_unstale":"2024-03-08T06:00:43.251221+0000","last_undegraded":"2024-03-08T06:00:43.251221+0000","last_fullsized":"2024-03-08T06:00:43.251221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:52.669024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751556+0000","last_change":"2024-03-08T06:00:35.166949+0000","last_active":"2024-03-08T06:00:42.751556+0000","last_peered":"2024-03-08T06:00:42.751556+0000","last_clean":"2024-03-08T06:00:42.751556+0000","last_became_active":"2024-03-08T06:00:35.166680+0000","last_became_peered":"2024-03-08T06:00:35.166680+0000","last_unstale":"2024-03-08T06:00:42.751556+0000","last_undegraded":"2024-03-08T06:00:42.751556+0000","last_fullsized":"2024-03-08T06:00:42.751556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:58:33.115927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251873+0000","last_change":"2024-03-08T06:00:40.771268+0000","last_active":"2024-03-08T06:00:43.251873+0000","last_peered":"2024-03-08T06:00:43.251873+0000","last_clean":"2024-03-08T06:00:43.251873+0000","last_became_active":"2024-03-08T06:00:40.770846+0000","last_became_peered":"2024-03-08T06:00:40.770846+0000","last_unstale":"2024-03-08T06:00:43.251873+0000","last_undegraded":"2024-03-08T06:00:43.251873+0000","last_fullsized":"2024-03-08T06:00:43.251873+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:34:22.301392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205219+0000","last_change":"2024-03-08T06:00:36.200350+0000","last_active":"2024-03-08T06:00:42.205219+0000","last_peered":"2024-03-08T06:00:42.205219+0000","last_clean":"2024-03-08T06:00:42.205219+0000","last_became_active":"2024-03-08T06:00:36.200036+0000","last_became_peered":"2024-03-08T06:00:36.200036+0000","last_unstale":"2024-03-08T06:00:42.205219+0000","last_undegraded":"2024-03-08T06:00:42.205219+0000","last_fullsized":"2024-03-08T06:00:42.205219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:10:17.180499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251960+0000","last_change":"2024-03-08T06:00:35.176857+0000","last_active":"2024-03-08T06:00:43.251960+0000","last_peered":"2024-03-08T06:00:43.251960+0000","last_clean":"2024-03-08T06:00:43.251960+0000","last_became_active":"2024-03-08T06:00:35.176593+0000","last_became_peered":"2024-03-08T06:00:35.176593+0000","last_unstale":"2024-03-08T06:00:43.251960+0000","last_undegraded":"2024-03-08T06:00:43.251960+0000","last_fullsized":"2024-03-08T06:00:43.251960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:41.059111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751648+0000","last_change":"2024-03-08T06:00:26.178419+0000","last_active":"2024-03-08T06:00:42.751648+0000","last_peered":"2024-03-08T06:00:42.751648+0000","last_clean":"2024-03-08T06:00:42.751648+0000","last_became_active":"2024-03-08T06:00:26.175826+0000","last_became_peered":"2024-03-08T06:00:26.175826+0000","last_unstale":"2024-03-08T06:00:42.751648+0000","last_undegraded":"2024-03-08T06:00:42.751648+0000","last_fullsized":"2024-03-08T06:00:42.751648+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_clean_scrub_stamp":"2024-03-08T06:00:24.029492+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:36:00.682517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254739+0000","last_change":"2024-03-08T06:00:40.773106+0000","last_active":"2024-03-08T06:00:43.254739+0000","last_peered":"2024-03-08T06:00:43.254739+0000","last_clean":"2024-03-08T06:00:43.254739+0000","last_became_active":"2024-03-08T06:00:40.772665+0000","last_became_peered":"2024-03-08T06:00:40.772665+0000","last_unstale":"2024-03-08T06:00:43.254739+0000","last_undegraded":"2024-03-08T06:00:43.254739+0000","last_fullsized":"2024-03-08T06:00:43.254739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:02:19.499764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752004+0000","last_change":"2024-03-08T06:00:36.204915+0000","last_active":"2024-03-08T06:00:42.752004+0000","last_peered":"2024-03-08T06:00:42.752004+0000","last_clean":"2024-03-08T06:00:42.752004+0000","last_became_active":"2024-03-08T06:00:36.204663+0000","last_became_peered":"2024-03-08T06:00:36.204663+0000","last_unstale":"2024-03-08T06:00:42.752004+0000","last_undegraded":"2024-03-08T06:00:42.752004+0000","last_fullsized":"2024-03-08T06:00:42.752004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:05.458469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252634+0000","last_change":"2024-03-08T06:00:35.173339+0000","last_active":"2024-03-08T06:00:43.252634+0000","last_peered":"2024-03-08T06:00:43.252634+0000","last_clean":"2024-03-08T06:00:43.252634+0000","last_became_active":"2024-03-08T06:00:35.173024+0000","last_became_peered":"2024-03-08T06:00:35.173024+0000","last_unstale":"2024-03-08T06:00:43.252634+0000","last_undegraded":"2024-03-08T06:00:43.252634+0000","last_fullsized":"2024-03-08T06:00:43.252634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:41.059111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254629+0000","last_change":"2024-03-08T06:00:40.771194+0000","last_active":"2024-03-08T06:00:43.254629+0000","last_peered":"2024-03-08T06:00:43.254629+0000","last_clean":"2024-03-08T06:00:43.254629+0000","last_became_active":"2024-03-08T06:00:40.770762+0000","last_became_peered":"2024-03-08T06:00:40.770762+0000","last_unstale":"2024-03-08T06:00:43.254629+0000","last_undegraded":"2024-03-08T06:00:43.254629+0000","last_fullsized":"2024-03-08T06:00:43.254629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:21:03.123682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252111+0000","last_change":"2024-03-08T06:00:36.186111+0000","last_active":"2024-03-08T06:00:43.252111+0000","last_peered":"2024-03-08T06:00:43.252111+0000","last_clean":"2024-03-08T06:00:43.252111+0000","last_became_active":"2024-03-08T06:00:36.185865+0000","last_became_peered":"2024-03-08T06:00:36.185865+0000","last_unstale":"2024-03-08T06:00:43.252111+0000","last_undegraded":"2024-03-08T06:00:43.252111+0000","last_fullsized":"2024-03-08T06:00:43.252111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:19:26.058130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251711+0000","last_change":"2024-03-08T06:00:35.166769+0000","last_active":"2024-03-08T06:00:43.251711+0000","last_peered":"2024-03-08T06:00:43.251711+0000","last_clean":"2024-03-08T06:00:43.251711+0000","last_became_active":"2024-03-08T06:00:35.166396+0000","last_became_peered":"2024-03-08T06:00:35.166396+0000","last_unstale":"2024-03-08T06:00:43.251711+0000","last_undegraded":"2024-03-08T06:00:43.251711+0000","last_fullsized":"2024-03-08T06:00:43.251711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:30:06.058910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251907+0000","last_change":"2024-03-08T06:00:40.219431+0000","last_active":"2024-03-08T06:00:43.251907+0000","last_peered":"2024-03-08T06:00:43.251907+0000","last_clean":"2024-03-08T06:00:43.251907+0000","last_became_active":"2024-03-08T06:00:40.219091+0000","last_became_peered":"2024-03-08T06:00:40.219091+0000","last_unstale":"2024-03-08T06:00:43.251907+0000","last_undegraded":"2024-03-08T06:00:43.251907+0000","last_fullsized":"2024-03-08T06:00:43.251907+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:24:20.856936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751480+0000","last_change":"2024-03-08T06:00:36.202917+0000","last_active":"2024-03-08T06:00:42.751480+0000","last_peered":"2024-03-08T06:00:42.751480+0000","last_clean":"2024-03-08T06:00:42.751480+0000","last_became_active":"2024-03-08T06:00:36.202100+0000","last_became_peered":"2024-03-08T06:00:36.202100+0000","last_unstale":"2024-03-08T06:00:42.751480+0000","last_undegraded":"2024-03-08T06:00:42.751480+0000","last_fullsized":"2024-03-08T06:00:42.751480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:31.645981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252205+0000","last_change":"2024-03-08T06:00:35.171650+0000","last_active":"2024-03-08T06:00:43.252205+0000","last_peered":"2024-03-08T06:00:43.252205+0000","last_clean":"2024-03-08T06:00:43.252205+0000","last_became_active":"2024-03-08T06:00:35.171310+0000","last_became_peered":"2024-03-08T06:00:35.171310+0000","last_unstale":"2024-03-08T06:00:43.252205+0000","last_undegraded":"2024-03-08T06:00:43.252205+0000","last_fullsized":"2024-03-08T06:00:43.252205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251524+0000","last_change":"2024-03-08T06:00:40.774574+0000","last_active":"2024-03-08T06:00:43.251524+0000","last_peered":"2024-03-08T06:00:43.251524+0000","last_clean":"2024-03-08T06:00:43.251524+0000","last_became_active":"2024-03-08T06:00:40.774172+0000","last_became_peered":"2024-03-08T06:00:40.774172+0000","last_unstale":"2024-03-08T06:00:43.251524+0000","last_undegraded":"2024-03-08T06:00:43.251524+0000","last_fullsized":"2024-03-08T06:00:43.251524+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:48:26.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752182+0000","last_change":"2024-03-08T06:00:36.195883+0000","last_active":"2024-03-08T06:00:42.752182+0000","last_peered":"2024-03-08T06:00:42.752182+0000","last_clean":"2024-03-08T06:00:42.752182+0000","last_became_active":"2024-03-08T06:00:36.195641+0000","last_became_peered":"2024-03-08T06:00:36.195641+0000","last_unstale":"2024-03-08T06:00:42.752182+0000","last_undegraded":"2024-03-08T06:00:42.752182+0000","last_fullsized":"2024-03-08T06:00:42.752182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:02:15.458747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206956+0000","last_change":"2024-03-08T06:00:35.165484+0000","last_active":"2024-03-08T06:00:42.206956+0000","last_peered":"2024-03-08T06:00:42.206956+0000","last_clean":"2024-03-08T06:00:42.206956+0000","last_became_active":"2024-03-08T06:00:35.165147+0000","last_became_peered":"2024-03-08T06:00:35.165147+0000","last_unstale":"2024-03-08T06:00:42.206956+0000","last_undegraded":"2024-03-08T06:00:42.206956+0000","last_fullsized":"2024-03-08T06:00:42.206956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:07:54.742119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206806+0000","last_change":"2024-03-08T06:00:40.221435+0000","last_active":"2024-03-08T06:00:42.206806+0000","last_peered":"2024-03-08T06:00:42.206806+0000","last_clean":"2024-03-08T06:00:42.206806+0000","last_became_active":"2024-03-08T06:00:40.220886+0000","last_became_peered":"2024-03-08T06:00:40.220886+0000","last_unstale":"2024-03-08T06:00:42.206806+0000","last_undegraded":"2024-03-08T06:00:42.206806+0000","last_fullsized":"2024-03-08T06:00:42.206806+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:24:20.856936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751752+0000","last_change":"2024-03-08T06:00:36.202146+0000","last_active":"2024-03-08T06:00:42.751752+0000","last_peered":"2024-03-08T06:00:42.751752+0000","last_clean":"2024-03-08T06:00:42.751752+0000","last_became_active":"2024-03-08T06:00:36.201880+0000","last_became_peered":"2024-03-08T06:00:36.201880+0000","last_unstale":"2024-03-08T06:00:42.751752+0000","last_undegraded":"2024-03-08T06:00:42.751752+0000","last_fullsized":"2024-03-08T06:00:42.751752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:45:22.429641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252369+0000","last_change":"2024-03-08T06:00:35.168305+0000","last_active":"2024-03-08T06:00:43.252369+0000","last_peered":"2024-03-08T06:00:43.252369+0000","last_clean":"2024-03-08T06:00:43.252369+0000","last_became_active":"2024-03-08T06:00:35.168007+0000","last_became_peered":"2024-03-08T06:00:35.168007+0000","last_unstale":"2024-03-08T06:00:43.252369+0000","last_undegraded":"2024-03-08T06:00:43.252369+0000","last_fullsized":"2024-03-08T06:00:43.252369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:57:55.324372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251733+0000","last_change":"2024-03-08T06:00:40.217477+0000","last_active":"2024-03-08T06:00:43.251733+0000","last_peered":"2024-03-08T06:00:43.251733+0000","last_clean":"2024-03-08T06:00:43.251733+0000","last_became_active":"2024-03-08T06:00:40.217140+0000","last_became_peered":"2024-03-08T06:00:40.217140+0000","last_unstale":"2024-03-08T06:00:43.251733+0000","last_undegraded":"2024-03-08T06:00:43.251733+0000","last_fullsized":"2024-03-08T06:00:43.251733+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:08:55.908866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254784+0000","last_change":"2024-03-08T06:00:36.188881+0000","last_active":"2024-03-08T06:00:43.254784+0000","last_peered":"2024-03-08T06:00:43.254784+0000","last_clean":"2024-03-08T06:00:43.254784+0000","last_became_active":"2024-03-08T06:00:36.188323+0000","last_became_peered":"2024-03-08T06:00:36.188323+0000","last_unstale":"2024-03-08T06:00:43.254784+0000","last_undegraded":"2024-03-08T06:00:43.254784+0000","last_fullsized":"2024-03-08T06:00:43.254784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:11:34.950063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253637+0000","last_change":"2024-03-08T06:00:35.163917+0000","last_active":"2024-03-08T06:00:43.253637+0000","last_peered":"2024-03-08T06:00:43.253637+0000","last_clean":"2024-03-08T06:00:43.253637+0000","last_became_active":"2024-03-08T06:00:35.163650+0000","last_became_peered":"2024-03-08T06:00:35.163650+0000","last_unstale":"2024-03-08T06:00:43.253637+0000","last_undegraded":"2024-03-08T06:00:43.253637+0000","last_fullsized":"2024-03-08T06:00:43.253637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:11.262668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205799+0000","last_change":"2024-03-08T06:00:40.773617+0000","last_active":"2024-03-08T06:00:42.205799+0000","last_peered":"2024-03-08T06:00:42.205799+0000","last_clean":"2024-03-08T06:00:42.205799+0000","last_became_active":"2024-03-08T06:00:40.773144+0000","last_became_peered":"2024-03-08T06:00:40.773144+0000","last_unstale":"2024-03-08T06:00:42.205799+0000","last_undegraded":"2024-03-08T06:00:42.205799+0000","last_fullsized":"2024-03-08T06:00:42.205799+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:32:08.762183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751941+0000","last_change":"2024-03-08T06:00:36.202751+0000","last_active":"2024-03-08T06:00:42.751941+0000","last_peered":"2024-03-08T06:00:42.751941+0000","last_clean":"2024-03-08T06:00:42.751941+0000","last_became_active":"2024-03-08T06:00:36.202228+0000","last_became_peered":"2024-03-08T06:00:36.202228+0000","last_unstale":"2024-03-08T06:00:42.751941+0000","last_undegraded":"2024-03-08T06:00:42.751941+0000","last_fullsized":"2024-03-08T06:00:42.751941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:16:14.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251319+0000","last_change":"2024-03-08T06:00:35.176896+0000","last_active":"2024-03-08T06:00:43.251319+0000","last_peered":"2024-03-08T06:00:43.251319+0000","last_clean":"2024-03-08T06:00:43.251319+0000","last_became_active":"2024-03-08T06:00:35.176665+0000","last_became_peered":"2024-03-08T06:00:35.176665+0000","last_unstale":"2024-03-08T06:00:43.251319+0000","last_undegraded":"2024-03-08T06:00:43.251319+0000","last_fullsized":"2024-03-08T06:00:43.251319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:11.262668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205674+0000","last_change":"2024-03-08T06:00:40.771254+0000","last_active":"2024-03-08T06:00:42.205674+0000","last_peered":"2024-03-08T06:00:42.205674+0000","last_clean":"2024-03-08T06:00:42.205674+0000","last_became_active":"2024-03-08T06:00:40.770828+0000","last_became_peered":"2024-03-08T06:00:40.770828+0000","last_unstale":"2024-03-08T06:00:42.205674+0000","last_undegraded":"2024-03-08T06:00:42.205674+0000","last_fullsized":"2024-03-08T06:00:42.205674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:06:36.831014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251667+0000","last_change":"2024-03-08T06:00:36.192057+0000","last_active":"2024-03-08T06:00:43.251667+0000","last_peered":"2024-03-08T06:00:43.251667+0000","last_clean":"2024-03-08T06:00:43.251667+0000","last_became_active":"2024-03-08T06:00:36.191786+0000","last_became_peered":"2024-03-08T06:00:36.191786+0000","last_unstale":"2024-03-08T06:00:43.251667+0000","last_undegraded":"2024-03-08T06:00:43.251667+0000","last_fullsized":"2024-03-08T06:00:43.251667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:10:17.180499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251277+0000","last_change":"2024-03-08T06:00:35.164201+0000","last_active":"2024-03-08T06:00:43.251277+0000","last_peered":"2024-03-08T06:00:43.251277+0000","last_clean":"2024-03-08T06:00:43.251277+0000","last_became_active":"2024-03-08T06:00:35.163882+0000","last_became_peered":"2024-03-08T06:00:35.163882+0000","last_unstale":"2024-03-08T06:00:43.251277+0000","last_undegraded":"2024-03-08T06:00:43.251277+0000","last_fullsized":"2024-03-08T06:00:43.251277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:43:03.634887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252307+0000","last_change":"2024-03-08T06:00:40.773272+0000","last_active":"2024-03-08T06:00:43.252307+0000","last_peered":"2024-03-08T06:00:43.252307+0000","last_clean":"2024-03-08T06:00:43.252307+0000","last_became_active":"2024-03-08T06:00:40.772833+0000","last_became_peered":"2024-03-08T06:00:40.772833+0000","last_unstale":"2024-03-08T06:00:43.252307+0000","last_undegraded":"2024-03-08T06:00:43.252307+0000","last_fullsized":"2024-03-08T06:00:43.252307+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:53:52.773407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751887+0000","last_change":"2024-03-08T06:00:36.191731+0000","last_active":"2024-03-08T06:00:42.751887+0000","last_peered":"2024-03-08T06:00:42.751887+0000","last_clean":"2024-03-08T06:00:42.751887+0000","last_became_active":"2024-03-08T06:00:36.191469+0000","last_became_peered":"2024-03-08T06:00:36.191469+0000","last_unstale":"2024-03-08T06:00:42.751887+0000","last_undegraded":"2024-03-08T06:00:42.751887+0000","last_fullsized":"2024-03-08T06:00:42.751887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:59.082666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:45.428919+0000","last_change":"2024-03-08T06:00:35.171245+0000","last_active":"2024-03-08T06:00:45.428919+0000","last_peered":"2024-03-08T06:00:45.428919+0000","last_clean":"2024-03-08T06:00:45.428919+0000","last_became_active":"2024-03-08T06:00:35.170999+0000","last_became_peered":"2024-03-08T06:00:35.170999+0000","last_unstale":"2024-03-08T06:00:45.428919+0000","last_undegraded":"2024-03-08T06:00:45.428919+0000","last_fullsized":"2024-03-08T06:00:45.428919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252315+0000","last_change":"2024-03-08T06:00:40.222311+0000","last_active":"2024-03-08T06:00:43.252315+0000","last_peered":"2024-03-08T06:00:43.252315+0000","last_clean":"2024-03-08T06:00:43.252315+0000","last_became_active":"2024-03-08T06:00:40.221923+0000","last_became_peered":"2024-03-08T06:00:40.221923+0000","last_unstale":"2024-03-08T06:00:43.252315+0000","last_undegraded":"2024-03-08T06:00:43.252315+0000","last_fullsized":"2024-03-08T06:00:43.252315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:34:22.301392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206554+0000","last_change":"2024-03-08T06:00:36.190623+0000","last_active":"2024-03-08T06:00:42.206554+0000","last_peered":"2024-03-08T06:00:42.206554+0000","last_clean":"2024-03-08T06:00:42.206554+0000","last_became_active":"2024-03-08T06:00:36.190343+0000","last_became_peered":"2024-03-08T06:00:36.190343+0000","last_unstale":"2024-03-08T06:00:42.206554+0000","last_undegraded":"2024-03-08T06:00:42.206554+0000","last_fullsized":"2024-03-08T06:00:42.206554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:39.370412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252587+0000","last_change":"2024-03-08T06:00:35.169188+0000","last_active":"2024-03-08T06:00:43.252587+0000","last_peered":"2024-03-08T06:00:43.252587+0000","last_clean":"2024-03-08T06:00:43.252587+0000","last_became_active":"2024-03-08T06:00:35.168945+0000","last_became_peered":"2024-03-08T06:00:35.168945+0000","last_unstale":"2024-03-08T06:00:43.252587+0000","last_undegraded":"2024-03-08T06:00:43.252587+0000","last_fullsized":"2024-03-08T06:00:43.252587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:58:33.115927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206843+0000","last_change":"2024-03-08T06:00:40.219289+0000","last_active":"2024-03-08T06:00:42.206843+0000","last_peered":"2024-03-08T06:00:42.206843+0000","last_clean":"2024-03-08T06:00:42.206843+0000","last_became_active":"2024-03-08T06:00:40.218831+0000","last_became_peered":"2024-03-08T06:00:40.218831+0000","last_unstale":"2024-03-08T06:00:42.206843+0000","last_undegraded":"2024-03-08T06:00:42.206843+0000","last_fullsized":"2024-03-08T06:00:42.206843+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:08:55.908866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251809+0000","last_change":"2024-03-08T06:00:36.191492+0000","last_active":"2024-03-08T06:00:43.251809+0000","last_peered":"2024-03-08T06:00:43.251809+0000","last_clean":"2024-03-08T06:00:43.251809+0000","last_became_active":"2024-03-08T06:00:36.190895+0000","last_became_peered":"2024-03-08T06:00:36.190895+0000","last_unstale":"2024-03-08T06:00:43.251809+0000","last_undegraded":"2024-03-08T06:00:43.251809+0000","last_fullsized":"2024-03-08T06:00:43.251809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252270+0000","last_change":"2024-03-08T06:00:35.174075+0000","last_active":"2024-03-08T06:00:43.252270+0000","last_peered":"2024-03-08T06:00:43.252270+0000","last_clean":"2024-03-08T06:00:43.252270+0000","last_became_active":"2024-03-08T06:00:35.173803+0000","last_became_peered":"2024-03-08T06:00:35.173803+0000","last_unstale":"2024-03-08T06:00:43.252270+0000","last_undegraded":"2024-03-08T06:00:43.252270+0000","last_fullsized":"2024-03-08T06:00:43.252270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:03.718334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205087+0000","last_change":"2024-03-08T06:00:40.222466+0000","last_active":"2024-03-08T06:00:42.205087+0000","last_peered":"2024-03-08T06:00:42.205087+0000","last_clean":"2024-03-08T06:00:42.205087+0000","last_became_active":"2024-03-08T06:00:40.222083+0000","last_became_peered":"2024-03-08T06:00:40.222083+0000","last_unstale":"2024-03-08T06:00:42.205087+0000","last_undegraded":"2024-03-08T06:00:42.205087+0000","last_fullsized":"2024-03-08T06:00:42.205087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:21:08.958898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252185+0000","last_change":"2024-03-08T06:00:36.191479+0000","last_active":"2024-03-08T06:00:43.252185+0000","last_peered":"2024-03-08T06:00:43.252185+0000","last_clean":"2024-03-08T06:00:43.252185+0000","last_became_active":"2024-03-08T06:00:36.189667+0000","last_became_peered":"2024-03-08T06:00:36.189667+0000","last_unstale":"2024-03-08T06:00:43.252185+0000","last_undegraded":"2024-03-08T06:00:43.252185+0000","last_fullsized":"2024-03-08T06:00:43.252185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:59.082666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253827+0000","last_change":"2024-03-08T06:00:35.173447+0000","last_active":"2024-03-08T06:00:43.253827+0000","last_peered":"2024-03-08T06:00:43.253827+0000","last_clean":"2024-03-08T06:00:43.253827+0000","last_became_active":"2024-03-08T06:00:35.173050+0000","last_became_peered":"2024-03-08T06:00:35.173050+0000","last_unstale":"2024-03-08T06:00:43.253827+0000","last_undegraded":"2024-03-08T06:00:43.253827+0000","last_fullsized":"2024-03-08T06:00:43.253827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:41.059111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205049+0000","last_change":"2024-03-08T06:00:40.772940+0000","last_active":"2024-03-08T06:00:42.205049+0000","last_peered":"2024-03-08T06:00:42.205049+0000","last_clean":"2024-03-08T06:00:42.205049+0000","last_became_active":"2024-03-08T06:00:40.772593+0000","last_became_peered":"2024-03-08T06:00:40.772593+0000","last_unstale":"2024-03-08T06:00:42.205049+0000","last_undegraded":"2024-03-08T06:00:42.205049+0000","last_fullsized":"2024-03-08T06:00:42.205049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:58:58.974013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252132+0000","last_change":"2024-03-08T06:00:40.773346+0000","last_active":"2024-03-08T06:00:43.252132+0000","last_peered":"2024-03-08T06:00:43.252132+0000","last_clean":"2024-03-08T06:00:43.252132+0000","last_became_active":"2024-03-08T06:00:40.772977+0000","last_became_peered":"2024-03-08T06:00:40.772977+0000","last_unstale":"2024-03-08T06:00:43.252132+0000","last_undegraded":"2024-03-08T06:00:43.252132+0000","last_fullsized":"2024-03-08T06:00:43.252132+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:33.029056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251192+0000","last_change":"2024-03-08T06:00:36.182265+0000","last_active":"2024-03-08T06:00:43.251192+0000","last_peered":"2024-03-08T06:00:43.251192+0000","last_clean":"2024-03-08T06:00:43.251192+0000","last_became_active":"2024-03-08T06:00:36.181732+0000","last_became_peered":"2024-03-08T06:00:36.181732+0000","last_unstale":"2024-03-08T06:00:43.251192+0000","last_undegraded":"2024-03-08T06:00:43.251192+0000","last_fullsized":"2024-03-08T06:00:43.251192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:40:16.053974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251308+0000","last_change":"2024-03-08T06:00:35.173204+0000","last_active":"2024-03-08T06:00:43.251308+0000","last_peered":"2024-03-08T06:00:43.251308+0000","last_clean":"2024-03-08T06:00:43.251308+0000","last_became_active":"2024-03-08T06:00:35.172937+0000","last_became_peered":"2024-03-08T06:00:35.172937+0000","last_unstale":"2024-03-08T06:00:43.251308+0000","last_undegraded":"2024-03-08T06:00:43.251308+0000","last_fullsized":"2024-03-08T06:00:43.251308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:20.756853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254582+0000","last_change":"2024-03-08T06:00:40.223795+0000","last_active":"2024-03-08T06:00:43.254582+0000","last_peered":"2024-03-08T06:00:43.254582+0000","last_clean":"2024-03-08T06:00:43.254582+0000","last_became_active":"2024-03-08T06:00:40.223283+0000","last_became_peered":"2024-03-08T06:00:40.223283+0000","last_unstale":"2024-03-08T06:00:43.254582+0000","last_undegraded":"2024-03-08T06:00:43.254582+0000","last_fullsized":"2024-03-08T06:00:43.254582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:09.499486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205033+0000","last_change":"2024-03-08T06:00:36.185555+0000","last_active":"2024-03-08T06:00:42.205033+0000","last_peered":"2024-03-08T06:00:42.205033+0000","last_clean":"2024-03-08T06:00:42.205033+0000","last_became_active":"2024-03-08T06:00:36.185294+0000","last_became_peered":"2024-03-08T06:00:36.185294+0000","last_unstale":"2024-03-08T06:00:42.205033+0000","last_undegraded":"2024-03-08T06:00:42.205033+0000","last_fullsized":"2024-03-08T06:00:42.205033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:13.003781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205103+0000","last_change":"2024-03-08T06:00:35.171146+0000","last_active":"2024-03-08T06:00:42.205103+0000","last_peered":"2024-03-08T06:00:42.205103+0000","last_clean":"2024-03-08T06:00:42.205103+0000","last_became_active":"2024-03-08T06:00:35.170885+0000","last_became_peered":"2024-03-08T06:00:35.170885+0000","last_unstale":"2024-03-08T06:00:42.205103+0000","last_undegraded":"2024-03-08T06:00:42.205103+0000","last_fullsized":"2024-03-08T06:00:42.205103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:59:03.457310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253633+0000","last_change":"2024-03-08T06:00:35.163289+0000","last_active":"2024-03-08T06:00:43.253633+0000","last_peered":"2024-03-08T06:00:43.253633+0000","last_clean":"2024-03-08T06:00:43.253633+0000","last_became_active":"2024-03-08T06:00:35.163005+0000","last_became_peered":"2024-03-08T06:00:35.163005+0000","last_unstale":"2024-03-08T06:00:43.253633+0000","last_undegraded":"2024-03-08T06:00:43.253633+0000","last_fullsized":"2024-03-08T06:00:43.253633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:14:38.366092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252056+0000","last_change":"2024-03-08T06:00:36.194840+0000","last_active":"2024-03-08T06:00:43.252056+0000","last_peered":"2024-03-08T06:00:43.252056+0000","last_clean":"2024-03-08T06:00:43.252056+0000","last_became_active":"2024-03-08T06:00:36.194547+0000","last_became_peered":"2024-03-08T06:00:36.194547+0000","last_unstale":"2024-03-08T06:00:43.252056+0000","last_undegraded":"2024-03-08T06:00:43.252056+0000","last_fullsized":"2024-03-08T06:00:43.252056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:11:34.950063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253014+0000","last_change":"2024-03-08T06:00:40.219835+0000","last_active":"2024-03-08T06:00:43.253014+0000","last_peered":"2024-03-08T06:00:43.253014+0000","last_clean":"2024-03-08T06:00:43.253014+0000","last_became_active":"2024-03-08T06:00:40.219483+0000","last_became_peered":"2024-03-08T06:00:40.219483+0000","last_unstale":"2024-03-08T06:00:43.253014+0000","last_undegraded":"2024-03-08T06:00:43.253014+0000","last_fullsized":"2024-03-08T06:00:43.253014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:41:06.159304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205357+0000","last_change":"2024-03-08T06:00:35.171304+0000","last_active":"2024-03-08T06:00:42.205357+0000","last_peered":"2024-03-08T06:00:42.205357+0000","last_clean":"2024-03-08T06:00:42.205357+0000","last_became_active":"2024-03-08T06:00:35.170979+0000","last_became_peered":"2024-03-08T06:00:35.170979+0000","last_unstale":"2024-03-08T06:00:42.205357+0000","last_undegraded":"2024-03-08T06:00:42.205357+0000","last_fullsized":"2024-03-08T06:00:42.205357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:34.142458+0000","last_clean_scrub_stamp":"2024-03-08T06:00:34.142458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:41.059111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206410+0000","last_change":"2024-03-08T06:00:36.179352+0000","last_active":"2024-03-08T06:00:42.206410+0000","last_peered":"2024-03-08T06:00:42.206410+0000","last_clean":"2024-03-08T06:00:42.206410+0000","last_became_active":"2024-03-08T06:00:36.179123+0000","last_became_peered":"2024-03-08T06:00:36.179123+0000","last_unstale":"2024-03-08T06:00:42.206410+0000","last_undegraded":"2024-03-08T06:00:42.206410+0000","last_fullsized":"2024-03-08T06:00:42.206410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:12:12.266989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205576+0000","last_change":"2024-03-08T06:00:40.220569+0000","last_active":"2024-03-08T06:00:42.205576+0000","last_peered":"2024-03-08T06:00:42.205576+0000","last_clean":"2024-03-08T06:00:42.205576+0000","last_became_active":"2024-03-08T06:00:40.220153+0000","last_became_peered":"2024-03-08T06:00:40.220153+0000","last_unstale":"2024-03-08T06:00:42.205576+0000","last_undegraded":"2024-03-08T06:00:42.205576+0000","last_fullsized":"2024-03-08T06:00:42.205576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:39.187795+0000","last_clean_scrub_stamp":"2024-03-08T06:00:39.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:44:28.955585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254222+0000","last_change":"2024-03-08T06:00:36.189112+0000","last_active":"2024-03-08T06:00:43.254222+0000","last_peered":"2024-03-08T06:00:43.254222+0000","last_clean":"2024-03-08T06:00:43.254222+0000","last_became_active":"2024-03-08T06:00:36.188650+0000","last_became_peered":"2024-03-08T06:00:36.188650+0000","last_unstale":"2024-03-08T06:00:43.254222+0000","last_undegraded":"2024-03-08T06:00:43.254222+0000","last_fullsized":"2024-03-08T06:00:43.254222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:13.003781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252434+0000","last_change":"2024-03-08T06:00:36.188109+0000","last_active":"2024-03-08T06:00:43.252434+0000","last_peered":"2024-03-08T06:00:43.252434+0000","last_clean":"2024-03-08T06:00:43.252434+0000","last_became_active":"2024-03-08T06:00:36.187798+0000","last_became_peered":"2024-03-08T06:00:36.187798+0000","last_unstale":"2024-03-08T06:00:43.252434+0000","last_undegraded":"2024-03-08T06:00:43.252434+0000","last_fullsized":"2024-03-08T06:00:43.252434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:30:07.063230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251880+0000","last_change":"2024-03-08T06:00:36.193357+0000","last_active":"2024-03-08T06:00:43.251880+0000","last_peered":"2024-03-08T06:00:43.251880+0000","last_clean":"2024-03-08T06:00:43.251880+0000","last_became_active":"2024-03-08T06:00:36.191586+0000","last_became_peered":"2024-03-08T06:00:36.191586+0000","last_unstale":"2024-03-08T06:00:43.251880+0000","last_undegraded":"2024-03-08T06:00:43.251880+0000","last_fullsized":"2024-03-08T06:00:43.251880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:44:44.951176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251957+0000","last_change":"2024-03-08T06:00:36.186266+0000","last_active":"2024-03-08T06:00:43.251957+0000","last_peered":"2024-03-08T06:00:43.251957+0000","last_clean":"2024-03-08T06:00:43.251957+0000","last_became_active":"2024-03-08T06:00:36.185663+0000","last_became_peered":"2024-03-08T06:00:36.185663+0000","last_unstale":"2024-03-08T06:00:43.251957+0000","last_undegraded":"2024-03-08T06:00:43.251957+0000","last_fullsized":"2024-03-08T06:00:43.251957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:53:20.034194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205842+0000","last_change":"2024-03-08T06:00:36.192645+0000","last_active":"2024-03-08T06:00:42.205842+0000","last_peered":"2024-03-08T06:00:42.205842+0000","last_clean":"2024-03-08T06:00:42.205842+0000","last_became_active":"2024-03-08T06:00:36.192391+0000","last_became_peered":"2024-03-08T06:00:36.192391+0000","last_unstale":"2024-03-08T06:00:42.205842+0000","last_undegraded":"2024-03-08T06:00:42.205842+0000","last_fullsized":"2024-03-08T06:00:42.205842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:45:22.429641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252413+0000","last_change":"2024-03-08T06:00:36.181871+0000","last_active":"2024-03-08T06:00:43.252413+0000","last_peered":"2024-03-08T06:00:43.252413+0000","last_clean":"2024-03-08T06:00:43.252413+0000","last_became_active":"2024-03-08T06:00:36.181557+0000","last_became_peered":"2024-03-08T06:00:36.181557+0000","last_unstale":"2024-03-08T06:00:43.252413+0000","last_undegraded":"2024-03-08T06:00:43.252413+0000","last_fullsized":"2024-03-08T06:00:43.252413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:10:17.180499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254922+0000","last_change":"2024-03-08T06:00:36.185046+0000","last_active":"2024-03-08T06:00:43.254922+0000","last_peered":"2024-03-08T06:00:43.254922+0000","last_clean":"2024-03-08T06:00:43.254922+0000","last_became_active":"2024-03-08T06:00:36.184066+0000","last_became_peered":"2024-03-08T06:00:36.184066+0000","last_unstale":"2024-03-08T06:00:43.254922+0000","last_undegraded":"2024-03-08T06:00:43.254922+0000","last_fullsized":"2024-03-08T06:00:43.254922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205846+0000","last_change":"2024-03-08T06:00:36.186595+0000","last_active":"2024-03-08T06:00:42.205846+0000","last_peered":"2024-03-08T06:00:42.205846+0000","last_clean":"2024-03-08T06:00:42.205846+0000","last_became_active":"2024-03-08T06:00:36.186261+0000","last_became_peered":"2024-03-08T06:00:36.186261+0000","last_unstale":"2024-03-08T06:00:42.205846+0000","last_undegraded":"2024-03-08T06:00:42.205846+0000","last_fullsized":"2024-03-08T06:00:42.205846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:16:14.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251609+0000","last_change":"2024-03-08T06:00:36.186618+0000","last_active":"2024-03-08T06:00:43.251609+0000","last_peered":"2024-03-08T06:00:43.251609+0000","last_clean":"2024-03-08T06:00:43.251609+0000","last_became_active":"2024-03-08T06:00:36.186068+0000","last_became_peered":"2024-03-08T06:00:36.186068+0000","last_unstale":"2024-03-08T06:00:43.251609+0000","last_undegraded":"2024-03-08T06:00:43.251609+0000","last_fullsized":"2024-03-08T06:00:43.251609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:39:59.986300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205158+0000","last_change":"2024-03-08T06:00:36.186759+0000","last_active":"2024-03-08T06:00:42.205158+0000","last_peered":"2024-03-08T06:00:42.205158+0000","last_clean":"2024-03-08T06:00:42.205158+0000","last_became_active":"2024-03-08T06:00:36.186338+0000","last_became_peered":"2024-03-08T06:00:36.186338+0000","last_unstale":"2024-03-08T06:00:42.205158+0000","last_undegraded":"2024-03-08T06:00:42.205158+0000","last_fullsized":"2024-03-08T06:00:42.205158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:40:16.053974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251835+0000","last_change":"2024-03-08T06:00:36.197002+0000","last_active":"2024-03-08T06:00:43.251835+0000","last_peered":"2024-03-08T06:00:43.251835+0000","last_clean":"2024-03-08T06:00:43.251835+0000","last_became_active":"2024-03-08T06:00:36.196750+0000","last_became_peered":"2024-03-08T06:00:36.196750+0000","last_unstale":"2024-03-08T06:00:43.251835+0000","last_undegraded":"2024-03-08T06:00:43.251835+0000","last_fullsized":"2024-03-08T06:00:43.251835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:45:22.429641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251708+0000","last_change":"2024-03-08T06:00:36.184956+0000","last_active":"2024-03-08T06:00:43.251708+0000","last_peered":"2024-03-08T06:00:43.251708+0000","last_clean":"2024-03-08T06:00:43.251708+0000","last_became_active":"2024-03-08T06:00:36.184694+0000","last_became_peered":"2024-03-08T06:00:36.184694+0000","last_unstale":"2024-03-08T06:00:43.251708+0000","last_undegraded":"2024-03-08T06:00:43.251708+0000","last_fullsized":"2024-03-08T06:00:43.251708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:41:02.118287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254526+0000","last_change":"2024-03-08T06:00:36.187667+0000","last_active":"2024-03-08T06:00:43.254526+0000","last_peered":"2024-03-08T06:00:43.254526+0000","last_clean":"2024-03-08T06:00:43.254526+0000","last_became_active":"2024-03-08T06:00:36.187407+0000","last_became_peered":"2024-03-08T06:00:36.187407+0000","last_unstale":"2024-03-08T06:00:43.254526+0000","last_undegraded":"2024-03-08T06:00:43.254526+0000","last_fullsized":"2024-03-08T06:00:43.254526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:10:17.180499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752121+0000","last_change":"2024-03-08T06:00:36.198182+0000","last_active":"2024-03-08T06:00:42.752121+0000","last_peered":"2024-03-08T06:00:42.752121+0000","last_clean":"2024-03-08T06:00:42.752121+0000","last_became_active":"2024-03-08T06:00:36.197894+0000","last_became_peered":"2024-03-08T06:00:36.197894+0000","last_unstale":"2024-03-08T06:00:42.752121+0000","last_undegraded":"2024-03-08T06:00:42.752121+0000","last_fullsized":"2024-03-08T06:00:42.752121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:02.331109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.253403+0000","last_change":"2024-03-08T06:00:36.180664+0000","last_active":"2024-03-08T06:00:43.253403+0000","last_peered":"2024-03-08T06:00:43.253403+0000","last_clean":"2024-03-08T06:00:43.253403+0000","last_became_active":"2024-03-08T06:00:36.180389+0000","last_became_peered":"2024-03-08T06:00:36.180389+0000","last_unstale":"2024-03-08T06:00:43.253403+0000","last_undegraded":"2024-03-08T06:00:43.253403+0000","last_fullsized":"2024-03-08T06:00:43.253403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:58:34.120247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252430+0000","last_change":"2024-03-08T06:00:36.189412+0000","last_active":"2024-03-08T06:00:43.252430+0000","last_peered":"2024-03-08T06:00:43.252430+0000","last_clean":"2024-03-08T06:00:43.252430+0000","last_became_active":"2024-03-08T06:00:36.187130+0000","last_became_peered":"2024-03-08T06:00:36.187130+0000","last_unstale":"2024-03-08T06:00:43.252430+0000","last_undegraded":"2024-03-08T06:00:43.252430+0000","last_fullsized":"2024-03-08T06:00:43.252430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:05.458469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251511+0000","last_change":"2024-03-08T06:00:36.186949+0000","last_active":"2024-03-08T06:00:43.251511+0000","last_peered":"2024-03-08T06:00:43.251511+0000","last_clean":"2024-03-08T06:00:43.251511+0000","last_became_active":"2024-03-08T06:00:36.186386+0000","last_became_peered":"2024-03-08T06:00:36.186386+0000","last_unstale":"2024-03-08T06:00:43.251511+0000","last_undegraded":"2024-03-08T06:00:43.251511+0000","last_fullsized":"2024-03-08T06:00:43.251511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251280+0000","last_change":"2024-03-08T06:00:36.178773+0000","last_active":"2024-03-08T06:00:43.251280+0000","last_peered":"2024-03-08T06:00:43.251280+0000","last_clean":"2024-03-08T06:00:43.251280+0000","last_became_active":"2024-03-08T06:00:36.178515+0000","last_became_peered":"2024-03-08T06:00:36.178515+0000","last_unstale":"2024-03-08T06:00:43.251280+0000","last_undegraded":"2024-03-08T06:00:43.251280+0000","last_fullsized":"2024-03-08T06:00:43.251280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:11:34.950063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251635+0000","last_change":"2024-03-08T06:00:36.179334+0000","last_active":"2024-03-08T06:00:43.251635+0000","last_peered":"2024-03-08T06:00:43.251635+0000","last_clean":"2024-03-08T06:00:43.251635+0000","last_became_active":"2024-03-08T06:00:36.179056+0000","last_became_peered":"2024-03-08T06:00:36.179056+0000","last_unstale":"2024-03-08T06:00:43.251635+0000","last_undegraded":"2024-03-08T06:00:43.251635+0000","last_fullsized":"2024-03-08T06:00:43.251635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:08:13.003781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.205268+0000","last_change":"2024-03-08T06:00:36.182864+0000","last_active":"2024-03-08T06:00:42.205268+0000","last_peered":"2024-03-08T06:00:42.205268+0000","last_clean":"2024-03-08T06:00:42.205268+0000","last_became_active":"2024-03-08T06:00:36.182290+0000","last_became_peered":"2024-03-08T06:00:36.182290+0000","last_unstale":"2024-03-08T06:00:42.205268+0000","last_undegraded":"2024-03-08T06:00:42.205268+0000","last_fullsized":"2024-03-08T06:00:42.205268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.206446+0000","last_change":"2024-03-08T06:00:36.184044+0000","last_active":"2024-03-08T06:00:42.206446+0000","last_peered":"2024-03-08T06:00:42.206446+0000","last_clean":"2024-03-08T06:00:42.206446+0000","last_became_active":"2024-03-08T06:00:36.183745+0000","last_became_peered":"2024-03-08T06:00:36.183745+0000","last_unstale":"2024-03-08T06:00:42.206446+0000","last_undegraded":"2024-03-08T06:00:42.206446+0000","last_fullsized":"2024-03-08T06:00:42.206446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:58:34.120247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254652+0000","last_change":"2024-03-08T06:00:36.200379+0000","last_active":"2024-03-08T06:00:43.254652+0000","last_peered":"2024-03-08T06:00:43.254652+0000","last_clean":"2024-03-08T06:00:43.254652+0000","last_became_active":"2024-03-08T06:00:36.200023+0000","last_became_peered":"2024-03-08T06:00:36.200023+0000","last_unstale":"2024-03-08T06:00:43.254652+0000","last_undegraded":"2024-03-08T06:00:43.254652+0000","last_fullsized":"2024-03-08T06:00:43.254652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:40:16.053974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.254844+0000","last_change":"2024-03-08T06:00:36.200329+0000","last_active":"2024-03-08T06:00:43.254844+0000","last_peered":"2024-03-08T06:00:43.254844+0000","last_clean":"2024-03-08T06:00:43.254844+0000","last_became_active":"2024-03-08T06:00:36.200069+0000","last_became_peered":"2024-03-08T06:00:36.200069+0000","last_unstale":"2024-03-08T06:00:43.254844+0000","last_undegraded":"2024-03-08T06:00:43.254844+0000","last_fullsized":"2024-03-08T06:00:43.254844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:12:38.376146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751472+0000","last_change":"2024-03-08T06:00:36.205202+0000","last_active":"2024-03-08T06:00:42.751472+0000","last_peered":"2024-03-08T06:00:42.751472+0000","last_clean":"2024-03-08T06:00:42.751472+0000","last_became_active":"2024-03-08T06:00:36.204866+0000","last_became_peered":"2024-03-08T06:00:36.204866+0000","last_unstale":"2024-03-08T06:00:42.751472+0000","last_undegraded":"2024-03-08T06:00:42.751472+0000","last_fullsized":"2024-03-08T06:00:42.751472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:44:44.951176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.204931+0000","last_change":"2024-03-08T06:00:36.188976+0000","last_active":"2024-03-08T06:00:42.204931+0000","last_peered":"2024-03-08T06:00:42.204931+0000","last_clean":"2024-03-08T06:00:42.204931+0000","last_became_active":"2024-03-08T06:00:36.188717+0000","last_became_peered":"2024-03-08T06:00:36.188717+0000","last_unstale":"2024-03-08T06:00:42.204931+0000","last_undegraded":"2024-03-08T06:00:42.204931+0000","last_fullsized":"2024-03-08T06:00:42.204931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:11:34.950063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751727+0000","last_change":"2024-03-08T06:00:36.197547+0000","last_active":"2024-03-08T06:00:42.751727+0000","last_peered":"2024-03-08T06:00:42.751727+0000","last_clean":"2024-03-08T06:00:42.751727+0000","last_became_active":"2024-03-08T06:00:36.197295+0000","last_became_peered":"2024-03-08T06:00:36.197295+0000","last_unstale":"2024-03-08T06:00:42.751727+0000","last_undegraded":"2024-03-08T06:00:42.751727+0000","last_fullsized":"2024-03-08T06:00:42.751727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:39:59.986300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251773+0000","last_change":"2024-03-08T06:00:36.191642+0000","last_active":"2024-03-08T06:00:43.251773+0000","last_peered":"2024-03-08T06:00:43.251773+0000","last_clean":"2024-03-08T06:00:43.251773+0000","last_became_active":"2024-03-08T06:00:36.191242+0000","last_became_peered":"2024-03-08T06:00:36.191242+0000","last_unstale":"2024-03-08T06:00:43.251773+0000","last_undegraded":"2024-03-08T06:00:43.251773+0000","last_fullsized":"2024-03-08T06:00:43.251773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.751820+0000","last_change":"2024-03-08T06:00:36.197506+0000","last_active":"2024-03-08T06:00:42.751820+0000","last_peered":"2024-03-08T06:00:42.751820+0000","last_clean":"2024-03-08T06:00:42.751820+0000","last_became_active":"2024-03-08T06:00:36.197208+0000","last_became_peered":"2024-03-08T06:00:36.197208+0000","last_unstale":"2024-03-08T06:00:42.751820+0000","last_undegraded":"2024-03-08T06:00:42.751820+0000","last_fullsized":"2024-03-08T06:00:42.751820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:59:39.518589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252005+0000","last_change":"2024-03-08T06:00:36.186669+0000","last_active":"2024-03-08T06:00:43.252005+0000","last_peered":"2024-03-08T06:00:43.252005+0000","last_clean":"2024-03-08T06:00:43.252005+0000","last_became_active":"2024-03-08T06:00:36.186417+0000","last_became_peered":"2024-03-08T06:00:36.186417+0000","last_unstale":"2024-03-08T06:00:43.252005+0000","last_undegraded":"2024-03-08T06:00:43.252005+0000","last_fullsized":"2024-03-08T06:00:43.252005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:59:39.518589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.251432+0000","last_change":"2024-03-08T06:00:36.186124+0000","last_active":"2024-03-08T06:00:43.251432+0000","last_peered":"2024-03-08T06:00:43.251432+0000","last_clean":"2024-03-08T06:00:43.251432+0000","last_became_active":"2024-03-08T06:00:36.185808+0000","last_became_peered":"2024-03-08T06:00:36.185808+0000","last_unstale":"2024-03-08T06:00:43.251432+0000","last_undegraded":"2024-03-08T06:00:43.251432+0000","last_fullsized":"2024-03-08T06:00:43.251432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:01:37.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:43.252630+0000","last_change":"2024-03-08T06:00:36.183150+0000","last_active":"2024-03-08T06:00:43.252630+0000","last_peered":"2024-03-08T06:00:43.252630+0000","last_clean":"2024-03-08T06:00:43.252630+0000","last_became_active":"2024-03-08T06:00:36.182716+0000","last_became_peered":"2024-03-08T06:00:36.182716+0000","last_unstale":"2024-03-08T06:00:43.252630+0000","last_undegraded":"2024-03-08T06:00:43.252630+0000","last_fullsized":"2024-03-08T06:00:43.252630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:32:04.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-08T06:00:42.752346+0000","last_change":"2024-03-08T06:00:36.198478+0000","last_active":"2024-03-08T06:00:42.752346+0000","last_peered":"2024-03-08T06:00:42.752346+0000","last_clean":"2024-03-08T06:00:42.752346+0000","last_became_active":"2024-03-08T06:00:36.198218+0000","last_became_peered":"2024-03-08T06:00:36.198218+0000","last_unstale":"2024-03-08T06:00:42.752346+0000","last_undegraded":"2024-03-08T06:00:42.752346+0000","last_fullsized":"2024-03-08T06:00:42.752346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:35.146779+0000","last_clean_scrub_stamp":"2024-03-08T06:00:35.146779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:19:26.058130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":876544,"data_stored":703400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":116292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":112540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":112576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":116292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":112450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":112610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":703456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:01:03.858 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-08T06:01:03.858 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-08T06:01:03.858 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-08T06:01:03.858 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-08T06:01:03.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.993+0000 7f775f45c640 1 Processor -- start 2024-03-08T06:01:03.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.993+0000 7f775f45c640 1 -- start start 2024-03-08T06:01:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f775f45c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f77580b6d00 0x7f77580d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f775f45c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77580cf180 con 0x7f775815dde0 2024-03-08T06:01:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f775f45c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77580d1ed0 con 0x7f77580cec10 2024-03-08T06:01:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f775f45c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77580d21b0 con 0x7f77580b6d00 2024-03-08T06:01:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f775d1f8640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f775815dde0 0x7f7758162250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57634/0 (socket says 172.21.15.76:57634) 2024-03-08T06:01:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f775d1f8640 1 -- 172.21.15.76:0/2226927240 learned_addr learned my addr 172.21.15.76:0/2226927240 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f774ffff640 1 -- 172.21.15.76:0/2226927240 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826626172 0 0) 0x7f77580cf180 con 0x7f775815dde0 2024-03-08T06:01:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f774ffff640 1 -- 172.21.15.76:0/2226927240 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7744003520 con 0x7f775815dde0 2024-03-08T06:01:03.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f775c9f7640 1 --2- 172.21.15.76:0/2226927240 >> v2:172.21.15.81:3300/0 conn(0x7f77580b6d00 0x7f77580d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:03.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f774ffff640 1 -- 172.21.15.76:0/2226927240 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2833816943 0 0) 0x7f7744003520 con 0x7f775815dde0 2024-03-08T06:01:03.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f774ffff640 1 -- 172.21.15.76:0/2226927240 >> v2:172.21.15.81:3300/0 conn(0x7f77580b6d00 msgr2=0x7f77580d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:03.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f774ffff640 1 --2- 172.21.15.76:0/2226927240 >> v2:172.21.15.81:3300/0 conn(0x7f77580b6d00 0x7f77580d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f774ffff640 1 -- 172.21.15.76:0/2226927240 >> v1:172.21.15.81:6789/0 conn(0x7f77580cec10 legacy=0x7f77580d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f774ffff640 1 -- 172.21.15.76:0/2226927240 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77580d09c0 con 0x7f775815dde0 2024-03-08T06:01:03.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f774ffff640 1 -- 172.21.15.76:0/2226927240 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f7754001670 con 0x7f775815dde0 2024-03-08T06:01:03.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f774ffff640 1 -- 172.21.15.76:0/2226927240 >> v1:172.21.15.76:6789/0 conn(0x7f775815dde0 legacy=0x7f7758162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:03.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f774ffff640 1 --2- 172.21.15.76:0/2226927240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7744003a70 0x7f7744003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:03.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f774ffff640 1 --2- 172.21.15.76:0/2226927240 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7744004800 0x7f7744004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:03.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f774ffff640 1 --2- 172.21.15.76:0/2226927240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77440053c0 0x7f774400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:03.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f774ffff640 1 -- 172.21.15.76:0/2226927240 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77580d21b0 con 0x7f77440053c0 2024-03-08T06:01:03.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f774ffff640 1 -- 172.21.15.76:0/2226927240 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774400c260 con 0x7f7744003a70 2024-03-08T06:01:03.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.994+0000 7f774ffff640 1 -- 172.21.15.76:0/2226927240 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774400c460 con 0x7f7744004800 2024-03-08T06:01:03.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.995+0000 7f775d1f8640 1 --2- 172.21.15.76:0/2226927240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7744003a70 0x7f7744003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:03.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.995+0000 7f775d9f9640 1 --2- 172.21.15.76:0/2226927240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77440053c0 0x7f774400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:03.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.995+0000 7f775c9f7640 1 --2- 172.21.15.76:0/2226927240 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7744004800 0x7f7744004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:03.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.995+0000 7f775d9f9640 1 -- 172.21.15.76:0/2226927240 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7744004800 msgr2=0x7f7744004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:03.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.995+0000 7f775d9f9640 1 --2- 172.21.15.76:0/2226927240 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7744004800 0x7f7744004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.995+0000 7f775d9f9640 1 -- 172.21.15.76:0/2226927240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7744003a70 msgr2=0x7f7744003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:03.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.995+0000 7f775d9f9640 1 --2- 172.21.15.76:0/2226927240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7744003a70 0x7f7744003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:03.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.995+0000 7f775d9f9640 1 -- 172.21.15.76:0/2226927240 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7744003520 con 0x7f77440053c0 2024-03-08T06:01:03.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.995+0000 7f775d1f8640 1 --2- 172.21.15.76:0/2226927240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7744003a70 0x7f7744003e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:03.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.996+0000 7f775d9f9640 1 -- 172.21.15.76:0/2226927240 reap_dead start 2024-03-08T06:01:03.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.996+0000 7f775d9f9640 1 --2- 172.21.15.76:0/2226927240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77440053c0 0x7f774400b9c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7750001e50 tx=0x7f7744005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a827efd90e23ed server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:03.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.996+0000 7f774ffff640 1 -- 172.21.15.76:0/2226927240 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754007c60 con 0x7f77440053c0 2024-03-08T06:01:03.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.996+0000 7f774ffff640 1 -- 172.21.15.76:0/2226927240 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7750003bb0 con 0x7f77440053c0 2024-03-08T06:01:04.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.996+0000 7f774ffff640 1 -- 172.21.15.76:0/2226927240 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750001a70 con 0x7f77440053c0 2024-03-08T06:01:04.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.996+0000 7f775f45c640 1 -- 172.21.15.76:0/2226927240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77440053c0 msgr2=0x7f774400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.996+0000 7f775f45c640 1 --2- 172.21.15.76:0/2226927240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77440053c0 0x7f774400b9c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7750001e50 tx=0x7f7744005830 comp rx=0 tx=0).stop 2024-03-08T06:01:04.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.996+0000 7f775f45c640 1 -- 172.21.15.76:0/2226927240 shutdown_connections 2024-03-08T06:01:04.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.996+0000 7f775f45c640 1 --2- 172.21.15.76:0/2226927240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77440053c0 0x7f774400b9c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.996+0000 7f775f45c640 1 -- 172.21.15.76:0/2226927240 >> 172.21.15.76:0/2226927240 conn(0x7f77580b56e0 msgr2=0x7f77580b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:04.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.996+0000 7f775f45c640 1 -- 172.21.15.76:0/2226927240 shutdown_connections 2024-03-08T06:01:04.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.996+0000 7f775f45c640 1 -- 172.21.15.76:0/2226927240 wait complete. 2024-03-08T06:01:04.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.998+0000 7f775f45c640 1 Processor -- start 2024-03-08T06:01:04.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.998+0000 7f775f45c640 1 -- start start 2024-03-08T06:01:04.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.998+0000 7f775f45c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77440053c0 0x7f77581512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.998+0000 7f775f45c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7758151af0 0x7f775815a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.998+0000 7f775f45c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7758152020 0x7f775815aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.998+0000 7f775f45c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77580d0480 con 0x7f77440053c0 2024-03-08T06:01:04.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.998+0000 7f775f45c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77580d0da0 con 0x7f7758152020 2024-03-08T06:01:04.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.998+0000 7f775f45c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77580d0600 con 0x7f7758151af0 2024-03-08T06:01:04.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.998+0000 7f775d1f8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77440053c0 0x7f77581512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.998+0000 7f775d1f8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77440053c0 0x7f77581512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51694/0 (socket says 172.21.15.76:51694) 2024-03-08T06:01:04.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.998+0000 7f775d1f8640 1 -- 172.21.15.76:0/807675743 learned_addr learned my addr 172.21.15.76:0/807675743 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:04.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.999+0000 7f775c9f7640 1 --2- 172.21.15.76:0/807675743 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7758151af0 0x7f775815a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.999+0000 7f775d9f9640 1 --2- 172.21.15.76:0/807675743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7758152020 0x7f775815aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.999+0000 7f775d1f8640 1 -- 172.21.15.76:0/807675743 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7758151af0 msgr2=0x7f775815a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.999+0000 7f775d1f8640 1 --2- 172.21.15.76:0/807675743 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7758151af0 0x7f775815a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.999+0000 7f775d1f8640 1 -- 172.21.15.76:0/807675743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7758152020 msgr2=0x7f775815aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.999+0000 7f775d1f8640 1 --2- 172.21.15.76:0/807675743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7758152020 0x7f775815aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.999+0000 7f775d1f8640 1 -- 172.21.15.76:0/807675743 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77580c7a70 con 0x7f77440053c0 2024-03-08T06:01:04.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.999+0000 7f775c9f7640 1 --2- 172.21.15.76:0/807675743 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7758151af0 0x7f775815a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:04.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.999+0000 7f775d1f8640 1 --2- 172.21.15.76:0/807675743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77440053c0 0x7f77581512d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7754000fb0 tx=0x7f7754007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:04.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.999+0000 7f774dffb640 1 -- 172.21.15.76:0/807675743 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775400f730 con 0x7f77440053c0 2024-03-08T06:01:04.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.999+0000 7f774dffb640 1 -- 172.21.15.76:0/807675743 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7754018070 con 0x7f77440053c0 2024-03-08T06:01:04.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.999+0000 7f775f45c640 1 -- 172.21.15.76:0/807675743 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77580c7d50 con 0x7f77440053c0 2024-03-08T06:01:04.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.999+0000 7f774dffb640 1 -- 172.21.15.76:0/807675743 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775400faf0 con 0x7f77440053c0 2024-03-08T06:01:04.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:03.999+0000 7f775f45c640 1 -- 172.21.15.76:0/807675743 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77580c8310 con 0x7f77440053c0 2024-03-08T06:01:04.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.000+0000 7f775f45c640 1 -- 172.21.15.76:0/807675743 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77580b5340 con 0x7f77440053c0 2024-03-08T06:01:04.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.000+0000 7f774dffb640 1 -- 172.21.15.76:0/807675743 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f775400fc90 con 0x7f77440053c0 2024-03-08T06:01:04.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.001+0000 7f774dffb640 1 --2- 172.21.15.76:0/807675743 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f773c06b4d0 0x7f773c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.001+0000 7f774dffb640 1 -- 172.21.15.76:0/807675743 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f775408a590 con 0x7f77440053c0 2024-03-08T06:01:04.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.001+0000 7f775c9f7640 1 --2- 172.21.15.76:0/807675743 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f773c06b4d0 0x7f773c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.001+0000 7f775c9f7640 1 --2- 172.21.15.76:0/807675743 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f773c06b4d0 0x7f773c06d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f775815b6d0 tx=0x7f7748008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:04.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.005+0000 7f774dffb640 1 -- 172.21.15.76:0/807675743 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f77580b5340 con 0x7f77440053c0 2024-03-08T06:01:04.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.212+0000 7f775f45c640 1 -- 172.21.15.76:0/807675743 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f77580728a0 con 0x7f77440053c0 2024-03-08T06:01:04.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.212+0000 7f774dffb640 1 -- 172.21.15.76:0/807675743 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f77580728a0 con 0x7f77440053c0 2024-03-08T06:01:04.213 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:04.213 INFO:teuthology.orchestra.run.smithi076.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-08T06:01:04.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.214+0000 7f775f45c640 1 -- 172.21.15.76:0/807675743 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f773c06b4d0 msgr2=0x7f773c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.214+0000 7f775f45c640 1 --2- 172.21.15.76:0/807675743 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f773c06b4d0 0x7f773c06d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f775815b6d0 tx=0x7f7748008040 comp rx=0 tx=0).stop 2024-03-08T06:01:04.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.214+0000 7f775f45c640 1 -- 172.21.15.76:0/807675743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77440053c0 msgr2=0x7f77581512d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.214+0000 7f775f45c640 1 --2- 172.21.15.76:0/807675743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77440053c0 0x7f77581512d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7754000fb0 tx=0x7f7754007430 comp rx=0 tx=0).stop 2024-03-08T06:01:04.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.214+0000 7f775f45c640 1 -- 172.21.15.76:0/807675743 shutdown_connections 2024-03-08T06:01:04.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.214+0000 7f775f45c640 1 --2- 172.21.15.76:0/807675743 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f773c06b4d0 0x7f773c06d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.214+0000 7f775f45c640 1 --2- 172.21.15.76:0/807675743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7758152020 0x7f775815aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.214+0000 7f775f45c640 1 --2- 172.21.15.76:0/807675743 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7758151af0 0x7f775815a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.214+0000 7f775f45c640 1 --2- 172.21.15.76:0/807675743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77440053c0 0x7f77581512d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.214+0000 7f775f45c640 1 -- 172.21.15.76:0/807675743 >> 172.21.15.76:0/807675743 conn(0x7f77580b56e0 msgr2=0x7f77580d3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:04.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.214+0000 7f775f45c640 1 -- 172.21.15.76:0/807675743 shutdown_connections 2024-03-08T06:01:04.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.214+0000 7f775f45c640 1 -- 172.21.15.76:0/807675743 wait complete. 2024-03-08T06:01:04.227 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-08T06:01:04.227 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:04.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.327+0000 7fdc1810c640 1 Processor -- start 2024-03-08T06:01:04.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.328+0000 7fdc1810c640 1 -- start start 2024-03-08T06:01:04.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.328+0000 7fdc1810c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdc10071390 0x7fdc100cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.328+0000 7fdc1810c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc100cd310 0x7fdc100cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.328+0000 7fdc1810c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1006d880 con 0x7fdc1006d3f0 2024-03-08T06:01:04.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.328+0000 7fdc1810c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1006e010 con 0x7fdc10071390 2024-03-08T06:01:04.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.328+0000 7fdc1810c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1006e2f0 con 0x7fdc100cd310 2024-03-08T06:01:04.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc156a7640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdc1006d3f0 0x7fdc10071770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57644/0 (socket says 172.21.15.76:57644) 2024-03-08T06:01:04.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc156a7640 1 -- 172.21.15.76:0/2610825363 learned_addr learned my addr 172.21.15.76:0/2610825363 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:04.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc166a9640 1 --2- 172.21.15.76:0/2610825363 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc100cd310 0x7fdc100cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc15ea8640 1 --2- 172.21.15.76:0/2610825363 >> v2:172.21.15.81:3300/0 conn(0x7fdc10071390 0x7fdc100cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc14ea6640 1 -- 172.21.15.76:0/2610825363 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188472969 0 0) 0x7fdc1006d880 con 0x7fdc1006d3f0 2024-03-08T06:01:04.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc14ea6640 1 -- 172.21.15.76:0/2610825363 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbfc003520 con 0x7fdc1006d3f0 2024-03-08T06:01:04.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc14ea6640 1 -- 172.21.15.76:0/2610825363 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4175067625 0 0) 0x7fdbfc003520 con 0x7fdc1006d3f0 2024-03-08T06:01:04.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc14ea6640 1 -- 172.21.15.76:0/2610825363 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc100cd310 msgr2=0x7fdc100cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:04.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc14ea6640 1 --2- 172.21.15.76:0/2610825363 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc100cd310 0x7fdc100cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc14ea6640 1 -- 172.21.15.76:0/2610825363 >> v2:172.21.15.81:3300/0 conn(0x7fdc10071390 msgr2=0x7fdc100cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:04.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc14ea6640 1 --2- 172.21.15.76:0/2610825363 >> v2:172.21.15.81:3300/0 conn(0x7fdc10071390 0x7fdc100cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc14ea6640 1 -- 172.21.15.76:0/2610825363 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc100ce7b0 con 0x7fdc1006d3f0 2024-03-08T06:01:04.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc14ea6640 1 -- 172.21.15.76:0/2610825363 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fdc00001670 con 0x7fdc1006d3f0 2024-03-08T06:01:04.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc14ea6640 1 -- 172.21.15.76:0/2610825363 >> v1:172.21.15.76:6789/0 conn(0x7fdc1006d3f0 legacy=0x7fdc10071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc14ea6640 1 --2- 172.21.15.76:0/2610825363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbfc003a20 0x7fdbfc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc14ea6640 1 --2- 172.21.15.76:0/2610825363 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc100cd310 0x7fdbfc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc14ea6640 1 --2- 172.21.15.76:0/2610825363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbfc004d40 0x7fdbfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc14ea6640 1 -- 172.21.15.76:0/2610825363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1006e010 con 0x7fdbfc004d40 2024-03-08T06:01:04.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc14ea6640 1 -- 172.21.15.76:0/2610825363 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1006e2f0 con 0x7fdbfc003a20 2024-03-08T06:01:04.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc14ea6640 1 -- 172.21.15.76:0/2610825363 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc00b9c0 con 0x7fdc100cd310 2024-03-08T06:01:04.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc166a9640 1 --2- 172.21.15.76:0/2610825363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbfc004d40 0x7fdbfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc15ea8640 1 --2- 172.21.15.76:0/2610825363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbfc003a20 0x7fdbfc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc156a7640 1 --2- 172.21.15.76:0/2610825363 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc100cd310 0x7fdbfc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.329+0000 7fdc166a9640 1 -- 172.21.15.76:0/2610825363 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc100cd310 msgr2=0x7fdbfc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:04.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.330+0000 7fdc166a9640 1 --2- 172.21.15.76:0/2610825363 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc100cd310 0x7fdbfc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.330+0000 7fdc166a9640 1 -- 172.21.15.76:0/2610825363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbfc003a20 msgr2=0x7fdbfc003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:04.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.330+0000 7fdc166a9640 1 --2- 172.21.15.76:0/2610825363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbfc003a20 0x7fdbfc003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.330+0000 7fdc166a9640 1 -- 172.21.15.76:0/2610825363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc003520 con 0x7fdbfc004d40 2024-03-08T06:01:04.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.330+0000 7fdc15ea8640 1 --2- 172.21.15.76:0/2610825363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbfc003a20 0x7fdbfc003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:04.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.330+0000 7fdc166a9640 1 --2- 172.21.15.76:0/2610825363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbfc004d40 0x7fdbfc0051c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009fc0 tx=0x7fdbfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e909c31d5be72f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:04.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.330+0000 7fdc14ea6640 1 -- 172.21.15.76:0/2610825363 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08039070 con 0x7fdbfc004d40 2024-03-08T06:01:04.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.330+0000 7fdc14ea6640 1 -- 172.21.15.76:0/2610825363 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0802cc20 con 0x7fdbfc004d40 2024-03-08T06:01:04.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.330+0000 7fdc14ea6640 1 -- 172.21.15.76:0/2610825363 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08009c10 con 0x7fdbfc004d40 2024-03-08T06:01:04.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.331+0000 7fdc1810c640 1 -- 172.21.15.76:0/2610825363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbfc004d40 msgr2=0x7fdbfc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.331+0000 7fdc1810c640 1 --2- 172.21.15.76:0/2610825363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbfc004d40 0x7fdbfc0051c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009fc0 tx=0x7fdbfc005830 comp rx=0 tx=0).stop 2024-03-08T06:01:04.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.331+0000 7fdc166a9640 1 -- 172.21.15.76:0/2610825363 reap_dead start 2024-03-08T06:01:04.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.331+0000 7fdc1810c640 1 -- 172.21.15.76:0/2610825363 shutdown_connections 2024-03-08T06:01:04.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.331+0000 7fdc1810c640 1 -- 172.21.15.76:0/2610825363 >> 172.21.15.76:0/2610825363 conn(0x7fdc100cf740 msgr2=0x7fdc100cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:04.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.331+0000 7fdc1810c640 1 -- 172.21.15.76:0/2610825363 shutdown_connections 2024-03-08T06:01:04.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.331+0000 7fdc1810c640 1 -- 172.21.15.76:0/2610825363 wait complete. 2024-03-08T06:01:04.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.332+0000 7fdc1810c640 1 Processor -- start 2024-03-08T06:01:04.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.332+0000 7fdc1810c640 1 -- start start 2024-03-08T06:01:04.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc1810c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbfc004d40 0x7fdc10158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc1810c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc100c4e70 0x7fdc100c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc1810c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc100c7bf0 0x7fdc101f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc1810c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1015c060 con 0x7fdc100c4e70 2024-03-08T06:01:04.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc1810c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1015bee0 con 0x7fdbfc004d40 2024-03-08T06:01:04.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc1810c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc100cd7c0 con 0x7fdc100c7bf0 2024-03-08T06:01:04.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc156a7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc100c4e70 0x7fdc100c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc156a7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc100c4e70 0x7fdc100c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51712/0 (socket says 172.21.15.76:51712) 2024-03-08T06:01:04.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc156a7640 1 -- 172.21.15.76:0/629043145 learned_addr learned my addr 172.21.15.76:0/629043145 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:04.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc15ea8640 1 --2- 172.21.15.76:0/629043145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbfc004d40 0x7fdc10158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc166a9640 1 --2- 172.21.15.76:0/629043145 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc100c7bf0 0x7fdc101f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc156a7640 1 -- 172.21.15.76:0/629043145 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc100c7bf0 msgr2=0x7fdc101f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc156a7640 1 --2- 172.21.15.76:0/629043145 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc100c7bf0 0x7fdc101f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc156a7640 1 -- 172.21.15.76:0/629043145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbfc004d40 msgr2=0x7fdc10158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc156a7640 1 --2- 172.21.15.76:0/629043145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbfc004d40 0x7fdc10158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc156a7640 1 -- 172.21.15.76:0/629043145 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc100c6680 con 0x7fdc100c4e70 2024-03-08T06:01:04.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc15ea8640 1 --2- 172.21.15.76:0/629043145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbfc004d40 0x7fdc10158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:04.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.333+0000 7fdc156a7640 1 --2- 172.21.15.76:0/629043145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc100c4e70 0x7fdc100c76b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdc000017e0 tx=0x7fdc000077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:04.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.334+0000 7fdc067fc640 1 -- 172.21.15.76:0/629043145 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000f9c0 con 0x7fdc100c4e70 2024-03-08T06:01:04.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.334+0000 7fdc067fc640 1 -- 172.21.15.76:0/629043145 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc00018070 con 0x7fdc100c4e70 2024-03-08T06:01:04.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.334+0000 7fdc1810c640 1 -- 172.21.15.76:0/629043145 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc100c6960 con 0x7fdc100c4e70 2024-03-08T06:01:04.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.334+0000 7fdc067fc640 1 -- 172.21.15.76:0/629043145 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000fd80 con 0x7fdc100c4e70 2024-03-08T06:01:04.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.334+0000 7fdc1810c640 1 -- 172.21.15.76:0/629043145 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc101e65c0 con 0x7fdc100c4e70 2024-03-08T06:01:04.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.335+0000 7fdc1810c640 1 -- 172.21.15.76:0/629043145 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1004f030 con 0x7fdc100c4e70 2024-03-08T06:01:04.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.335+0000 7fdc067fc640 1 -- 172.21.15.76:0/629043145 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc0001d460 con 0x7fdc100c4e70 2024-03-08T06:01:04.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.335+0000 7fdc067fc640 1 --2- 172.21.15.76:0/629043145 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdbf406b4d0 0x7fdbf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.335+0000 7fdc067fc640 1 -- 172.21.15.76:0/629043145 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdc0008a750 con 0x7fdc100c4e70 2024-03-08T06:01:04.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.336+0000 7fdc15ea8640 1 --2- 172.21.15.76:0/629043145 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdbf406b4d0 0x7fdbf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.336+0000 7fdc15ea8640 1 --2- 172.21.15.76:0/629043145 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdbf406b4d0 0x7fdbf406d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009d60 tx=0x7fdc0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:04.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.340+0000 7fdc067fc640 1 -- 172.21.15.76:0/629043145 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc00059290 con 0x7fdc100c4e70 2024-03-08T06:01:04.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.512+0000 7fdc1810c640 1 -- 172.21.15.76:0/629043145 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc100c7fd0 con 0x7fdc100c4e70 2024-03-08T06:01:04.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.513+0000 7fdc067fc640 1 -- 172.21.15.76:0/629043145 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18430 (secure 0 0 0) 0x7fdc00058b90 con 0x7fdc100c4e70 2024-03-08T06:01:04.514 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:04.514 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":26,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:42.202011+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:04.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.515+0000 7fdc1810c640 1 -- 172.21.15.76:0/629043145 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdbf406b4d0 msgr2=0x7fdbf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.515+0000 7fdc1810c640 1 --2- 172.21.15.76:0/629043145 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdbf406b4d0 0x7fdbf406d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009d60 tx=0x7fdc0c008040 comp rx=0 tx=0).stop 2024-03-08T06:01:04.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.516+0000 7fdc1810c640 1 -- 172.21.15.76:0/629043145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc100c4e70 msgr2=0x7fdc100c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.516+0000 7fdc1810c640 1 --2- 172.21.15.76:0/629043145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc100c4e70 0x7fdc100c76b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdc000017e0 tx=0x7fdc000077a0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.516+0000 7fdc1810c640 1 -- 172.21.15.76:0/629043145 shutdown_connections 2024-03-08T06:01:04.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.516+0000 7fdc1810c640 1 --2- 172.21.15.76:0/629043145 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdbf406b4d0 0x7fdbf406d990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.516+0000 7fdc1810c640 1 --2- 172.21.15.76:0/629043145 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc100c7bf0 0x7fdc101f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.516+0000 7fdc1810c640 1 --2- 172.21.15.76:0/629043145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc100c4e70 0x7fdc100c76b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.516+0000 7fdc1810c640 1 --2- 172.21.15.76:0/629043145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbfc004d40 0x7fdc10158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.516+0000 7fdc1810c640 1 -- 172.21.15.76:0/629043145 >> 172.21.15.76:0/629043145 conn(0x7fdc100cf740 msgr2=0x7fdc1015ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:04.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.516+0000 7fdc1810c640 1 -- 172.21.15.76:0/629043145 shutdown_connections 2024-03-08T06:01:04.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.516+0000 7fdc1810c640 1 -- 172.21.15.76:0/629043145 wait complete. 2024-03-08T06:01:04.527 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:00:33.926651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:00:34.474030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T06:00:38.207853+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T06:01:04.528 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:01:04.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.630+0000 7fba7d26b640 1 Processor -- start 2024-03-08T06:01:04.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.630+0000 7fba7d26b640 1 -- start start 2024-03-08T06:01:04.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.631+0000 7fba7d26b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fba7806ac80 0x7fba7806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.631+0000 7fba7d26b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fba780cfd30 0x7fba780d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.631+0000 7fba7d26b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba780d06a0 con 0x7fba780afb60 2024-03-08T06:01:04.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.631+0000 7fba7d26b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba780d1570 con 0x7fba780cfd30 2024-03-08T06:01:04.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.631+0000 7fba7d26b640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7806fe20 con 0x7fba7806ac80 2024-03-08T06:01:04.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.631+0000 7fba777fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fba780cfd30 0x7fba780d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.631+0000 7fba777fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fba780cfd30 0x7fba780d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51716/0 (socket says 172.21.15.76:51716) 2024-03-08T06:01:04.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.631+0000 7fba777fe640 1 -- 172.21.15.76:0/3548413723 learned_addr learned my addr 172.21.15.76:0/3548413723 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:04.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.631+0000 7fba767fc640 1 --2- 172.21.15.76:0/3548413723 >> v2:172.21.15.81:3300/0 conn(0x7fba7806ac80 0x7fba7806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.632+0000 7fba777fe640 1 -- 172.21.15.76:0/3548413723 >> v2:172.21.15.81:3300/0 conn(0x7fba7806ac80 msgr2=0x7fba7806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:04.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.632+0000 7fba777fe640 1 --2- 172.21.15.76:0/3548413723 >> v2:172.21.15.81:3300/0 conn(0x7fba7806ac80 0x7fba7806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.632+0000 7fba777fe640 1 -- 172.21.15.76:0/3548413723 >> v1:172.21.15.81:6789/0 conn(0x7fba780afb60 legacy=0x7fba780d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.632+0000 7fba777fe640 1 -- 172.21.15.76:0/3548413723 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba78070100 con 0x7fba780cfd30 2024-03-08T06:01:04.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.632+0000 7fba777fe640 1 --2- 172.21.15.76:0/3548413723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba780cfd30 0x7fba780d0160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fba780d0870 tx=0x7fba7002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db05e89455bca994 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:04.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.632+0000 7fba75ffb640 1 -- 172.21.15.76:0/3548413723 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70034070 con 0x7fba780cfd30 2024-03-08T06:01:04.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.633+0000 7fba75ffb640 1 -- 172.21.15.76:0/3548413723 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba70030860 con 0x7fba780cfd30 2024-03-08T06:01:04.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.633+0000 7fba75ffb640 1 -- 172.21.15.76:0/3548413723 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70030c30 con 0x7fba780cfd30 2024-03-08T06:01:04.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.633+0000 7fba7d26b640 1 -- 172.21.15.76:0/3548413723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba780cfd30 msgr2=0x7fba780d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.633+0000 7fba7d26b640 1 --2- 172.21.15.76:0/3548413723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba780cfd30 0x7fba780d0160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fba780d0870 tx=0x7fba7002b9a0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.633+0000 7fba7d26b640 1 -- 172.21.15.76:0/3548413723 shutdown_connections 2024-03-08T06:01:04.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.633+0000 7fba7d26b640 1 --2- 172.21.15.76:0/3548413723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba780cfd30 0x7fba780d0160 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.633+0000 7fba7d26b640 1 --2- 172.21.15.76:0/3548413723 >> v2:172.21.15.81:3300/0 conn(0x7fba7806ac80 0x7fba7806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.633+0000 7fba7d26b640 1 -- 172.21.15.76:0/3548413723 >> 172.21.15.76:0/3548413723 conn(0x7fba78099a60 msgr2=0x7fba78099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:04.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.633+0000 7fba7d26b640 1 -- 172.21.15.76:0/3548413723 shutdown_connections 2024-03-08T06:01:04.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.633+0000 7fba7d26b640 1 -- 172.21.15.76:0/3548413723 wait complete. 2024-03-08T06:01:04.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.634+0000 7fba7d26b640 1 Processor -- start 2024-03-08T06:01:04.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.634+0000 7fba7d26b640 1 -- start start 2024-03-08T06:01:04.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.635+0000 7fba7d26b640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba7806ac80 0x7fba781591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.635+0000 7fba7d26b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba780afb60 0x7fba780b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.635+0000 7fba7d26b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba780cfd30 0x7fba7806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.635+0000 7fba7d26b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7806fb10 con 0x7fba780afb60 2024-03-08T06:01:04.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.635+0000 7fba7d26b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba780d0940 con 0x7fba780cfd30 2024-03-08T06:01:04.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.635+0000 7fba7d26b640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba780aff40 con 0x7fba7806ac80 2024-03-08T06:01:04.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.635+0000 7fba767fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba780afb60 0x7fba780b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.635+0000 7fba767fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba780afb60 0x7fba780b5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51720/0 (socket says 172.21.15.76:51720) 2024-03-08T06:01:04.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.635+0000 7fba767fc640 1 -- 172.21.15.76:0/294277948 learned_addr learned my addr 172.21.15.76:0/294277948 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:04.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.635+0000 7fba777fe640 1 --2- 172.21.15.76:0/294277948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba780cfd30 0x7fba7806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.635+0000 7fba76ffd640 1 --2- 172.21.15.76:0/294277948 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba7806ac80 0x7fba781591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.635+0000 7fba767fc640 1 -- 172.21.15.76:0/294277948 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba7806ac80 msgr2=0x7fba781591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.635+0000 7fba767fc640 1 --2- 172.21.15.76:0/294277948 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba7806ac80 0x7fba781591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.635+0000 7fba767fc640 1 -- 172.21.15.76:0/294277948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba780cfd30 msgr2=0x7fba7806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.635+0000 7fba767fc640 1 --2- 172.21.15.76:0/294277948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba780cfd30 0x7fba7806e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.636+0000 7fba767fc640 1 -- 172.21.15.76:0/294277948 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba780c8d50 con 0x7fba780afb60 2024-03-08T06:01:04.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.636+0000 7fba777fe640 1 --2- 172.21.15.76:0/294277948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba780cfd30 0x7fba7806e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:04.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.636+0000 7fba76ffd640 1 --2- 172.21.15.76:0/294277948 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba7806ac80 0x7fba781591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:04.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.636+0000 7fba767fc640 1 --2- 172.21.15.76:0/294277948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba780afb60 0x7fba780b5bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00e9e0 tx=0x7fba6c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:04.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.636+0000 7fba5f7fe640 1 -- 172.21.15.76:0/294277948 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c017070 con 0x7fba780afb60 2024-03-08T06:01:04.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.636+0000 7fba5f7fe640 1 -- 172.21.15.76:0/294277948 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c00c9d0 con 0x7fba780afb60 2024-03-08T06:01:04.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.636+0000 7fba7d26b640 1 -- 172.21.15.76:0/294277948 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba780c9030 con 0x7fba780afb60 2024-03-08T06:01:04.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.636+0000 7fba5f7fe640 1 -- 172.21.15.76:0/294277948 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00cdc0 con 0x7fba780afb60 2024-03-08T06:01:04.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.636+0000 7fba7d26b640 1 -- 172.21.15.76:0/294277948 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba780c95f0 con 0x7fba780afb60 2024-03-08T06:01:04.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.637+0000 7fba7d26b640 1 -- 172.21.15.76:0/294277948 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba78067d60 con 0x7fba780afb60 2024-03-08T06:01:04.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.637+0000 7fba5f7fe640 1 -- 172.21.15.76:0/294277948 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba6c01a070 con 0x7fba780afb60 2024-03-08T06:01:04.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.638+0000 7fba5f7fe640 1 --2- 172.21.15.76:0/294277948 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.638+0000 7fba5f7fe640 1 -- 172.21.15.76:0/294277948 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fba6c08a8f0 con 0x7fba780afb60 2024-03-08T06:01:04.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.638+0000 7fba76ffd640 1 --2- 172.21.15.76:0/294277948 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.638+0000 7fba76ffd640 1 --2- 172.21.15.76:0/294277948 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fba6406b4d0 0x7fba6406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fba7806f160 tx=0x7fba68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:04.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.642+0000 7fba5f7fe640 1 -- 172.21.15.76:0/294277948 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba78067d60 con 0x7fba780afb60 2024-03-08T06:01:04.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.815+0000 7fba7d26b640 1 -- 172.21.15.76:0/294277948 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba780d0160 con 0x7fba780afb60 2024-03-08T06:01:04.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.815+0000 7fba5f7fe640 1 -- 172.21.15.76:0/294277948 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba780d0160 con 0x7fba780afb60 2024-03-08T06:01:04.815 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:01:04.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.817+0000 7fba7d26b640 1 -- 172.21.15.76:0/294277948 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fba6406b4d0 msgr2=0x7fba6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.817+0000 7fba7d26b640 1 --2- 172.21.15.76:0/294277948 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fba6406b4d0 0x7fba6406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fba7806f160 tx=0x7fba68009040 comp rx=0 tx=0).stop 2024-03-08T06:01:04.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.817+0000 7fba7d26b640 1 -- 172.21.15.76:0/294277948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba780afb60 msgr2=0x7fba780b5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.817+0000 7fba7d26b640 1 --2- 172.21.15.76:0/294277948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba780afb60 0x7fba780b5bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00e9e0 tx=0x7fba6c00eea0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.817+0000 7fba7d26b640 1 -- 172.21.15.76:0/294277948 shutdown_connections 2024-03-08T06:01:04.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.817+0000 7fba7d26b640 1 --2- 172.21.15.76:0/294277948 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.817+0000 7fba7d26b640 1 --2- 172.21.15.76:0/294277948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba780cfd30 0x7fba7806e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.817+0000 7fba7d26b640 1 --2- 172.21.15.76:0/294277948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba780afb60 0x7fba780b5bf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.817+0000 7fba7d26b640 1 --2- 172.21.15.76:0/294277948 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba7806ac80 0x7fba781591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.817+0000 7fba7d26b640 1 -- 172.21.15.76:0/294277948 >> 172.21.15.76:0/294277948 conn(0x7fba78099a60 msgr2=0x7fba780b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:04.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.817+0000 7fba7d26b640 1 -- 172.21.15.76:0/294277948 shutdown_connections 2024-03-08T06:01:04.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.817+0000 7fba7d26b640 1 -- 172.21.15.76:0/294277948 wait complete. 2024-03-08T06:01:04.829 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:01:04.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.931+0000 7fd894efb640 1 Processor -- start 2024-03-08T06:01:04.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.931+0000 7fd894efb640 1 -- start start 2024-03-08T06:01:04.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.932+0000 7fd894efb640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd89006d8a0 0x7fd890162020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.932+0000 7fd894efb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd890162560 0x7fd89015dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.932+0000 7fd894efb640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd890161840 con 0x7fd8900ce5e0 2024-03-08T06:01:04.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.932+0000 7fd894efb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd890161b90 con 0x7fd890162560 2024-03-08T06:01:04.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.932+0000 7fd894efb640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd89015eb30 con 0x7fd89006d8a0 2024-03-08T06:01:04.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.932+0000 7fd88ed9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd890162560 0x7fd89015dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.932+0000 7fd88ed9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd890162560 0x7fd89015dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51734/0 (socket says 172.21.15.76:51734) 2024-03-08T06:01:04.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.932+0000 7fd88ed9d640 1 -- 172.21.15.76:0/487667613 learned_addr learned my addr 172.21.15.76:0/487667613 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:04.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.932+0000 7fd88dd9b640 1 --2- 172.21.15.76:0/487667613 >> v2:172.21.15.81:3300/0 conn(0x7fd89006d8a0 0x7fd890162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.932+0000 7fd88ed9d640 1 -- 172.21.15.76:0/487667613 >> v2:172.21.15.81:3300/0 conn(0x7fd89006d8a0 msgr2=0x7fd890162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:04.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.932+0000 7fd88ed9d640 1 --2- 172.21.15.76:0/487667613 >> v2:172.21.15.81:3300/0 conn(0x7fd89006d8a0 0x7fd890162020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.932+0000 7fd88ed9d640 1 -- 172.21.15.76:0/487667613 >> v1:172.21.15.81:6789/0 conn(0x7fd8900ce5e0 legacy=0x7fd8900ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.932+0000 7fd88ed9d640 1 -- 172.21.15.76:0/487667613 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89015ee10 con 0x7fd890162560 2024-03-08T06:01:04.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.932+0000 7fd88dd9b640 1 --2- 172.21.15.76:0/487667613 >> v2:172.21.15.81:3300/0 conn(0x7fd89006d8a0 0x7fd890162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:04.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.933+0000 7fd88ed9d640 1 --2- 172.21.15.76:0/487667613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd890162560 0x7fd89015dfb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd8900d29f0 tx=0x7fd88802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d67dde906af199 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:04.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.933+0000 7fd88d59a640 1 -- 172.21.15.76:0/487667613 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88802cbb0 con 0x7fd890162560 2024-03-08T06:01:04.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.933+0000 7fd88d59a640 1 -- 172.21.15.76:0/487667613 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88802cd50 con 0x7fd890162560 2024-03-08T06:01:04.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.933+0000 7fd88d59a640 1 -- 172.21.15.76:0/487667613 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888034610 con 0x7fd890162560 2024-03-08T06:01:04.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.933+0000 7fd894efb640 1 -- 172.21.15.76:0/487667613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd890162560 msgr2=0x7fd89015dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.933+0000 7fd894efb640 1 --2- 172.21.15.76:0/487667613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd890162560 0x7fd89015dfb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd8900d29f0 tx=0x7fd88802c010 comp rx=0 tx=0).stop 2024-03-08T06:01:04.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.934+0000 7fd894efb640 1 -- 172.21.15.76:0/487667613 shutdown_connections 2024-03-08T06:01:04.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.934+0000 7fd894efb640 1 --2- 172.21.15.76:0/487667613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd890162560 0x7fd89015dfb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.934+0000 7fd894efb640 1 --2- 172.21.15.76:0/487667613 >> v2:172.21.15.81:3300/0 conn(0x7fd89006d8a0 0x7fd890162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.934+0000 7fd894efb640 1 -- 172.21.15.76:0/487667613 >> 172.21.15.76:0/487667613 conn(0x7fd89006ae60 msgr2=0x7fd89006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:04.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.934+0000 7fd894efb640 1 -- 172.21.15.76:0/487667613 shutdown_connections 2024-03-08T06:01:04.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.934+0000 7fd894efb640 1 -- 172.21.15.76:0/487667613 wait complete. 2024-03-08T06:01:04.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.935+0000 7fd894efb640 1 Processor -- start 2024-03-08T06:01:04.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.935+0000 7fd894efb640 1 -- start start 2024-03-08T06:01:04.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd894efb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd89006d8a0 0x7fd8900b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd894efb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8900ce5e0 0x7fd8900b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd894efb640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd890162560 0x7fd8900b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd894efb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89015e670 con 0x7fd8900ce5e0 2024-03-08T06:01:04.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd894efb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89015e4f0 con 0x7fd89006d8a0 2024-03-08T06:01:04.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd894efb640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89015e7f0 con 0x7fd890162560 2024-03-08T06:01:04.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd88dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8900ce5e0 0x7fd8900b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd88dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8900ce5e0 0x7fd8900b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51748/0 (socket says 172.21.15.76:51748) 2024-03-08T06:01:04.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd88dd9b640 1 -- 172.21.15.76:0/2243046402 learned_addr learned my addr 172.21.15.76:0/2243046402 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:04.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd88e59c640 1 --2- 172.21.15.76:0/2243046402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd89006d8a0 0x7fd8900b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd88ed9d640 1 --2- 172.21.15.76:0/2243046402 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd890162560 0x7fd8900b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd88dd9b640 1 -- 172.21.15.76:0/2243046402 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd890162560 msgr2=0x7fd8900b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd88dd9b640 1 --2- 172.21.15.76:0/2243046402 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd890162560 0x7fd8900b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd88dd9b640 1 -- 172.21.15.76:0/2243046402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd89006d8a0 msgr2=0x7fd8900b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:04.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd88dd9b640 1 --2- 172.21.15.76:0/2243046402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd89006d8a0 0x7fd8900b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:04.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd88dd9b640 1 -- 172.21.15.76:0/2243046402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8900b58e0 con 0x7fd8900ce5e0 2024-03-08T06:01:04.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd88e59c640 1 --2- 172.21.15.76:0/2243046402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd89006d8a0 0x7fd8900b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:04.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd88dd9b640 1 --2- 172.21.15.76:0/2243046402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8900ce5e0 0x7fd8900b2810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd88400d7d0 tx=0x7fd88400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:04.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd87effd640 1 -- 172.21.15.76:0/2243046402 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd884004030 con 0x7fd8900ce5e0 2024-03-08T06:01:04.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd87effd640 1 -- 172.21.15.76:0/2243046402 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8840041d0 con 0x7fd8900ce5e0 2024-03-08T06:01:04.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd894efb640 1 -- 172.21.15.76:0/2243046402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8900b5bc0 con 0x7fd8900ce5e0 2024-03-08T06:01:04.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd87effd640 1 -- 172.21.15.76:0/2243046402 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8840096a0 con 0x7fd8900ce5e0 2024-03-08T06:01:04.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.936+0000 7fd894efb640 1 -- 172.21.15.76:0/2243046402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8900b6180 con 0x7fd8900ce5e0 2024-03-08T06:01:04.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.937+0000 7fd894efb640 1 -- 172.21.15.76:0/2243046402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89004f030 con 0x7fd8900ce5e0 2024-03-08T06:01:04.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.938+0000 7fd87effd640 1 -- 172.21.15.76:0/2243046402 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd884009840 con 0x7fd8900ce5e0 2024-03-08T06:01:04.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.938+0000 7fd87effd640 1 --2- 172.21.15.76:0/2243046402 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd87006b500 0x7fd87006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:04.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.938+0000 7fd87effd640 1 -- 172.21.15.76:0/2243046402 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd88408a830 con 0x7fd8900ce5e0 2024-03-08T06:01:04.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.938+0000 7fd88e59c640 1 --2- 172.21.15.76:0/2243046402 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd87006b500 0x7fd87006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:04.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.939+0000 7fd88e59c640 1 --2- 172.21.15.76:0/2243046402 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd87006b500 0x7fd87006d9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd8900b4900 tx=0x7fd880009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:04.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:04.942+0000 7fd87effd640 1 -- 172.21.15.76:0/2243046402 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd884059370 con 0x7fd8900ce5e0 2024-03-08T06:01:05.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.114+0000 7fd894efb640 1 -- 172.21.15.76:0/2243046402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd8900b4330 con 0x7fd8900ce5e0 2024-03-08T06:01:05.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.114+0000 7fd87effd640 1 -- 172.21.15.76:0/2243046402 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd884058c70 con 0x7fd8900ce5e0 2024-03-08T06:01:05.115 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:05.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.116+0000 7fd894efb640 1 -- 172.21.15.76:0/2243046402 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd87006b500 msgr2=0x7fd87006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.116+0000 7fd894efb640 1 --2- 172.21.15.76:0/2243046402 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd87006b500 0x7fd87006d9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd8900b4900 tx=0x7fd880009040 comp rx=0 tx=0).stop 2024-03-08T06:01:05.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.116+0000 7fd894efb640 1 -- 172.21.15.76:0/2243046402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8900ce5e0 msgr2=0x7fd8900b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.116+0000 7fd894efb640 1 --2- 172.21.15.76:0/2243046402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8900ce5e0 0x7fd8900b2810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd88400d7d0 tx=0x7fd88400dc90 comp rx=0 tx=0).stop 2024-03-08T06:01:05.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.116+0000 7fd894efb640 1 -- 172.21.15.76:0/2243046402 shutdown_connections 2024-03-08T06:01:05.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.116+0000 7fd894efb640 1 --2- 172.21.15.76:0/2243046402 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd87006b500 0x7fd87006d9c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.116+0000 7fd894efb640 1 --2- 172.21.15.76:0/2243046402 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd890162560 0x7fd8900b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.116+0000 7fd894efb640 1 --2- 172.21.15.76:0/2243046402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8900ce5e0 0x7fd8900b2810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.116+0000 7fd894efb640 1 --2- 172.21.15.76:0/2243046402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd89006d8a0 0x7fd8900b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.116+0000 7fd894efb640 1 -- 172.21.15.76:0/2243046402 >> 172.21.15.76:0/2243046402 conn(0x7fd89006ae60 msgr2=0x7fd890160960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:05.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.116+0000 7fd894efb640 1 -- 172.21.15.76:0/2243046402 shutdown_connections 2024-03-08T06:01:05.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.116+0000 7fd894efb640 1 -- 172.21.15.76:0/2243046402 wait complete. 2024-03-08T06:01:05.128 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:01:05.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.230+0000 7f462f508640 1 Processor -- start 2024-03-08T06:01:05.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.230+0000 7f462f508640 1 -- start start 2024-03-08T06:01:05.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.231+0000 7f462f508640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f46280ce5e0 0x7f462806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.231+0000 7f462f508640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46280ce9c0 0x7f4628162020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.231+0000 7f462f508640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4628161840 con 0x7f4628162560 2024-03-08T06:01:05.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.231+0000 7f462f508640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4628161b90 con 0x7f46280ce5e0 2024-03-08T06:01:05.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.231+0000 7f462f508640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462815eb30 con 0x7f46280ce9c0 2024-03-08T06:01:05.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.231+0000 7f462caa3640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46280ce9c0 0x7f4628162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.231+0000 7f462d2a4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f46280ce5e0 0x7f462806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.231+0000 7f462caa3640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46280ce9c0 0x7f4628162020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37570/0 (socket says 172.21.15.76:37570) 2024-03-08T06:01:05.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.231+0000 7f462daa5640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f4628162560 0x7f462815dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:40250/0 (socket says 172.21.15.76:40250) 2024-03-08T06:01:05.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.231+0000 7f462caa3640 1 -- 172.21.15.76:0/1557292529 learned_addr learned my addr 172.21.15.76:0/1557292529 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:05.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.231+0000 7f462caa3640 1 -- 172.21.15.76:0/1557292529 >> v2:172.21.15.81:3300/0 conn(0x7f46280ce5e0 msgr2=0x7f462806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:05.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.231+0000 7f462caa3640 1 --2- 172.21.15.76:0/1557292529 >> v2:172.21.15.81:3300/0 conn(0x7f46280ce5e0 0x7f462806d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.231+0000 7f462caa3640 1 -- 172.21.15.76:0/1557292529 >> v1:172.21.15.81:6789/0 conn(0x7f4628162560 legacy=0x7f462815dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.231+0000 7f462caa3640 1 -- 172.21.15.76:0/1557292529 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462815ee10 con 0x7f46280ce9c0 2024-03-08T06:01:05.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.231+0000 7f462d2a4640 1 --2- 172.21.15.76:0/1557292529 >> v2:172.21.15.81:3300/0 conn(0x7f46280ce5e0 0x7f462806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:05.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.232+0000 7f462caa3640 1 --2- 172.21.15.76:0/1557292529 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46280ce9c0 0x7f4628162020 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4618009870 tx=0x7f461802ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b5b377475c0ffaa server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:05.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.232+0000 7f461ffff640 1 -- 172.21.15.76:0/1557292529 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461802f840 con 0x7f46280ce9c0 2024-03-08T06:01:05.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.232+0000 7f461ffff640 1 -- 172.21.15.76:0/1557292529 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461802f9e0 con 0x7f46280ce9c0 2024-03-08T06:01:05.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.232+0000 7f461ffff640 1 -- 172.21.15.76:0/1557292529 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461802fdb0 con 0x7f46280ce9c0 2024-03-08T06:01:05.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.233+0000 7f462f508640 1 -- 172.21.15.76:0/1557292529 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46280ce9c0 msgr2=0x7f4628162020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.233+0000 7f462f508640 1 --2- 172.21.15.76:0/1557292529 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46280ce9c0 0x7f4628162020 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4618009870 tx=0x7f461802ba20 comp rx=0 tx=0).stop 2024-03-08T06:01:05.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.233+0000 7f462f508640 1 -- 172.21.15.76:0/1557292529 shutdown_connections 2024-03-08T06:01:05.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.233+0000 7f462f508640 1 --2- 172.21.15.76:0/1557292529 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46280ce9c0 0x7f4628162020 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.233+0000 7f462f508640 1 --2- 172.21.15.76:0/1557292529 >> v2:172.21.15.81:3300/0 conn(0x7f46280ce5e0 0x7f462806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.233+0000 7f462f508640 1 -- 172.21.15.76:0/1557292529 >> 172.21.15.76:0/1557292529 conn(0x7f462806ae60 msgr2=0x7f462806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:05.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.233+0000 7f462f508640 1 -- 172.21.15.76:0/1557292529 shutdown_connections 2024-03-08T06:01:05.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.233+0000 7f462f508640 1 -- 172.21.15.76:0/1557292529 wait complete. 2024-03-08T06:01:05.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.234+0000 7f462f508640 1 Processor -- start 2024-03-08T06:01:05.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.234+0000 7f462f508640 1 -- start start 2024-03-08T06:01:05.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.235+0000 7f462f508640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46280ce5e0 0x7f46280b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.235+0000 7f462f508640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46280ce9c0 0x7f462814edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.235+0000 7f462f508640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4628162560 0x7f46280b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.235+0000 7f462f508640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462815e6c0 con 0x7f4628162560 2024-03-08T06:01:05.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.235+0000 7f462f508640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46280cee20 con 0x7f46280ce9c0 2024-03-08T06:01:05.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.235+0000 7f462f508640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462815e840 con 0x7f46280ce5e0 2024-03-08T06:01:05.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.235+0000 7f462daa5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4628162560 0x7f46280b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.235+0000 7f462daa5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4628162560 0x7f46280b5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51754/0 (socket says 172.21.15.76:51754) 2024-03-08T06:01:05.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.235+0000 7f462daa5640 1 -- 172.21.15.76:0/653036771 learned_addr learned my addr 172.21.15.76:0/653036771 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:05.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.235+0000 7f462caa3640 1 --2- 172.21.15.76:0/653036771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46280ce9c0 0x7f462814edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.235+0000 7f462d2a4640 1 --2- 172.21.15.76:0/653036771 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46280ce5e0 0x7f46280b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.235+0000 7f462daa5640 1 -- 172.21.15.76:0/653036771 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46280ce5e0 msgr2=0x7f46280b6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.235+0000 7f462daa5640 1 --2- 172.21.15.76:0/653036771 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46280ce5e0 0x7f46280b6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.235+0000 7f462daa5640 1 -- 172.21.15.76:0/653036771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46280ce9c0 msgr2=0x7f462814edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.235+0000 7f462daa5640 1 --2- 172.21.15.76:0/653036771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46280ce9c0 0x7f462814edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.235+0000 7f462daa5640 1 -- 172.21.15.76:0/653036771 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462814f300 con 0x7f4628162560 2024-03-08T06:01:05.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.236+0000 7f462daa5640 1 --2- 172.21.15.76:0/653036771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4628162560 0x7f46280b5b50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f462000c780 tx=0x7f462000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:05.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.236+0000 7f461dffb640 1 -- 172.21.15.76:0/653036771 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620017070 con 0x7f4628162560 2024-03-08T06:01:05.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.236+0000 7f461dffb640 1 -- 172.21.15.76:0/653036771 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462000a640 con 0x7f4628162560 2024-03-08T06:01:05.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.236+0000 7f461dffb640 1 -- 172.21.15.76:0/653036771 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462000aa30 con 0x7f4628162560 2024-03-08T06:01:05.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.236+0000 7f462f508640 1 -- 172.21.15.76:0/653036771 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462814f5e0 con 0x7f4628162560 2024-03-08T06:01:05.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.236+0000 7f462f508640 1 -- 172.21.15.76:0/653036771 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46280afe20 con 0x7f4628162560 2024-03-08T06:01:05.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.237+0000 7f462f508640 1 -- 172.21.15.76:0/653036771 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462814f7b0 con 0x7f4628162560 2024-03-08T06:01:05.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.237+0000 7f461dffb640 1 -- 172.21.15.76:0/653036771 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f462001a070 con 0x7f4628162560 2024-03-08T06:01:05.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.237+0000 7f461dffb640 1 --2- 172.21.15.76:0/653036771 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f460c06b4d0 0x7f460c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.237+0000 7f461dffb640 1 -- 172.21.15.76:0/653036771 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4620089ac0 con 0x7f4628162560 2024-03-08T06:01:05.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.238+0000 7f462d2a4640 1 --2- 172.21.15.76:0/653036771 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f460c06b4d0 0x7f460c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.238+0000 7f462d2a4640 1 --2- 172.21.15.76:0/653036771 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f460c06b4d0 0x7f460c06d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4624009560 tx=0x7f4624009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:05.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.242+0000 7f461dffb640 1 -- 172.21.15.76:0/653036771 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f462814f300 con 0x7f4628162560 2024-03-08T06:01:05.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.413+0000 7f462f508640 1 -- 172.21.15.76:0/653036771 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f46280b4330 con 0x7f4628162560 2024-03-08T06:01:05.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.413+0000 7f461dffb640 1 -- 172.21.15.76:0/653036771 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4620057f30 con 0x7f4628162560 2024-03-08T06:01:05.413 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:01:05.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.415+0000 7f462f508640 1 -- 172.21.15.76:0/653036771 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f460c06b4d0 msgr2=0x7f460c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.415+0000 7f462f508640 1 --2- 172.21.15.76:0/653036771 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f460c06b4d0 0x7f460c06d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4624009560 tx=0x7f4624009190 comp rx=0 tx=0).stop 2024-03-08T06:01:05.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.415+0000 7f462f508640 1 -- 172.21.15.76:0/653036771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4628162560 msgr2=0x7f46280b5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.415+0000 7f462f508640 1 --2- 172.21.15.76:0/653036771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4628162560 0x7f46280b5b50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f462000c780 tx=0x7f462000cc40 comp rx=0 tx=0).stop 2024-03-08T06:01:05.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.415+0000 7f462f508640 1 -- 172.21.15.76:0/653036771 shutdown_connections 2024-03-08T06:01:05.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.415+0000 7f462f508640 1 --2- 172.21.15.76:0/653036771 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f460c06b4d0 0x7f460c06d990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.415+0000 7f462f508640 1 --2- 172.21.15.76:0/653036771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4628162560 0x7f46280b5b50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.415+0000 7f462f508640 1 --2- 172.21.15.76:0/653036771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46280ce9c0 0x7f462814edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.415+0000 7f462f508640 1 --2- 172.21.15.76:0/653036771 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46280ce5e0 0x7f46280b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.415+0000 7f462f508640 1 -- 172.21.15.76:0/653036771 >> 172.21.15.76:0/653036771 conn(0x7f462806ae60 msgr2=0x7f462806bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:05.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.415+0000 7f462f508640 1 -- 172.21.15.76:0/653036771 shutdown_connections 2024-03-08T06:01:05.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.415+0000 7f462f508640 1 -- 172.21.15.76:0/653036771 wait complete. 2024-03-08T06:01:05.426 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T06:01:05.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.528+0000 7fcbc8072640 1 Processor -- start 2024-03-08T06:01:05.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.529+0000 7fcbc8072640 1 -- start start 2024-03-08T06:01:05.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.529+0000 7fcbc8072640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcbc015b260 0x7fcbc00b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.529+0000 7fcbc8072640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcbc00b2c50 0x7fcbc015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.529+0000 7fcbc8072640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc015b820 con 0x7fcbc00aee10 2024-03-08T06:01:05.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.529+0000 7fcbc8072640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc00b30a0 con 0x7fcbc00b2c50 2024-03-08T06:01:05.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.529+0000 7fcbc8072640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc015e490 con 0x7fcbc015b260 2024-03-08T06:01:05.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc5e0e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fcbc00aee10 0x7fcbc015dc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57650/0 (socket says 172.21.15.76:57650) 2024-03-08T06:01:05.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc5e0e640 1 -- 172.21.15.76:0/715753715 learned_addr learned my addr 172.21.15.76:0/715753715 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:05.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc560d640 1 --2- 172.21.15.76:0/715753715 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcbc015b260 0x7fcbc00b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc660f640 1 --2- 172.21.15.76:0/715753715 >> v2:172.21.15.81:3300/0 conn(0x7fcbc00b2c50 0x7fcbc015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc4e0c640 1 -- 172.21.15.76:0/715753715 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651177791 0 0) 0x7fcbc015b820 con 0x7fcbc00aee10 2024-03-08T06:01:05.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc4e0c640 1 -- 172.21.15.76:0/715753715 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbac003520 con 0x7fcbc00aee10 2024-03-08T06:01:05.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc4e0c640 1 -- 172.21.15.76:0/715753715 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 176052343 0 0) 0x7fcbac003520 con 0x7fcbc00aee10 2024-03-08T06:01:05.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc4e0c640 1 -- 172.21.15.76:0/715753715 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcbc015b260 msgr2=0x7fcbc00b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:05.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc4e0c640 1 --2- 172.21.15.76:0/715753715 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcbc015b260 0x7fcbc00b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc4e0c640 1 -- 172.21.15.76:0/715753715 >> v2:172.21.15.81:3300/0 conn(0x7fcbc00b2c50 msgr2=0x7fcbc015bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:05.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc4e0c640 1 --2- 172.21.15.76:0/715753715 >> v2:172.21.15.81:3300/0 conn(0x7fcbc00b2c50 0x7fcbc015bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc4e0c640 1 -- 172.21.15.76:0/715753715 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc015e770 con 0x7fcbc00aee10 2024-03-08T06:01:05.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc4e0c640 1 -- 172.21.15.76:0/715753715 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fcbbc001670 con 0x7fcbc00aee10 2024-03-08T06:01:05.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc4e0c640 1 -- 172.21.15.76:0/715753715 >> v1:172.21.15.76:6789/0 conn(0x7fcbc00aee10 legacy=0x7fcbc015dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc4e0c640 1 --2- 172.21.15.76:0/715753715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbac003a20 0x7fcbac003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc4e0c640 1 --2- 172.21.15.76:0/715753715 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcbc015b260 0x7fcbac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc4e0c640 1 --2- 172.21.15.76:0/715753715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbac004d40 0x7fcbac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc4e0c640 1 -- 172.21.15.76:0/715753715 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00b30a0 con 0x7fcbac003a20 2024-03-08T06:01:05.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc4e0c640 1 -- 172.21.15.76:0/715753715 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc015e490 con 0x7fcbac004d40 2024-03-08T06:01:05.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc4e0c640 1 -- 172.21.15.76:0/715753715 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac00dc20 con 0x7fcbc015b260 2024-03-08T06:01:05.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc5e0e640 1 --2- 172.21.15.76:0/715753715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbac003a20 0x7fcbac003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc660f640 1 --2- 172.21.15.76:0/715753715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbac004d40 0x7fcbac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.530+0000 7fcbc560d640 1 --2- 172.21.15.76:0/715753715 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcbc015b260 0x7fcbac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.531+0000 7fcbc5e0e640 1 -- 172.21.15.76:0/715753715 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcbc015b260 msgr2=0x7fcbac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:05.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.531+0000 7fcbc5e0e640 1 --2- 172.21.15.76:0/715753715 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcbc015b260 0x7fcbac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.531+0000 7fcbc5e0e640 1 -- 172.21.15.76:0/715753715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbac004d40 msgr2=0x7fcbac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:05.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.531+0000 7fcbc5e0e640 1 --2- 172.21.15.76:0/715753715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbac004d40 0x7fcbac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.531+0000 7fcbc5e0e640 1 -- 172.21.15.76:0/715753715 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac003520 con 0x7fcbac003a20 2024-03-08T06:01:05.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.531+0000 7fcbc5e0e640 1 --2- 172.21.15.76:0/715753715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbac003a20 0x7fcbac003e40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00c210 tx=0x7fcbac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34353f1f56b21675 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:05.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.531+0000 7fcbc4e0c640 1 -- 172.21.15.76:0/715753715 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc007d50 con 0x7fcbac003a20 2024-03-08T06:01:05.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.531+0000 7fcbc4e0c640 1 -- 172.21.15.76:0/715753715 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc030070 con 0x7fcbac003a20 2024-03-08T06:01:05.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.531+0000 7fcbc4e0c640 1 -- 172.21.15.76:0/715753715 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc02cd80 con 0x7fcbac003a20 2024-03-08T06:01:05.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.532+0000 7fcbc8072640 1 -- 172.21.15.76:0/715753715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbac003a20 msgr2=0x7fcbac003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.532+0000 7fcbc8072640 1 --2- 172.21.15.76:0/715753715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbac003a20 0x7fcbac003e40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00c210 tx=0x7fcbac005830 comp rx=0 tx=0).stop 2024-03-08T06:01:05.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.532+0000 7fcbc660f640 1 -- 172.21.15.76:0/715753715 reap_dead start 2024-03-08T06:01:05.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.532+0000 7fcbc8072640 1 -- 172.21.15.76:0/715753715 shutdown_connections 2024-03-08T06:01:05.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.532+0000 7fcbc8072640 1 -- 172.21.15.76:0/715753715 >> 172.21.15.76:0/715753715 conn(0x7fcbc006ac80 msgr2=0x7fcbc006b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:05.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.532+0000 7fcbc8072640 1 -- 172.21.15.76:0/715753715 shutdown_connections 2024-03-08T06:01:05.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.532+0000 7fcbc8072640 1 -- 172.21.15.76:0/715753715 wait complete. 2024-03-08T06:01:05.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.533+0000 7fcbc8072640 1 Processor -- start 2024-03-08T06:01:05.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.533+0000 7fcbc8072640 1 -- start start 2024-03-08T06:01:05.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc8072640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcbac003a20 0x7fcbc0140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc8072640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbc01ef5b0 0x7fcbc0154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc8072640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc00adec0 0x7fcbc006e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc8072640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00b2710 con 0x7fcbc00adec0 2024-03-08T06:01:05.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc8072640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc015ea50 con 0x7fcbc01ef5b0 2024-03-08T06:01:05.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc8072640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00b2890 con 0x7fcbac003a20 2024-03-08T06:01:05.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc660f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc00adec0 0x7fcbc006e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc660f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc00adec0 0x7fcbc006e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51784/0 (socket says 172.21.15.76:51784) 2024-03-08T06:01:05.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc660f640 1 -- 172.21.15.76:0/3558037628 learned_addr learned my addr 172.21.15.76:0/3558037628 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:05.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc5e0e640 1 --2- 172.21.15.76:0/3558037628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcbac003a20 0x7fcbc0140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc560d640 1 --2- 172.21.15.76:0/3558037628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbc01ef5b0 0x7fcbc0154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc660f640 1 -- 172.21.15.76:0/3558037628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcbac003a20 msgr2=0x7fcbc0140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc660f640 1 --2- 172.21.15.76:0/3558037628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcbac003a20 0x7fcbc0140380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc660f640 1 -- 172.21.15.76:0/3558037628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbc01ef5b0 msgr2=0x7fcbc0154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc660f640 1 --2- 172.21.15.76:0/3558037628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbc01ef5b0 0x7fcbc0154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc660f640 1 -- 172.21.15.76:0/3558037628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc01551c0 con 0x7fcbc00adec0 2024-03-08T06:01:05.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.534+0000 7fcbc660f640 1 --2- 172.21.15.76:0/3558037628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc00adec0 0x7fcbc006e020 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0143e50 tx=0x7fcbb800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:05.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.535+0000 7fcbb67fc640 1 -- 172.21.15.76:0/3558037628 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb800fe10 con 0x7fcbc00adec0 2024-03-08T06:01:05.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.535+0000 7fcbb67fc640 1 -- 172.21.15.76:0/3558037628 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb8015430 con 0x7fcbc00adec0 2024-03-08T06:01:05.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.535+0000 7fcbb67fc640 1 -- 172.21.15.76:0/3558037628 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8015820 con 0x7fcbc00adec0 2024-03-08T06:01:05.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.535+0000 7fcbc8072640 1 -- 172.21.15.76:0/3558037628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc01efb20 con 0x7fcbc00adec0 2024-03-08T06:01:05.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.535+0000 7fcbc8072640 1 -- 172.21.15.76:0/3558037628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc01f00e0 con 0x7fcbc00adec0 2024-03-08T06:01:05.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.536+0000 7fcbc8072640 1 -- 172.21.15.76:0/3558037628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc004f030 con 0x7fcbc00adec0 2024-03-08T06:01:05.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.536+0000 7fcbb67fc640 1 -- 172.21.15.76:0/3558037628 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbb80159c0 con 0x7fcbc00adec0 2024-03-08T06:01:05.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.536+0000 7fcbb67fc640 1 --2- 172.21.15.76:0/3558037628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcba406b500 0x7fcba406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.536+0000 7fcbb67fc640 1 -- 172.21.15.76:0/3558037628 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcbb808b270 con 0x7fcbc00adec0 2024-03-08T06:01:05.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.536+0000 7fcbc5e0e640 1 --2- 172.21.15.76:0/3558037628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcba406b500 0x7fcba406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.537+0000 7fcbc5e0e640 1 --2- 172.21.15.76:0/3558037628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcba406b500 0x7fcba406d9c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc034000 tx=0x7fcbbc007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:05.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.541+0000 7fcbb67fc640 1 -- 172.21.15.76:0/3558037628 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcbb8019030 con 0x7fcbc00adec0 2024-03-08T06:01:05.657 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:01:05.659 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:01:05.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.712+0000 7fcbc8072640 1 -- 172.21.15.76:0/3558037628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcbc00b0460 con 0x7fcbc00adec0 2024-03-08T06:01:05.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.713+0000 7fcbb67fc640 1 -- 172.21.15.76:0/3558037628 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcbb8010040 con 0x7fcbc00adec0 2024-03-08T06:01:05.713 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:05.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.715+0000 7fcbc8072640 1 -- 172.21.15.76:0/3558037628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcba406b500 msgr2=0x7fcba406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.715+0000 7fcbc8072640 1 --2- 172.21.15.76:0/3558037628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcba406b500 0x7fcba406d9c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc034000 tx=0x7fcbbc007000 comp rx=0 tx=0).stop 2024-03-08T06:01:05.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.715+0000 7fcbc8072640 1 -- 172.21.15.76:0/3558037628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc00adec0 msgr2=0x7fcbc006e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.715+0000 7fcbc8072640 1 --2- 172.21.15.76:0/3558037628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc00adec0 0x7fcbc006e020 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0143e50 tx=0x7fcbb800f580 comp rx=0 tx=0).stop 2024-03-08T06:01:05.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.715+0000 7fcbc8072640 1 -- 172.21.15.76:0/3558037628 shutdown_connections 2024-03-08T06:01:05.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.715+0000 7fcbc8072640 1 --2- 172.21.15.76:0/3558037628 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcba406b500 0x7fcba406d9c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.715+0000 7fcbc8072640 1 --2- 172.21.15.76:0/3558037628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc00adec0 0x7fcbc006e020 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.715+0000 7fcbc8072640 1 --2- 172.21.15.76:0/3558037628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbc01ef5b0 0x7fcbc0154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.715+0000 7fcbc8072640 1 --2- 172.21.15.76:0/3558037628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcbac003a20 0x7fcbc0140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.715+0000 7fcbc8072640 1 -- 172.21.15.76:0/3558037628 >> 172.21.15.76:0/3558037628 conn(0x7fcbc006ac80 msgr2=0x7fcbc006ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:05.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.715+0000 7fcbc8072640 1 -- 172.21.15.76:0/3558037628 shutdown_connections 2024-03-08T06:01:05.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.715+0000 7fcbc8072640 1 -- 172.21.15.76:0/3558037628 wait complete. 2024-03-08T06:01:05.726 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:01:05.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.825+0000 7f13b31f6640 1 Processor -- start 2024-03-08T06:01:05.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.826+0000 7f13b31f6640 1 -- start start 2024-03-08T06:01:05.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.826+0000 7f13b31f6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f13ac15dc10 0x7f13ac1623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.826+0000 7f13b31f6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f13ac15dff0 0x7f13ac0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.826+0000 7f13b31f6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ac0cffc0 con 0x7f13ac0ce3f0 2024-03-08T06:01:05.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.826+0000 7f13b31f6640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ac06c780 con 0x7f13ac15dff0 2024-03-08T06:01:05.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.826+0000 7f13b31f6640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ac0d02a0 con 0x7f13ac15dc10 2024-03-08T06:01:05.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.826+0000 7f13abfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f13ac15dff0 0x7f13ac0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.826+0000 7f13abfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f13ac15dff0 0x7f13ac0cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51798/0 (socket says 172.21.15.76:51798) 2024-03-08T06:01:05.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.826+0000 7f13b1793640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f13ac0ce3f0 0x7f13ac0b3460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57656/0 (socket says 172.21.15.76:57656) 2024-03-08T06:01:05.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.826+0000 7f13abfff640 1 -- 172.21.15.76:0/2253489034 learned_addr learned my addr 172.21.15.76:0/2253489034 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:05.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.827+0000 7f13b0f92640 1 --2- 172.21.15.76:0/2253489034 >> v2:172.21.15.81:3300/0 conn(0x7f13ac15dc10 0x7f13ac1623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.827+0000 7f13abfff640 1 -- 172.21.15.76:0/2253489034 >> v2:172.21.15.81:3300/0 conn(0x7f13ac15dc10 msgr2=0x7f13ac1623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:05.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.827+0000 7f13abfff640 1 --2- 172.21.15.76:0/2253489034 >> v2:172.21.15.81:3300/0 conn(0x7f13ac15dc10 0x7f13ac1623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.827+0000 7f13abfff640 1 -- 172.21.15.76:0/2253489034 >> v1:172.21.15.76:6789/0 conn(0x7f13ac0ce3f0 legacy=0x7f13ac0b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.827+0000 7f13ab7fe640 1 -- 172.21.15.76:0/2253489034 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775114342 0 0) 0x7f13ac0cffc0 con 0x7f13ac0ce3f0 2024-03-08T06:01:05.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.827+0000 7f13abfff640 1 -- 172.21.15.76:0/2253489034 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac06c410 con 0x7f13ac15dff0 2024-03-08T06:01:05.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.827+0000 7f13b0f92640 1 --2- 172.21.15.76:0/2253489034 >> v2:172.21.15.81:3300/0 conn(0x7f13ac15dc10 0x7f13ac1623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:05.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.827+0000 7f13abfff640 1 --2- 172.21.15.76:0/2253489034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13ac15dff0 0x7f13ac0cf7c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009870 tx=0x7f13a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40393e68db7ccc2e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:05.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.828+0000 7f13ab7fe640 1 -- 172.21.15.76:0/2253489034 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0035020 con 0x7f13ac15dff0 2024-03-08T06:01:05.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.828+0000 7f13ab7fe640 1 -- 172.21.15.76:0/2253489034 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a0031870 con 0x7f13ac15dff0 2024-03-08T06:01:05.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.828+0000 7f13ab7fe640 1 -- 172.21.15.76:0/2253489034 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0031c40 con 0x7f13ac15dff0 2024-03-08T06:01:05.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.828+0000 7f13b31f6640 1 -- 172.21.15.76:0/2253489034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13ac15dff0 msgr2=0x7f13ac0cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.828+0000 7f13b31f6640 1 --2- 172.21.15.76:0/2253489034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13ac15dff0 0x7f13ac0cf7c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009870 tx=0x7f13a002b910 comp rx=0 tx=0).stop 2024-03-08T06:01:05.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.828+0000 7f13b31f6640 1 -- 172.21.15.76:0/2253489034 shutdown_connections 2024-03-08T06:01:05.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.828+0000 7f13b31f6640 1 --2- 172.21.15.76:0/2253489034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13ac15dff0 0x7f13ac0cf7c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.828+0000 7f13b31f6640 1 --2- 172.21.15.76:0/2253489034 >> v2:172.21.15.81:3300/0 conn(0x7f13ac15dc10 0x7f13ac1623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.828+0000 7f13b31f6640 1 -- 172.21.15.76:0/2253489034 >> 172.21.15.76:0/2253489034 conn(0x7f13ac06f5e0 msgr2=0x7f13ac06f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:05.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.828+0000 7f13b31f6640 1 -- 172.21.15.76:0/2253489034 shutdown_connections 2024-03-08T06:01:05.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.828+0000 7f13b31f6640 1 -- 172.21.15.76:0/2253489034 wait complete. 2024-03-08T06:01:05.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.829+0000 7f13b31f6640 1 Processor -- start 2024-03-08T06:01:05.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.829+0000 7f13b31f6640 1 -- start start 2024-03-08T06:01:05.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.830+0000 7f13b31f6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13ac0ce3f0 0x7f13ac1f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.830+0000 7f13b31f6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13ac15dc10 0x7f13ac1f0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.830+0000 7f13b31f6640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f13ac15dff0 0x7f13ac0b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.830+0000 7f13b31f6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac06dc20 con 0x7f13ac15dc10 2024-03-08T06:01:05.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.830+0000 7f13b31f6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac0d0580 con 0x7f13ac0ce3f0 2024-03-08T06:01:05.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.830+0000 7f13b31f6640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac06dda0 con 0x7f13ac15dff0 2024-03-08T06:01:05.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.830+0000 7f13abfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13ac15dc10 0x7f13ac1f0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.830+0000 7f13abfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13ac15dc10 0x7f13ac1f0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51814/0 (socket says 172.21.15.76:51814) 2024-03-08T06:01:05.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.830+0000 7f13abfff640 1 -- 172.21.15.76:0/139398012 learned_addr learned my addr 172.21.15.76:0/139398012 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:05.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.831+0000 7f13b0f92640 1 --2- 172.21.15.76:0/139398012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13ac0ce3f0 0x7f13ac1f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.831+0000 7f13b1793640 1 --2- 172.21.15.76:0/139398012 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f13ac15dff0 0x7f13ac0b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.831+0000 7f13abfff640 1 -- 172.21.15.76:0/139398012 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f13ac15dff0 msgr2=0x7f13ac0b67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.831+0000 7f13abfff640 1 --2- 172.21.15.76:0/139398012 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f13ac15dff0 0x7f13ac0b67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.831+0000 7f13abfff640 1 -- 172.21.15.76:0/139398012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13ac0ce3f0 msgr2=0x7f13ac1f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:05.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.831+0000 7f13abfff640 1 --2- 172.21.15.76:0/139398012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13ac0ce3f0 0x7f13ac1f44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:05.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.831+0000 7f13abfff640 1 -- 172.21.15.76:0/139398012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac1f4a00 con 0x7f13ac15dc10 2024-03-08T06:01:05.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.831+0000 7f13b1793640 1 --2- 172.21.15.76:0/139398012 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f13ac15dff0 0x7f13ac0b67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:05.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.831+0000 7f13abfff640 1 --2- 172.21.15.76:0/139398012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13ac15dc10 0x7f13ac1f0db0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f13a002b950 tx=0x7f13a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:05.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.831+0000 7f13a97fa640 1 -- 172.21.15.76:0/139398012 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0035040 con 0x7f13ac15dc10 2024-03-08T06:01:05.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.831+0000 7f13a97fa640 1 -- 172.21.15.76:0/139398012 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a0030d00 con 0x7f13ac15dc10 2024-03-08T06:01:05.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.831+0000 7f13a97fa640 1 -- 172.21.15.76:0/139398012 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a00456c0 con 0x7f13ac15dc10 2024-03-08T06:01:05.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.831+0000 7f13b31f6640 1 -- 172.21.15.76:0/139398012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ac0b4ba0 con 0x7f13ac15dc10 2024-03-08T06:01:05.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.831+0000 7f13b31f6640 1 -- 172.21.15.76:0/139398012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ac0b5160 con 0x7f13ac15dc10 2024-03-08T06:01:05.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.832+0000 7f13b31f6640 1 -- 172.21.15.76:0/139398012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ac04f030 con 0x7f13ac15dc10 2024-03-08T06:01:05.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.832+0000 7f13a97fa640 1 -- 172.21.15.76:0/139398012 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f13a0038020 con 0x7f13ac15dc10 2024-03-08T06:01:05.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.833+0000 7f13a97fa640 1 --2- 172.21.15.76:0/139398012 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f139806b4d0 0x7f139806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:05.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.833+0000 7f13a97fa640 1 -- 172.21.15.76:0/139398012 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f13a00ac7c0 con 0x7f13ac15dc10 2024-03-08T06:01:05.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.833+0000 7f13b0f92640 1 --2- 172.21.15.76:0/139398012 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f139806b4d0 0x7f139806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:05.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.833+0000 7f13b0f92640 1 --2- 172.21.15.76:0/139398012 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f139806b4d0 0x7f139806d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0ca5c0 tx=0x7f139c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:05.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:05.837+0000 7f13a97fa640 1 -- 172.21.15.76:0/139398012 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f13ac1f4a00 con 0x7f13ac15dc10 2024-03-08T06:01:06.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.044+0000 7f13b31f6640 1 -- 172.21.15.76:0/139398012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13ac0b7960 con 0x7f13ac15dc10 2024-03-08T06:01:06.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.045+0000 7f13a97fa640 1 -- 172.21.15.76:0/139398012 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f13a007b390 con 0x7f13ac15dc10 2024-03-08T06:01:06.046 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:06.046 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4249563566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4249563566},{"type":"v1","addr":"172.21.15.76:6835","nonce":4249563566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1514641617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1514641617},{"type":"v1","addr":"172.21.15.76:6837","nonce":1514641617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6834/1332442802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1332442802},{"type":"v1","addr":"172.21.15.81:6834","nonce":1332442802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:00:35.504231+0000","modified":"2024-03-08T06:00:57.594023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6835/2357927814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":2357927814},{"type":"v1","addr":"172.21.15.81:6835","nonce":2357927814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T06:01:06.046 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-03-08T06:01:06.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.047+0000 7f13b31f6640 1 -- 172.21.15.76:0/139398012 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f139806b4d0 msgr2=0x7f139806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.047+0000 7f13b31f6640 1 --2- 172.21.15.76:0/139398012 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f139806b4d0 0x7f139806d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0ca5c0 tx=0x7f139c008040 comp rx=0 tx=0).stop 2024-03-08T06:01:06.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.047+0000 7f13b31f6640 1 -- 172.21.15.76:0/139398012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13ac15dc10 msgr2=0x7f13ac1f0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.047+0000 7f13b31f6640 1 --2- 172.21.15.76:0/139398012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13ac15dc10 0x7f13ac1f0db0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f13a002b950 tx=0x7f13a00046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.047+0000 7f13b31f6640 1 -- 172.21.15.76:0/139398012 shutdown_connections 2024-03-08T06:01:06.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.047+0000 7f13b31f6640 1 --2- 172.21.15.76:0/139398012 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f139806b4d0 0x7f139806d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.047+0000 7f13b31f6640 1 --2- 172.21.15.76:0/139398012 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f13ac15dff0 0x7f13ac0b67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.047+0000 7f13b31f6640 1 --2- 172.21.15.76:0/139398012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13ac15dc10 0x7f13ac1f0db0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.047+0000 7f13b31f6640 1 --2- 172.21.15.76:0/139398012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13ac0ce3f0 0x7f13ac1f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.047+0000 7f13b31f6640 1 -- 172.21.15.76:0/139398012 >> 172.21.15.76:0/139398012 conn(0x7f13ac06f5e0 msgr2=0x7f13ac15eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:06.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.047+0000 7f13b31f6640 1 -- 172.21.15.76:0/139398012 shutdown_connections 2024-03-08T06:01:06.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.047+0000 7f13b31f6640 1 -- 172.21.15.76:0/139398012 wait complete. 2024-03-08T06:01:06.059 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:06.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.158+0000 7f787ddb6640 1 Processor -- start 2024-03-08T06:01:06.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.158+0000 7f787ddb6640 1 -- start start 2024-03-08T06:01:06.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f787ddb6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f787815e040 0x7f7878099b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f787ddb6640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f787806bfa0 0x7f787809a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f787ddb6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78780cfcc0 con 0x7f787815dc60 2024-03-08T06:01:06.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f787ddb6640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78780b4360 con 0x7f787815e040 2024-03-08T06:01:06.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f787ddb6640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78780cff60 con 0x7f787806bfa0 2024-03-08T06:01:06.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f78777fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f787815dc60 0x7f78780d1000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57658/0 (socket says 172.21.15.76:57658) 2024-03-08T06:01:06.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f78777fe640 1 -- 172.21.15.76:0/4247060519 learned_addr learned my addr 172.21.15.76:0/4247060519 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:06.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f7876ffd640 1 --2- 172.21.15.76:0/4247060519 >> v2:172.21.15.81:3300/0 conn(0x7f787815e040 0x7f7878099b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f7877fff640 1 --2- 172.21.15.76:0/4247060519 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f787806bfa0 0x7f787809a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f78767fc640 1 -- 172.21.15.76:0/4247060519 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1964708399 0 0) 0x7f78780cfcc0 con 0x7f787815dc60 2024-03-08T06:01:06.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f78767fc640 1 -- 172.21.15.76:0/4247060519 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7860003520 con 0x7f787815dc60 2024-03-08T06:01:06.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f78767fc640 1 -- 172.21.15.76:0/4247060519 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 367601812 0 0) 0x7f7860003520 con 0x7f787815dc60 2024-03-08T06:01:06.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f7877fff640 1 -- 172.21.15.76:0/4247060519 >> v2:172.21.15.81:3300/0 conn(0x7f787815e040 msgr2=0x7f7878099b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:06.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f7877fff640 1 --2- 172.21.15.76:0/4247060519 >> v2:172.21.15.81:3300/0 conn(0x7f787815e040 0x7f7878099b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f7877fff640 1 -- 172.21.15.76:0/4247060519 >> v1:172.21.15.76:6789/0 conn(0x7f787815dc60 legacy=0x7f78780d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f7877fff640 1 -- 172.21.15.76:0/4247060519 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78780d0240 con 0x7f787806bfa0 2024-03-08T06:01:06.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.159+0000 7f7876ffd640 1 --2- 172.21.15.76:0/4247060519 >> v2:172.21.15.81:3300/0 conn(0x7f787815e040 0x7f7878099b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:06.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.160+0000 7f7877fff640 1 --2- 172.21.15.76:0/4247060519 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f787806bfa0 0x7f787809a080 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7870009d90 tx=0x7f787002bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b07ddfcd6e0061b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:06.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.160+0000 7f78767fc640 1 -- 172.21.15.76:0/4247060519 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870039070 con 0x7f787806bfa0 2024-03-08T06:01:06.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.160+0000 7f78767fc640 1 -- 172.21.15.76:0/4247060519 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787002c8a0 con 0x7f787806bfa0 2024-03-08T06:01:06.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.160+0000 7f78767fc640 1 -- 172.21.15.76:0/4247060519 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868001690 con 0x7f787806bfa0 2024-03-08T06:01:06.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.161+0000 7f787ddb6640 1 -- 172.21.15.76:0/4247060519 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f787806bfa0 msgr2=0x7f787809a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.161+0000 7f787ddb6640 1 --2- 172.21.15.76:0/4247060519 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f787806bfa0 0x7f787809a080 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7870009d90 tx=0x7f787002bd70 comp rx=0 tx=0).stop 2024-03-08T06:01:06.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.161+0000 7f787ddb6640 1 -- 172.21.15.76:0/4247060519 shutdown_connections 2024-03-08T06:01:06.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.161+0000 7f787ddb6640 1 --2- 172.21.15.76:0/4247060519 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f787806bfa0 0x7f787809a080 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.161+0000 7f787ddb6640 1 --2- 172.21.15.76:0/4247060519 >> v2:172.21.15.81:3300/0 conn(0x7f787815e040 0x7f7878099b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.161+0000 7f787ddb6640 1 -- 172.21.15.76:0/4247060519 >> 172.21.15.76:0/4247060519 conn(0x7f78780b3440 msgr2=0x7f78780b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:06.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.161+0000 7f787ddb6640 1 -- 172.21.15.76:0/4247060519 shutdown_connections 2024-03-08T06:01:06.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.161+0000 7f787ddb6640 1 -- 172.21.15.76:0/4247060519 wait complete. 2024-03-08T06:01:06.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.162+0000 7f787ddb6640 1 Processor -- start 2024-03-08T06:01:06.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.162+0000 7f787ddb6640 1 -- start start 2024-03-08T06:01:06.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f787ddb6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f787806bfa0 0x7f78781f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f787ddb6640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f787815dc60 0x7f787815e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f787ddb6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f787815e040 0x7f7878070850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f787ddb6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78780d0ae0 con 0x7f787815e040 2024-03-08T06:01:06.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f787ddb6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78780d0800 con 0x7f787806bfa0 2024-03-08T06:01:06.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f787ddb6640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787809a890 con 0x7f787815dc60 2024-03-08T06:01:06.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f7877fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f787815e040 0x7f7878070850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f7877fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f787815e040 0x7f7878070850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51824/0 (socket says 172.21.15.76:51824) 2024-03-08T06:01:06.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f7877fff640 1 -- 172.21.15.76:0/656373719 learned_addr learned my addr 172.21.15.76:0/656373719 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:06.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f7876ffd640 1 --2- 172.21.15.76:0/656373719 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f787815dc60 0x7f787815e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f78777fe640 1 --2- 172.21.15.76:0/656373719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f787806bfa0 0x7f78781f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f7877fff640 1 -- 172.21.15.76:0/656373719 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f787815dc60 msgr2=0x7f787815e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f7877fff640 1 --2- 172.21.15.76:0/656373719 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f787815dc60 0x7f787815e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f7877fff640 1 -- 172.21.15.76:0/656373719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f787806bfa0 msgr2=0x7f78781f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f7877fff640 1 --2- 172.21.15.76:0/656373719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f787806bfa0 0x7f78781f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f7877fff640 1 -- 172.21.15.76:0/656373719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787815ee40 con 0x7f787815e040 2024-03-08T06:01:06.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.163+0000 7f78777fe640 1 --2- 172.21.15.76:0/656373719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f787806bfa0 0x7f78781f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:06.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.164+0000 7f7877fff640 1 --2- 172.21.15.76:0/656373719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f787815e040 0x7f7878070850 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7870037040 tx=0x7f7870004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:06.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.164+0000 7f785ffff640 1 -- 172.21.15.76:0/656373719 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870039050 con 0x7f787815e040 2024-03-08T06:01:06.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.164+0000 7f785ffff640 1 -- 172.21.15.76:0/656373719 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7870045070 con 0x7f787815e040 2024-03-08T06:01:06.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.164+0000 7f787ddb6640 1 -- 172.21.15.76:0/656373719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787815f120 con 0x7f787815e040 2024-03-08T06:01:06.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.164+0000 7f785ffff640 1 -- 172.21.15.76:0/656373719 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870009450 con 0x7f787815e040 2024-03-08T06:01:06.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.164+0000 7f787ddb6640 1 -- 172.21.15.76:0/656373719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78780c6450 con 0x7f787815e040 2024-03-08T06:01:06.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.165+0000 7f787ddb6640 1 -- 172.21.15.76:0/656373719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787804f030 con 0x7f787815e040 2024-03-08T06:01:06.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.165+0000 7f785ffff640 1 -- 172.21.15.76:0/656373719 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7870040690 con 0x7f787815e040 2024-03-08T06:01:06.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.165+0000 7f785ffff640 1 --2- 172.21.15.76:0/656373719 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f786406b4d0 0x7f786406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.166+0000 7f785ffff640 1 -- 172.21.15.76:0/656373719 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f78700acd10 con 0x7f787815e040 2024-03-08T06:01:06.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.166+0000 7f78777fe640 1 --2- 172.21.15.76:0/656373719 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f786406b4d0 0x7f786406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.166+0000 7f78777fe640 1 --2- 172.21.15.76:0/656373719 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f786406b4d0 0x7f786406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7868001200 tx=0x7f7868001da0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:06.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.170+0000 7f785ffff640 1 -- 172.21.15.76:0/656373719 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f787007b8e0 con 0x7f787815e040 2024-03-08T06:01:06.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.345+0000 7f787ddb6640 1 -- 172.21.15.76:0/656373719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f787809a670 con 0x7f787815e040 2024-03-08T06:01:06.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.347+0000 7f785ffff640 1 -- 172.21.15.76:0/656373719 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f787007b1e0 con 0x7f787815e040 2024-03-08T06:01:06.347 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:06.347 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":26,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:42.202011+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:06.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.349+0000 7f787ddb6640 1 -- 172.21.15.76:0/656373719 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f786406b4d0 msgr2=0x7f786406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.349+0000 7f787ddb6640 1 --2- 172.21.15.76:0/656373719 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f786406b4d0 0x7f786406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7868001200 tx=0x7f7868001da0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.349+0000 7f787ddb6640 1 -- 172.21.15.76:0/656373719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f787815e040 msgr2=0x7f7878070850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.349+0000 7f787ddb6640 1 --2- 172.21.15.76:0/656373719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f787815e040 0x7f7878070850 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7870037040 tx=0x7f7870004a40 comp rx=0 tx=0).stop 2024-03-08T06:01:06.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.349+0000 7f787ddb6640 1 -- 172.21.15.76:0/656373719 shutdown_connections 2024-03-08T06:01:06.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.349+0000 7f787ddb6640 1 --2- 172.21.15.76:0/656373719 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f786406b4d0 0x7f786406d990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.349+0000 7f787ddb6640 1 --2- 172.21.15.76:0/656373719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f787815e040 0x7f7878070850 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.349+0000 7f787ddb6640 1 --2- 172.21.15.76:0/656373719 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f787815dc60 0x7f787815e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.349+0000 7f787ddb6640 1 --2- 172.21.15.76:0/656373719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f787806bfa0 0x7f78781f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.349+0000 7f787ddb6640 1 -- 172.21.15.76:0/656373719 >> 172.21.15.76:0/656373719 conn(0x7f78780b3440 msgr2=0x7f78780d1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:06.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.349+0000 7f787ddb6640 1 -- 172.21.15.76:0/656373719 shutdown_connections 2024-03-08T06:01:06.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.349+0000 7f787ddb6640 1 -- 172.21.15.76:0/656373719 wait complete. 2024-03-08T06:01:06.364 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:00:33.926651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:00:34.474030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T06:00:38.207853+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T06:01:06.365 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:01:06.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.467+0000 7fb729ba0640 1 Processor -- start 2024-03-08T06:01:06.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.467+0000 7fb729ba0640 1 -- start start 2024-03-08T06:01:06.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.468+0000 7fb729ba0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb7240cef90 0x7fb7240d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.468+0000 7fb729ba0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb7240d1680 0x7fb7240b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.468+0000 7fb729ba0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7240d1c00 con 0x7fb72415de30 2024-03-08T06:01:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.468+0000 7fb729ba0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7240d3820 con 0x7fb7240d1680 2024-03-08T06:01:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.468+0000 7fb729ba0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7240d2810 con 0x7fb7240cef90 2024-03-08T06:01:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.468+0000 7fb728b9e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb72415de30 0x7fb7240ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57670/0 (socket says 172.21.15.76:57670) 2024-03-08T06:01:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.468+0000 7fb728b9e640 1 -- 172.21.15.76:0/16133863 learned_addr learned my addr 172.21.15.76:0/16133863 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.468+0000 7fb72939f640 1 --2- 172.21.15.76:0/16133863 >> v2:172.21.15.81:3300/0 conn(0x7fb7240d1680 0x7fb7240b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.468+0000 7fb723fff640 1 --2- 172.21.15.76:0/16133863 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb7240cef90 0x7fb7240d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.468+0000 7fb7237fe640 1 -- 172.21.15.76:0/16133863 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256491825 0 0) 0x7fb7240d1c00 con 0x7fb72415de30 2024-03-08T06:01:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.468+0000 7fb7237fe640 1 -- 172.21.15.76:0/16133863 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb710003520 con 0x7fb72415de30 2024-03-08T06:01:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb7237fe640 1 -- 172.21.15.76:0/16133863 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1986154273 0 0) 0x7fb710003520 con 0x7fb72415de30 2024-03-08T06:01:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb7237fe640 1 -- 172.21.15.76:0/16133863 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb7240cef90 msgr2=0x7fb7240d2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb7237fe640 1 --2- 172.21.15.76:0/16133863 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb7240cef90 0x7fb7240d2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb7237fe640 1 -- 172.21.15.76:0/16133863 >> v2:172.21.15.81:3300/0 conn(0x7fb7240d1680 msgr2=0x7fb7240b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb7237fe640 1 --2- 172.21.15.76:0/16133863 >> v2:172.21.15.81:3300/0 conn(0x7fb7240d1680 0x7fb7240b7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb7237fe640 1 -- 172.21.15.76:0/16133863 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7240d1ee0 con 0x7fb72415de30 2024-03-08T06:01:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb7237fe640 1 -- 172.21.15.76:0/16133863 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fb71c0019f0 con 0x7fb72415de30 2024-03-08T06:01:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb7237fe640 1 -- 172.21.15.76:0/16133863 >> v1:172.21.15.76:6789/0 conn(0x7fb72415de30 legacy=0x7fb7240ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb7237fe640 1 --2- 172.21.15.76:0/16133863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7100039b0 0x7fb710003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb7237fe640 1 --2- 172.21.15.76:0/16133863 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb7240cef90 0x7fb710004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb7237fe640 1 --2- 172.21.15.76:0/16133863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb710004d40 0x7fb7100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb7237fe640 1 -- 172.21.15.76:0/16133863 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7240d3820 con 0x7fb7100039b0 2024-03-08T06:01:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb7237fe640 1 -- 172.21.15.76:0/16133863 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7240d2810 con 0x7fb710004d40 2024-03-08T06:01:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb7237fe640 1 -- 172.21.15.76:0/16133863 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71000b9c0 con 0x7fb7240cef90 2024-03-08T06:01:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb728b9e640 1 --2- 172.21.15.76:0/16133863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7100039b0 0x7fb710003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb723fff640 1 --2- 172.21.15.76:0/16133863 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb7240cef90 0x7fb710004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb72939f640 1 --2- 172.21.15.76:0/16133863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb710004d40 0x7fb7100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb728b9e640 1 -- 172.21.15.76:0/16133863 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb7240cef90 msgr2=0x7fb710004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb728b9e640 1 --2- 172.21.15.76:0/16133863 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb7240cef90 0x7fb710004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb728b9e640 1 -- 172.21.15.76:0/16133863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb710004d40 msgr2=0x7fb7100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:06.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb728b9e640 1 --2- 172.21.15.76:0/16133863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb710004d40 0x7fb7100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.469+0000 7fb728b9e640 1 -- 172.21.15.76:0/16133863 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb710003520 con 0x7fb7100039b0 2024-03-08T06:01:06.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.470+0000 7fb728b9e640 1 --2- 172.21.15.76:0/16133863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7100039b0 0x7fb710003dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0013d0 tx=0x7fb710005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db240f014c1d8c35 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:06.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.470+0000 7fb7237fe640 1 -- 172.21.15.76:0/16133863 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00a4e0 con 0x7fb7100039b0 2024-03-08T06:01:06.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.470+0000 7fb7237fe640 1 -- 172.21.15.76:0/16133863 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71c034650 con 0x7fb7100039b0 2024-03-08T06:01:06.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.470+0000 7fb7237fe640 1 -- 172.21.15.76:0/16133863 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c034a40 con 0x7fb7100039b0 2024-03-08T06:01:06.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.471+0000 7fb729ba0640 1 -- 172.21.15.76:0/16133863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7100039b0 msgr2=0x7fb710003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.471+0000 7fb729ba0640 1 --2- 172.21.15.76:0/16133863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7100039b0 0x7fb710003dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0013d0 tx=0x7fb710005830 comp rx=0 tx=0).stop 2024-03-08T06:01:06.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.471+0000 7fb72939f640 1 -- 172.21.15.76:0/16133863 reap_dead start 2024-03-08T06:01:06.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.471+0000 7fb729ba0640 1 -- 172.21.15.76:0/16133863 shutdown_connections 2024-03-08T06:01:06.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.471+0000 7fb729ba0640 1 -- 172.21.15.76:0/16133863 >> 172.21.15.76:0/16133863 conn(0x7fb724067070 msgr2=0x7fb7240d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:06.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.471+0000 7fb729ba0640 1 -- 172.21.15.76:0/16133863 shutdown_connections 2024-03-08T06:01:06.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.471+0000 7fb729ba0640 1 -- 172.21.15.76:0/16133863 wait complete. 2024-03-08T06:01:06.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.472+0000 7fb729ba0640 1 Processor -- start 2024-03-08T06:01:06.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.472+0000 7fb729ba0640 1 -- start start 2024-03-08T06:01:06.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb729ba0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7100039b0 0x7fb7240b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb729ba0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb72415e210 0x7fb72414d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb729ba0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7240c9120 0x7fb7240c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb729ba0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7240d2c20 con 0x7fb7100039b0 2024-03-08T06:01:06.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb729ba0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7240d24a0 con 0x7fb7240c9120 2024-03-08T06:01:06.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb729ba0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7240d2da0 con 0x7fb72415e210 2024-03-08T06:01:06.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb728b9e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7100039b0 0x7fb7240b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb728b9e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7100039b0 0x7fb7240b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51834/0 (socket says 172.21.15.76:51834) 2024-03-08T06:01:06.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb728b9e640 1 -- 172.21.15.76:0/2039932573 learned_addr learned my addr 172.21.15.76:0/2039932573 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:06.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb723fff640 1 --2- 172.21.15.76:0/2039932573 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb72415e210 0x7fb72414d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb72939f640 1 --2- 172.21.15.76:0/2039932573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7240c9120 0x7fb7240c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb728b9e640 1 -- 172.21.15.76:0/2039932573 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb72415e210 msgr2=0x7fb72414d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb728b9e640 1 --2- 172.21.15.76:0/2039932573 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb72415e210 0x7fb72414d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb728b9e640 1 -- 172.21.15.76:0/2039932573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7240c9120 msgr2=0x7fb7240c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb728b9e640 1 --2- 172.21.15.76:0/2039932573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7240c9120 0x7fb7240c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb728b9e640 1 -- 172.21.15.76:0/2039932573 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72414dbb0 con 0x7fb7100039b0 2024-03-08T06:01:06.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb728b9e640 1 --2- 172.21.15.76:0/2039932573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7100039b0 0x7fb7240b2f60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00a980 tx=0x7fb71c001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.473+0000 7fb7217fa640 1 -- 172.21.15.76:0/2039932573 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c044070 con 0x7fb7100039b0 2024-03-08T06:01:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.474+0000 7fb7217fa640 1 -- 172.21.15.76:0/2039932573 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71c001a30 con 0x7fb7100039b0 2024-03-08T06:01:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.474+0000 7fb729ba0640 1 -- 172.21.15.76:0/2039932573 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72415e7b0 con 0x7fb7100039b0 2024-03-08T06:01:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.474+0000 7fb7217fa640 1 -- 172.21.15.76:0/2039932573 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c039040 con 0x7fb7100039b0 2024-03-08T06:01:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.474+0000 7fb729ba0640 1 -- 172.21.15.76:0/2039932573 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72415ed70 con 0x7fb7100039b0 2024-03-08T06:01:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.475+0000 7fb729ba0640 1 -- 172.21.15.76:0/2039932573 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72406a1d0 con 0x7fb7100039b0 2024-03-08T06:01:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.475+0000 7fb7217fa640 1 -- 172.21.15.76:0/2039932573 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb71c047070 con 0x7fb7100039b0 2024-03-08T06:01:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.475+0000 7fb7217fa640 1 --2- 172.21.15.76:0/2039932573 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb70406b4d0 0x7fb70406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.476+0000 7fb723fff640 1 --2- 172.21.15.76:0/2039932573 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb70406b4d0 0x7fb70406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.476+0000 7fb7217fa640 1 -- 172.21.15.76:0/2039932573 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb71c0abed0 con 0x7fb7100039b0 2024-03-08T06:01:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.476+0000 7fb723fff640 1 --2- 172.21.15.76:0/2039932573 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb70406b4d0 0x7fb70406d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb7240c9f40 tx=0x7fb714009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:06.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.480+0000 7fb7217fa640 1 -- 172.21.15.76:0/2039932573 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb72414dbb0 con 0x7fb7100039b0 2024-03-08T06:01:06.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.651+0000 7fb729ba0640 1 -- 172.21.15.76:0/2039932573 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb724067f10 con 0x7fb7100039b0 2024-03-08T06:01:06.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.651+0000 7fb7217fa640 1 -- 172.21.15.76:0/2039932573 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb71c07aad0 con 0x7fb7100039b0 2024-03-08T06:01:06.652 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:01:06.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.653+0000 7fb729ba0640 1 -- 172.21.15.76:0/2039932573 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb70406b4d0 msgr2=0x7fb70406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.653+0000 7fb729ba0640 1 --2- 172.21.15.76:0/2039932573 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb70406b4d0 0x7fb70406d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb7240c9f40 tx=0x7fb714009190 comp rx=0 tx=0).stop 2024-03-08T06:01:06.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.653+0000 7fb729ba0640 1 -- 172.21.15.76:0/2039932573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7100039b0 msgr2=0x7fb7240b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.653+0000 7fb729ba0640 1 --2- 172.21.15.76:0/2039932573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7100039b0 0x7fb7240b2f60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00a980 tx=0x7fb71c001360 comp rx=0 tx=0).stop 2024-03-08T06:01:06.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.653+0000 7fb729ba0640 1 -- 172.21.15.76:0/2039932573 shutdown_connections 2024-03-08T06:01:06.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.653+0000 7fb729ba0640 1 --2- 172.21.15.76:0/2039932573 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb70406b4d0 0x7fb70406d990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.653+0000 7fb729ba0640 1 --2- 172.21.15.76:0/2039932573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7240c9120 0x7fb7240c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.653+0000 7fb729ba0640 1 --2- 172.21.15.76:0/2039932573 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb72415e210 0x7fb72414d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.653+0000 7fb729ba0640 1 --2- 172.21.15.76:0/2039932573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7100039b0 0x7fb7240b2f60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.653+0000 7fb729ba0640 1 -- 172.21.15.76:0/2039932573 >> 172.21.15.76:0/2039932573 conn(0x7fb724067070 msgr2=0x7fb72415c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:06.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.653+0000 7fb729ba0640 1 -- 172.21.15.76:0/2039932573 shutdown_connections 2024-03-08T06:01:06.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.653+0000 7fb729ba0640 1 -- 172.21.15.76:0/2039932573 wait complete. 2024-03-08T06:01:06.665 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:01:06.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.767+0000 7f8882284640 1 Processor -- start 2024-03-08T06:01:06.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.767+0000 7f8882284640 1 -- start start 2024-03-08T06:01:06.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.768+0000 7f8882284640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f887c06ac80 0x7f887c0d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.768+0000 7f8882284640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f887c15bd70 0x7f887c0ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.768+0000 7f8882284640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887c0d05e0 con 0x7f887c0b2f20 2024-03-08T06:01:06.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.768+0000 7f8882284640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c0cc5b0 con 0x7f887c06ac80 2024-03-08T06:01:06.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.768+0000 7f8882284640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0cc890 con 0x7f887c15bd70 2024-03-08T06:01:06.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.768+0000 7f887bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f887c06ac80 0x7f887c0d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.768+0000 7f887b7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f887c0b2f20 0x7f887c06b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57674/0 (socket says 172.21.15.76:57674) 2024-03-08T06:01:06.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.768+0000 7f887b7fe640 1 -- 172.21.15.76:0/2076130480 learned_addr learned my addr 172.21.15.76:0/2076130480 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:06.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.769+0000 7f8880821640 1 --2- 172.21.15.76:0/2076130480 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f887c15bd70 0x7f887c0ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.769+0000 7f887bfff640 1 -- 172.21.15.76:0/2076130480 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f887c15bd70 msgr2=0x7f887c0ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:06.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.769+0000 7f887bfff640 1 --2- 172.21.15.76:0/2076130480 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f887c15bd70 0x7f887c0ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.769+0000 7f887bfff640 1 -- 172.21.15.76:0/2076130480 >> v1:172.21.15.76:6789/0 conn(0x7f887c0b2f20 legacy=0x7f887c06b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.769+0000 7f887bfff640 1 -- 172.21.15.76:0/2076130480 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c0ccb70 con 0x7f887c06ac80 2024-03-08T06:01:06.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.769+0000 7f887affd640 1 -- 172.21.15.76:0/2076130480 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759539731 0 0) 0x7f887c0d05e0 con 0x7f887c0b2f20 2024-03-08T06:01:06.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.769+0000 7f887bfff640 1 --2- 172.21.15.76:0/2076130480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c06ac80 0x7f887c0d00a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f886c0097a0 tx=0x7f886c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2e4fc56e38807e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:06.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.769+0000 7f887affd640 1 -- 172.21.15.76:0/2076130480 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c035020 con 0x7f887c06ac80 2024-03-08T06:01:06.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.769+0000 7f887affd640 1 -- 172.21.15.76:0/2076130480 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c031830 con 0x7f887c06ac80 2024-03-08T06:01:06.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.769+0000 7f887affd640 1 -- 172.21.15.76:0/2076130480 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c031c00 con 0x7f887c06ac80 2024-03-08T06:01:06.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.770+0000 7f8882284640 1 -- 172.21.15.76:0/2076130480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c06ac80 msgr2=0x7f887c0d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.770+0000 7f8882284640 1 --2- 172.21.15.76:0/2076130480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c06ac80 0x7f887c0d00a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f886c0097a0 tx=0x7f886c02b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.770+0000 7f8882284640 1 -- 172.21.15.76:0/2076130480 shutdown_connections 2024-03-08T06:01:06.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.770+0000 7f8882284640 1 --2- 172.21.15.76:0/2076130480 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f887c15bd70 0x7f887c0ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.770+0000 7f8882284640 1 --2- 172.21.15.76:0/2076130480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c06ac80 0x7f887c0d00a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.770+0000 7f8882284640 1 -- 172.21.15.76:0/2076130480 >> 172.21.15.76:0/2076130480 conn(0x7f887c0b2320 msgr2=0x7f887c0b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:06.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.770+0000 7f8882284640 1 -- 172.21.15.76:0/2076130480 shutdown_connections 2024-03-08T06:01:06.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.770+0000 7f8882284640 1 -- 172.21.15.76:0/2076130480 wait complete. 2024-03-08T06:01:06.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.771+0000 7f8882284640 1 Processor -- start 2024-03-08T06:01:06.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.771+0000 7f8882284640 1 -- start start 2024-03-08T06:01:06.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.772+0000 7f8882284640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f887c06ac80 0x7f887c146cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.772+0000 7f8882284640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c0b2f20 0x7f887c0c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.772+0000 7f8882284640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f887c15bd70 0x7f887c0c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.772+0000 7f8882284640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0cc1d0 con 0x7f887c0b2f20 2024-03-08T06:01:06.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.772+0000 7f8882284640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0cd130 con 0x7f887c15bd70 2024-03-08T06:01:06.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.772+0000 7f8882284640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0cc350 con 0x7f887c06ac80 2024-03-08T06:01:06.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.772+0000 7f887b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c0b2f20 0x7f887c0c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.772+0000 7f887b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c0b2f20 0x7f887c0c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51852/0 (socket says 172.21.15.76:51852) 2024-03-08T06:01:06.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.772+0000 7f887b7fe640 1 -- 172.21.15.76:0/1462123865 learned_addr learned my addr 172.21.15.76:0/1462123865 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:06.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.772+0000 7f887bfff640 1 --2- 172.21.15.76:0/1462123865 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f887c06ac80 0x7f887c146cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.772+0000 7f887b7fe640 1 -- 172.21.15.76:0/1462123865 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f887c06ac80 msgr2=0x7f887c146cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.772+0000 7f8880821640 1 --2- 172.21.15.76:0/1462123865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f887c15bd70 0x7f887c0c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.772+0000 7f887b7fe640 1 --2- 172.21.15.76:0/1462123865 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f887c06ac80 0x7f887c146cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.772+0000 7f887b7fe640 1 -- 172.21.15.76:0/1462123865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f887c15bd70 msgr2=0x7f887c0c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.772+0000 7f887b7fe640 1 --2- 172.21.15.76:0/1462123865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f887c15bd70 0x7f887c0c4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.772+0000 7f887b7fe640 1 -- 172.21.15.76:0/1462123865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c0c4610 con 0x7f887c0b2f20 2024-03-08T06:01:06.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.773+0000 7f887b7fe640 1 --2- 172.21.15.76:0/1462123865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c0b2f20 0x7f887c0c6b10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f886400c4d0 tx=0x7f886400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:06.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.773+0000 7f8878ff9640 1 -- 172.21.15.76:0/1462123865 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864018070 con 0x7f887c0b2f20 2024-03-08T06:01:06.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.773+0000 7f8878ff9640 1 -- 172.21.15.76:0/1462123865 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8864007500 con 0x7f887c0b2f20 2024-03-08T06:01:06.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.773+0000 7f8878ff9640 1 -- 172.21.15.76:0/1462123865 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864004a20 con 0x7f887c0b2f20 2024-03-08T06:01:06.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.773+0000 7f8882284640 1 -- 172.21.15.76:0/1462123865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887c0c48f0 con 0x7f887c0b2f20 2024-03-08T06:01:06.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.773+0000 7f8882284640 1 -- 172.21.15.76:0/1462123865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887c0c7050 con 0x7f887c0b2f20 2024-03-08T06:01:06.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.774+0000 7f8882284640 1 -- 172.21.15.76:0/1462123865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887c04f030 con 0x7f887c0b2f20 2024-03-08T06:01:06.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.774+0000 7f8878ff9640 1 -- 172.21.15.76:0/1462123865 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f886401b070 con 0x7f887c0b2f20 2024-03-08T06:01:06.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.775+0000 7f8878ff9640 1 --2- 172.21.15.76:0/1462123865 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f885c06b4d0 0x7f885c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:06.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.775+0000 7f8878ff9640 1 -- 172.21.15.76:0/1462123865 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8864089840 con 0x7f887c0b2f20 2024-03-08T06:01:06.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.775+0000 7f887bfff640 1 --2- 172.21.15.76:0/1462123865 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f885c06b4d0 0x7f885c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:06.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.775+0000 7f887bfff640 1 --2- 172.21.15.76:0/1462123865 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f885c06b4d0 0x7f885c06d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f886c036000 tx=0x7f886c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:06.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.779+0000 7f8878ff9640 1 -- 172.21.15.76:0/1462123865 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8864058380 con 0x7f887c0b2f20 2024-03-08T06:01:06.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.947+0000 7f8882284640 1 -- 172.21.15.76:0/1462123865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f887c0b0070 con 0x7f887c0b2f20 2024-03-08T06:01:06.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.948+0000 7f8878ff9640 1 -- 172.21.15.76:0/1462123865 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8864057c80 con 0x7f887c0b2f20 2024-03-08T06:01:06.948 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.950+0000 7f8882284640 1 -- 172.21.15.76:0/1462123865 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f885c06b4d0 msgr2=0x7f885c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.950+0000 7f8882284640 1 --2- 172.21.15.76:0/1462123865 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f885c06b4d0 0x7f885c06d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f886c036000 tx=0x7f886c008030 comp rx=0 tx=0).stop 2024-03-08T06:01:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.950+0000 7f8882284640 1 -- 172.21.15.76:0/1462123865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c0b2f20 msgr2=0x7f887c0c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.950+0000 7f8882284640 1 --2- 172.21.15.76:0/1462123865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c0b2f20 0x7f887c0c6b10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f886400c4d0 tx=0x7f886400c990 comp rx=0 tx=0).stop 2024-03-08T06:01:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.950+0000 7f8882284640 1 -- 172.21.15.76:0/1462123865 shutdown_connections 2024-03-08T06:01:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.950+0000 7f8882284640 1 --2- 172.21.15.76:0/1462123865 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f885c06b4d0 0x7f885c06d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.950+0000 7f8882284640 1 --2- 172.21.15.76:0/1462123865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f887c15bd70 0x7f887c0c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.950+0000 7f8882284640 1 --2- 172.21.15.76:0/1462123865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c0b2f20 0x7f887c0c6b10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.950+0000 7f8882284640 1 --2- 172.21.15.76:0/1462123865 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f887c06ac80 0x7f887c146cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.950+0000 7f8882284640 1 -- 172.21.15.76:0/1462123865 >> 172.21.15.76:0/1462123865 conn(0x7f887c0b2320 msgr2=0x7f887c0adf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.950+0000 7f8882284640 1 -- 172.21.15.76:0/1462123865 shutdown_connections 2024-03-08T06:01:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:06.950+0000 7f8882284640 1 -- 172.21.15.76:0/1462123865 wait complete. 2024-03-08T06:01:06.961 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:01:07.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.064+0000 7f109f1b6640 1 Processor -- start 2024-03-08T06:01:07.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.064+0000 7f109f1b6640 1 -- start start 2024-03-08T06:01:07.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.065+0000 7f109f1b6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f109815dcb0 0x7f10980ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.065+0000 7f109f1b6640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f10980af950 0x7f109806fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.065+0000 7f109f1b6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10980cd580 con 0x7f10980afdb0 2024-03-08T06:01:07.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.065+0000 7f109f1b6640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10980cd860 con 0x7f109815dcb0 2024-03-08T06:01:07.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.065+0000 7f109f1b6640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10980cdb40 con 0x7f10980af950 2024-03-08T06:01:07.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.065+0000 7f109d753640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f10980afdb0 0x7f109806b040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57676/0 (socket says 172.21.15.76:57676) 2024-03-08T06:01:07.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.065+0000 7f109cf52640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f109815dcb0 0x7f10980ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.065+0000 7f109d753640 1 -- 172.21.15.76:0/2015097112 learned_addr learned my addr 172.21.15.76:0/2015097112 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:07.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.065+0000 7f1097fff640 1 --2- 172.21.15.76:0/2015097112 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f10980af950 0x7f109806fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.065+0000 7f109cf52640 1 -- 172.21.15.76:0/2015097112 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f10980af950 msgr2=0x7f109806fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:07.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.065+0000 7f109cf52640 1 --2- 172.21.15.76:0/2015097112 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f10980af950 0x7f109806fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.065+0000 7f109cf52640 1 -- 172.21.15.76:0/2015097112 >> v1:172.21.15.76:6789/0 conn(0x7f10980afdb0 legacy=0x7f109806b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.065+0000 7f10977fe640 1 -- 172.21.15.76:0/2015097112 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3974870608 0 0) 0x7f10980cd580 con 0x7f10980afdb0 2024-03-08T06:01:07.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.065+0000 7f109cf52640 1 -- 172.21.15.76:0/2015097112 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10980cde20 con 0x7f109815dcb0 2024-03-08T06:01:07.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.066+0000 7f109cf52640 1 --2- 172.21.15.76:0/2015097112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f109815dcb0 0x7f10980ad600 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f108c0097a0 tx=0x7f108c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed16183656caa895 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:07.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.066+0000 7f10977fe640 1 -- 172.21.15.76:0/2015097112 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c035020 con 0x7f109815dcb0 2024-03-08T06:01:07.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.066+0000 7f10977fe640 1 -- 172.21.15.76:0/2015097112 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c031830 con 0x7f109815dcb0 2024-03-08T06:01:07.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.066+0000 7f10977fe640 1 -- 172.21.15.76:0/2015097112 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c031c00 con 0x7f109815dcb0 2024-03-08T06:01:07.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.067+0000 7f109f1b6640 1 -- 172.21.15.76:0/2015097112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f109815dcb0 msgr2=0x7f10980ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.067+0000 7f109f1b6640 1 --2- 172.21.15.76:0/2015097112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f109815dcb0 0x7f10980ad600 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f108c0097a0 tx=0x7f108c02b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.067+0000 7f109f1b6640 1 -- 172.21.15.76:0/2015097112 shutdown_connections 2024-03-08T06:01:07.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.067+0000 7f109f1b6640 1 --2- 172.21.15.76:0/2015097112 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f10980af950 0x7f109806fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.067+0000 7f109f1b6640 1 --2- 172.21.15.76:0/2015097112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f109815dcb0 0x7f10980ad600 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.067+0000 7f109f1b6640 1 -- 172.21.15.76:0/2015097112 >> 172.21.15.76:0/2015097112 conn(0x7f10980b2320 msgr2=0x7f10980b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:07.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.067+0000 7f109f1b6640 1 -- 172.21.15.76:0/2015097112 shutdown_connections 2024-03-08T06:01:07.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.067+0000 7f109f1b6640 1 -- 172.21.15.76:0/2015097112 wait complete. 2024-03-08T06:01:07.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.068+0000 7f109f1b6640 1 Processor -- start 2024-03-08T06:01:07.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.068+0000 7f109f1b6640 1 -- start start 2024-03-08T06:01:07.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109f1b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10980af950 0x7f109809a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109f1b6640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f10980afdb0 0x7f109814fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109f1b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f109815dcb0 0x7f10981501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109f1b6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10980cd360 con 0x7f109815dcb0 2024-03-08T06:01:07.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109f1b6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10980cd1e0 con 0x7f10980af950 2024-03-08T06:01:07.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109f1b6640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109815e090 con 0x7f10980afdb0 2024-03-08T06:01:07.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109d753640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f109815dcb0 0x7f10981501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109d753640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f109815dcb0 0x7f10981501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51866/0 (socket says 172.21.15.76:51866) 2024-03-08T06:01:07.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109d753640 1 -- 172.21.15.76:0/2171079631 learned_addr learned my addr 172.21.15.76:0/2171079631 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:07.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109cf52640 1 --2- 172.21.15.76:0/2171079631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10980af950 0x7f109809a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f1097fff640 1 --2- 172.21.15.76:0/2171079631 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f10980afdb0 0x7f109814fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109d753640 1 -- 172.21.15.76:0/2171079631 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f10980afdb0 msgr2=0x7f109814fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109d753640 1 --2- 172.21.15.76:0/2171079631 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f10980afdb0 0x7f109814fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109d753640 1 -- 172.21.15.76:0/2171079631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10980af950 msgr2=0x7f109809a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109d753640 1 --2- 172.21.15.76:0/2171079631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10980af950 0x7f109809a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109d753640 1 -- 172.21.15.76:0/2171079631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10981595d0 con 0x7f109815dcb0 2024-03-08T06:01:07.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109d753640 1 --2- 172.21.15.76:0/2171079631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f109815dcb0 0x7f10981501c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f109000c700 tx=0x7f109000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:07.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f10957fa640 1 -- 172.21.15.76:0/2171079631 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090018070 con 0x7f109815dcb0 2024-03-08T06:01:07.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f10957fa640 1 -- 172.21.15.76:0/2171079631 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109000ce10 con 0x7f109815dcb0 2024-03-08T06:01:07.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109f1b6640 1 -- 172.21.15.76:0/2171079631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109814a910 con 0x7f109815dcb0 2024-03-08T06:01:07.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f10957fa640 1 -- 172.21.15.76:0/2171079631 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109000a860 con 0x7f109815dcb0 2024-03-08T06:01:07.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.069+0000 7f109f1b6640 1 -- 172.21.15.76:0/2171079631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109814aed0 con 0x7f109815dcb0 2024-03-08T06:01:07.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.070+0000 7f109f1b6640 1 -- 172.21.15.76:0/2171079631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10980cecc0 con 0x7f109815dcb0 2024-03-08T06:01:07.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.071+0000 7f10957fa640 1 -- 172.21.15.76:0/2171079631 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f109001b070 con 0x7f109815dcb0 2024-03-08T06:01:07.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.071+0000 7f10957fa640 1 --2- 172.21.15.76:0/2171079631 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f107c06b4d0 0x7f107c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.071+0000 7f10957fa640 1 -- 172.21.15.76:0/2171079631 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1090089a00 con 0x7f109815dcb0 2024-03-08T06:01:07.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.071+0000 7f109cf52640 1 --2- 172.21.15.76:0/2171079631 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f107c06b4d0 0x7f107c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.072+0000 7f109cf52640 1 --2- 172.21.15.76:0/2171079631 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f107c06b4d0 0x7f107c06d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f108c036000 tx=0x7f108c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:07.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.076+0000 7f10957fa640 1 -- 172.21.15.76:0/2171079631 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f10980cecc0 con 0x7f109815dcb0 2024-03-08T06:01:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.248+0000 7f109f1b6640 1 -- 172.21.15.76:0/2171079631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f109804f030 con 0x7f109815dcb0 2024-03-08T06:01:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.248+0000 7f10957fa640 1 -- 172.21.15.76:0/2171079631 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1090057f80 con 0x7f109815dcb0 2024-03-08T06:01:07.249 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:01:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.250+0000 7f109f1b6640 1 -- 172.21.15.76:0/2171079631 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f107c06b4d0 msgr2=0x7f107c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.250+0000 7f109f1b6640 1 --2- 172.21.15.76:0/2171079631 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f107c06b4d0 0x7f107c06d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f108c036000 tx=0x7f108c008030 comp rx=0 tx=0).stop 2024-03-08T06:01:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.250+0000 7f109f1b6640 1 -- 172.21.15.76:0/2171079631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f109815dcb0 msgr2=0x7f10981501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.250+0000 7f109f1b6640 1 --2- 172.21.15.76:0/2171079631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f109815dcb0 0x7f10981501c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f109000c700 tx=0x7f109000cbc0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.250+0000 7f109f1b6640 1 -- 172.21.15.76:0/2171079631 shutdown_connections 2024-03-08T06:01:07.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.250+0000 7f109f1b6640 1 --2- 172.21.15.76:0/2171079631 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f107c06b4d0 0x7f107c06d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.250+0000 7f109f1b6640 1 --2- 172.21.15.76:0/2171079631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f109815dcb0 0x7f10981501c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.250+0000 7f109f1b6640 1 --2- 172.21.15.76:0/2171079631 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f10980afdb0 0x7f109814fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.250+0000 7f109f1b6640 1 --2- 172.21.15.76:0/2171079631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10980af950 0x7f109809a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.250+0000 7f109f1b6640 1 -- 172.21.15.76:0/2171079631 >> 172.21.15.76:0/2171079631 conn(0x7f10980b2320 msgr2=0x7f10980b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:07.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.250+0000 7f109f1b6640 1 -- 172.21.15.76:0/2171079631 shutdown_connections 2024-03-08T06:01:07.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.250+0000 7f109f1b6640 1 -- 172.21.15.76:0/2171079631 wait complete. 2024-03-08T06:01:07.262 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T06:01:07.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.365+0000 7fbb6829a640 1 Processor -- start 2024-03-08T06:01:07.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.365+0000 7fbb6829a640 1 -- start start 2024-03-08T06:01:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.366+0000 7fbb6829a640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb6006d8a0 0x7fbb60162020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.366+0000 7fbb6829a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb60162560 0x7fbb6015dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.366+0000 7fbb6829a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb60161840 con 0x7fbb600ce5e0 2024-03-08T06:01:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.366+0000 7fbb6829a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb60161b90 con 0x7fbb60162560 2024-03-08T06:01:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.366+0000 7fbb6829a640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6015eb30 con 0x7fbb6006d8a0 2024-03-08T06:01:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.366+0000 7fbb65835640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb6006d8a0 0x7fbb60162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.366+0000 7fbb66837640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb60162560 0x7fbb6015dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.366+0000 7fbb65835640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb6006d8a0 0x7fbb60162020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:37720/0 (socket says 172.21.15.76:37720) 2024-03-08T06:01:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.366+0000 7fbb65835640 1 -- 172.21.15.76:0/2665019604 learned_addr learned my addr 172.21.15.76:0/2665019604 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.366+0000 7fbb66837640 1 -- 172.21.15.76:0/2665019604 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb6006d8a0 msgr2=0x7fbb60162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.366+0000 7fbb66837640 1 --2- 172.21.15.76:0/2665019604 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb6006d8a0 0x7fbb60162020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.366+0000 7fbb66837640 1 -- 172.21.15.76:0/2665019604 >> v1:172.21.15.81:6789/0 conn(0x7fbb600ce5e0 legacy=0x7fbb600ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.366+0000 7fbb66837640 1 -- 172.21.15.76:0/2665019604 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6015ee10 con 0x7fbb60162560 2024-03-08T06:01:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.366+0000 7fbb65034640 1 -- 172.21.15.76:0/2665019604 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009946803 0 0) 0x7fbb60161840 con 0x7fbb600ce5e0 2024-03-08T06:01:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.366+0000 7fbb65835640 1 --2- 172.21.15.76:0/2665019604 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb6006d8a0 0x7fbb60162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:07.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.367+0000 7fbb66837640 1 --2- 172.21.15.76:0/2665019604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb60162560 0x7fbb6015dfb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbb580097a0 tx=0x7fbb5802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c18f3b93bee70a2b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:07.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.367+0000 7fbb65034640 1 -- 172.21.15.76:0/2665019604 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58034070 con 0x7fbb60162560 2024-03-08T06:01:07.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.367+0000 7fbb65034640 1 -- 172.21.15.76:0/2665019604 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb58030610 con 0x7fbb60162560 2024-03-08T06:01:07.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.367+0000 7fbb65034640 1 -- 172.21.15.76:0/2665019604 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb580309e0 con 0x7fbb60162560 2024-03-08T06:01:07.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.368+0000 7fbb6829a640 1 -- 172.21.15.76:0/2665019604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb60162560 msgr2=0x7fbb6015dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.368+0000 7fbb6829a640 1 --2- 172.21.15.76:0/2665019604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb60162560 0x7fbb6015dfb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbb580097a0 tx=0x7fbb5802b960 comp rx=0 tx=0).stop 2024-03-08T06:01:07.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.368+0000 7fbb6829a640 1 -- 172.21.15.76:0/2665019604 shutdown_connections 2024-03-08T06:01:07.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.368+0000 7fbb6829a640 1 --2- 172.21.15.76:0/2665019604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb60162560 0x7fbb6015dfb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.368+0000 7fbb6829a640 1 --2- 172.21.15.76:0/2665019604 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb6006d8a0 0x7fbb60162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.368+0000 7fbb6829a640 1 -- 172.21.15.76:0/2665019604 >> 172.21.15.76:0/2665019604 conn(0x7fbb6006ae60 msgr2=0x7fbb6006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:07.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.368+0000 7fbb6829a640 1 -- 172.21.15.76:0/2665019604 shutdown_connections 2024-03-08T06:01:07.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.368+0000 7fbb6829a640 1 -- 172.21.15.76:0/2665019604 wait complete. 2024-03-08T06:01:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.369+0000 7fbb6829a640 1 Processor -- start 2024-03-08T06:01:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.369+0000 7fbb6829a640 1 -- start start 2024-03-08T06:01:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb6829a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb6006d8a0 0x7fbb601f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb6829a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb600ce5e0 0x7fbb600b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb6829a640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb60162560 0x7fbb60155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb6829a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6015e670 con 0x7fbb600ce5e0 2024-03-08T06:01:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb6829a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6015e4f0 con 0x7fbb6006d8a0 2024-03-08T06:01:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb6829a640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6015e7f0 con 0x7fbb60162560 2024-03-08T06:01:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb65835640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb600ce5e0 0x7fbb600b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb66036640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb6006d8a0 0x7fbb601f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb65835640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb600ce5e0 0x7fbb600b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51870/0 (socket says 172.21.15.76:51870) 2024-03-08T06:01:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb65835640 1 -- 172.21.15.76:0/2630326703 learned_addr learned my addr 172.21.15.76:0/2630326703 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb66837640 1 --2- 172.21.15.76:0/2630326703 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb60162560 0x7fbb60155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb65835640 1 -- 172.21.15.76:0/2630326703 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb60162560 msgr2=0x7fbb60155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb65835640 1 --2- 172.21.15.76:0/2630326703 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb60162560 0x7fbb60155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb65835640 1 -- 172.21.15.76:0/2630326703 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb6006d8a0 msgr2=0x7fbb601f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb65835640 1 --2- 172.21.15.76:0/2630326703 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb6006d8a0 0x7fbb601f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb65835640 1 -- 172.21.15.76:0/2630326703 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb60156090 con 0x7fbb600ce5e0 2024-03-08T06:01:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb65835640 1 --2- 172.21.15.76:0/2630326703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb600ce5e0 0x7fbb600b4550 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000e9e0 tx=0x7fbb5000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb56ffd640 1 -- 172.21.15.76:0/2630326703 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50017070 con 0x7fbb600ce5e0 2024-03-08T06:01:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.370+0000 7fbb56ffd640 1 -- 172.21.15.76:0/2630326703 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5000c9d0 con 0x7fbb600ce5e0 2024-03-08T06:01:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.371+0000 7fbb6829a640 1 -- 172.21.15.76:0/2630326703 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb601f7e30 con 0x7fbb600ce5e0 2024-03-08T06:01:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.371+0000 7fbb56ffd640 1 -- 172.21.15.76:0/2630326703 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5000cd80 con 0x7fbb600ce5e0 2024-03-08T06:01:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.371+0000 7fbb6829a640 1 -- 172.21.15.76:0/2630326703 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb601f83f0 con 0x7fbb600ce5e0 2024-03-08T06:01:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.371+0000 7fbb6829a640 1 -- 172.21.15.76:0/2630326703 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6004f030 con 0x7fbb600ce5e0 2024-03-08T06:01:07.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.372+0000 7fbb56ffd640 1 -- 172.21.15.76:0/2630326703 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb5001a070 con 0x7fbb600ce5e0 2024-03-08T06:01:07.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.372+0000 7fbb56ffd640 1 --2- 172.21.15.76:0/2630326703 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbb4406b4d0 0x7fbb4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.372+0000 7fbb66036640 1 --2- 172.21.15.76:0/2630326703 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbb4406b4d0 0x7fbb4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.372+0000 7fbb56ffd640 1 -- 172.21.15.76:0/2630326703 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbb5008a2b0 con 0x7fbb600ce5e0 2024-03-08T06:01:07.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.373+0000 7fbb66036640 1 --2- 172.21.15.76:0/2630326703 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbb4406b4d0 0x7fbb4406d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0012e0 tx=0x7fbb5c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:07.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.377+0000 7fbb56ffd640 1 -- 172.21.15.76:0/2630326703 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb60156090 con 0x7fbb600ce5e0 2024-03-08T06:01:07.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.546+0000 7fbb6829a640 1 -- 172.21.15.76:0/2630326703 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbb600b4330 con 0x7fbb600ce5e0 2024-03-08T06:01:07.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.546+0000 7fbb56ffd640 1 -- 172.21.15.76:0/2630326703 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbb50058830 con 0x7fbb600ce5e0 2024-03-08T06:01:07.547 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:07.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.548+0000 7fbb6829a640 1 -- 172.21.15.76:0/2630326703 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbb4406b4d0 msgr2=0x7fbb4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.548+0000 7fbb6829a640 1 --2- 172.21.15.76:0/2630326703 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbb4406b4d0 0x7fbb4406d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0012e0 tx=0x7fbb5c009040 comp rx=0 tx=0).stop 2024-03-08T06:01:07.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.548+0000 7fbb6829a640 1 -- 172.21.15.76:0/2630326703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb600ce5e0 msgr2=0x7fbb600b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.548+0000 7fbb6829a640 1 --2- 172.21.15.76:0/2630326703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb600ce5e0 0x7fbb600b4550 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000e9e0 tx=0x7fbb5000eea0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.548+0000 7fbb6829a640 1 -- 172.21.15.76:0/2630326703 shutdown_connections 2024-03-08T06:01:07.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.548+0000 7fbb6829a640 1 --2- 172.21.15.76:0/2630326703 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbb4406b4d0 0x7fbb4406d990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.548+0000 7fbb6829a640 1 --2- 172.21.15.76:0/2630326703 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb60162560 0x7fbb60155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.548+0000 7fbb6829a640 1 --2- 172.21.15.76:0/2630326703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb600ce5e0 0x7fbb600b4550 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.548+0000 7fbb6829a640 1 --2- 172.21.15.76:0/2630326703 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb6006d8a0 0x7fbb601f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.548+0000 7fbb6829a640 1 -- 172.21.15.76:0/2630326703 >> 172.21.15.76:0/2630326703 conn(0x7fbb6006ae60 msgr2=0x7fbb6015fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:07.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.548+0000 7fbb6829a640 1 -- 172.21.15.76:0/2630326703 shutdown_connections 2024-03-08T06:01:07.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.548+0000 7fbb6829a640 1 -- 172.21.15.76:0/2630326703 wait complete. 2024-03-08T06:01:07.560 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:01:07.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.659+0000 7f377fa76640 1 Processor -- start 2024-03-08T06:01:07.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.659+0000 7f377fa76640 1 -- start start 2024-03-08T06:01:07.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.660+0000 7f377fa76640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f37780b0ef0 0x7f3778168cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.660+0000 7f377fa76640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377815e690 con 0x7f377815de00 2024-03-08T06:01:07.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.660+0000 7f377fa76640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377815e970 con 0x7f377815e1e0 2024-03-08T06:01:07.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.660+0000 7f377fa76640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37780ce800 con 0x7f37780b0ef0 2024-03-08T06:01:07.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.660+0000 7f377d812640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f377815de00 0x7f37780b1a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57690/0 (socket says 172.21.15.76:57690) 2024-03-08T06:01:07.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.660+0000 7f377d812640 1 -- 172.21.15.76:0/246807934 learned_addr learned my addr 172.21.15.76:0/246807934 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:07.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.661+0000 7f377e013640 1 --2- 172.21.15.76:0/246807934 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f37780b0ef0 0x7f3778168cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.661+0000 7f377c810640 1 -- 172.21.15.76:0/246807934 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317288530 0 0) 0x7f377815e690 con 0x7f377815de00 2024-03-08T06:01:07.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.661+0000 7f377c810640 1 -- 172.21.15.76:0/246807934 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3764003520 con 0x7f377815de00 2024-03-08T06:01:07.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.661+0000 7f377c810640 1 -- 172.21.15.76:0/246807934 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082096192 0 0) 0x7f377815e970 con 0x7f377815e1e0 2024-03-08T06:01:07.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.661+0000 7f377c810640 1 -- 172.21.15.76:0/246807934 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f377815e690 con 0x7f377815e1e0 2024-03-08T06:01:07.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.661+0000 7f377c810640 1 -- 172.21.15.76:0/246807934 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1756425935 0 0) 0x7f3764003520 con 0x7f377815de00 2024-03-08T06:01:07.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.661+0000 7f377e013640 1 -- 172.21.15.76:0/246807934 >> v1:172.21.15.81:6789/0 conn(0x7f377815e1e0 legacy=0x7f37780cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.661+0000 7f377e013640 1 -- 172.21.15.76:0/246807934 >> v1:172.21.15.76:6789/0 conn(0x7f377815de00 legacy=0x7f37780b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.661+0000 7f377e013640 1 -- 172.21.15.76:0/246807934 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377815ec50 con 0x7f37780b0ef0 2024-03-08T06:01:07.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.662+0000 7f377e013640 1 --2- 172.21.15.76:0/246807934 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f37780b0ef0 0x7f3778168cd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f37700097d0 tx=0x7f377002b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba63ad5a6288380a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:07.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.662+0000 7f377c810640 1 -- 172.21.15.76:0/246807934 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3770037070 con 0x7f37780b0ef0 2024-03-08T06:01:07.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.662+0000 7f377c810640 1 -- 172.21.15.76:0/246807934 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3770030510 con 0x7f37780b0ef0 2024-03-08T06:01:07.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.662+0000 7f377c810640 1 -- 172.21.15.76:0/246807934 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774001a10 con 0x7f37780b0ef0 2024-03-08T06:01:07.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.662+0000 7f377fa76640 1 -- 172.21.15.76:0/246807934 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f37780b0ef0 msgr2=0x7f3778168cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.662+0000 7f377fa76640 1 --2- 172.21.15.76:0/246807934 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f37780b0ef0 0x7f3778168cd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f37700097d0 tx=0x7f377002b800 comp rx=0 tx=0).stop 2024-03-08T06:01:07.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.662+0000 7f377fa76640 1 -- 172.21.15.76:0/246807934 shutdown_connections 2024-03-08T06:01:07.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.662+0000 7f377fa76640 1 --2- 172.21.15.76:0/246807934 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f37780b0ef0 0x7f3778168cd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.662+0000 7f377fa76640 1 -- 172.21.15.76:0/246807934 >> 172.21.15.76:0/246807934 conn(0x7f3778099a60 msgr2=0x7f37780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:07.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.662+0000 7f377fa76640 1 -- 172.21.15.76:0/246807934 shutdown_connections 2024-03-08T06:01:07.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.662+0000 7f377fa76640 1 -- 172.21.15.76:0/246807934 wait complete. 2024-03-08T06:01:07.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.664+0000 7f377fa76640 1 Processor -- start 2024-03-08T06:01:07.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.664+0000 7f377fa76640 1 -- start start 2024-03-08T06:01:07.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.664+0000 7f377fa76640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37780b0ef0 0x7f37780b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.664+0000 7f377fa76640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f377815de00 0x7f37780c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.664+0000 7f377fa76640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f377815e1e0 0x7f3778157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.664+0000 7f377fa76640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37780b2140 con 0x7f37780b0ef0 2024-03-08T06:01:07.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.664+0000 7f377fa76640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377815f210 con 0x7f377815de00 2024-03-08T06:01:07.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.664+0000 7f377fa76640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37780c88f0 con 0x7f377815e1e0 2024-03-08T06:01:07.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.664+0000 7f377d812640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37780b0ef0 0x7f37780b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.664+0000 7f377d812640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37780b0ef0 0x7f37780b6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51884/0 (socket says 172.21.15.76:51884) 2024-03-08T06:01:07.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.664+0000 7f377d812640 1 -- 172.21.15.76:0/4250191957 learned_addr learned my addr 172.21.15.76:0/4250191957 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:07.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.664+0000 7f377d011640 1 --2- 172.21.15.76:0/4250191957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f377815de00 0x7f37780c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.665+0000 7f377e013640 1 --2- 172.21.15.76:0/4250191957 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f377815e1e0 0x7f3778157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.665+0000 7f377d812640 1 -- 172.21.15.76:0/4250191957 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f377815e1e0 msgr2=0x7f3778157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.665+0000 7f377d812640 1 --2- 172.21.15.76:0/4250191957 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f377815e1e0 0x7f3778157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.665+0000 7f377d812640 1 -- 172.21.15.76:0/4250191957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f377815de00 msgr2=0x7f37780c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.665+0000 7f377d812640 1 --2- 172.21.15.76:0/4250191957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f377815de00 0x7f37780c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.665+0000 7f377d812640 1 -- 172.21.15.76:0/4250191957 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37780c8b30 con 0x7f37780b0ef0 2024-03-08T06:01:07.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.665+0000 7f377d011640 1 --2- 172.21.15.76:0/4250191957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f377815de00 0x7f37780c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:07.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.665+0000 7f377e013640 1 --2- 172.21.15.76:0/4250191957 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f377815e1e0 0x7f3778157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:07.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.665+0000 7f377d812640 1 --2- 172.21.15.76:0/4250191957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37780b0ef0 0x7f37780b6220 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f377400cac0 tx=0x7f377400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:07.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.665+0000 7f376e7fc640 1 -- 172.21.15.76:0/4250191957 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774018070 con 0x7f37780b0ef0 2024-03-08T06:01:07.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.665+0000 7f376e7fc640 1 -- 172.21.15.76:0/4250191957 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377400a6d0 con 0x7f37780b0ef0 2024-03-08T06:01:07.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.665+0000 7f377fa76640 1 -- 172.21.15.76:0/4250191957 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37780c8e10 con 0x7f37780b0ef0 2024-03-08T06:01:07.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.665+0000 7f376e7fc640 1 -- 172.21.15.76:0/4250191957 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377400ab00 con 0x7f37780b0ef0 2024-03-08T06:01:07.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.665+0000 7f377fa76640 1 -- 172.21.15.76:0/4250191957 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37780c93d0 con 0x7f37780b0ef0 2024-03-08T06:01:07.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.666+0000 7f377fa76640 1 -- 172.21.15.76:0/4250191957 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377809a020 con 0x7f37780b0ef0 2024-03-08T06:01:07.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.666+0000 7f376e7fc640 1 -- 172.21.15.76:0/4250191957 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f377401b070 con 0x7f37780b0ef0 2024-03-08T06:01:07.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.667+0000 7f376e7fc640 1 --2- 172.21.15.76:0/4250191957 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f375c06b500 0x7f375c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.667+0000 7f377d011640 1 --2- 172.21.15.76:0/4250191957 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f375c06b500 0x7f375c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.667+0000 7f376e7fc640 1 -- 172.21.15.76:0/4250191957 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3774089d40 con 0x7f37780b0ef0 2024-03-08T06:01:07.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.667+0000 7f377d011640 1 --2- 172.21.15.76:0/4250191957 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f375c06b500 0x7f375c06d9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3778158ea0 tx=0x7f3768009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:07.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.671+0000 7f376e7fc640 1 -- 172.21.15.76:0/4250191957 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f377809a020 con 0x7f37780b0ef0 2024-03-08T06:01:07.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.877+0000 7f377fa76640 1 -- 172.21.15.76:0/4250191957 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f377804f030 con 0x7f37780b0ef0 2024-03-08T06:01:07.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.878+0000 7f376e7fc640 1 -- 172.21.15.76:0/4250191957 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f3774058910 con 0x7f37780b0ef0 2024-03-08T06:01:07.878 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:07.878 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4249563566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4249563566},{"type":"v1","addr":"172.21.15.76:6835","nonce":4249563566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1514641617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1514641617},{"type":"v1","addr":"172.21.15.76:6837","nonce":1514641617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6834/1332442802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1332442802},{"type":"v1","addr":"172.21.15.81:6834","nonce":1332442802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:00:35.504231+0000","modified":"2024-03-08T06:00:57.594023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6835/2357927814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":2357927814},{"type":"v1","addr":"172.21.15.81:6835","nonce":2357927814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T06:01:07.878 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-03-08T06:01:07.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.879+0000 7f377fa76640 1 -- 172.21.15.76:0/4250191957 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f375c06b500 msgr2=0x7f375c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.879+0000 7f377fa76640 1 --2- 172.21.15.76:0/4250191957 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f375c06b500 0x7f375c06d9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3778158ea0 tx=0x7f3768009040 comp rx=0 tx=0).stop 2024-03-08T06:01:07.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.879+0000 7f377fa76640 1 -- 172.21.15.76:0/4250191957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37780b0ef0 msgr2=0x7f37780b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.879+0000 7f377fa76640 1 --2- 172.21.15.76:0/4250191957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37780b0ef0 0x7f37780b6220 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f377400cac0 tx=0x7f377400cf80 comp rx=0 tx=0).stop 2024-03-08T06:01:07.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.880+0000 7f377fa76640 1 -- 172.21.15.76:0/4250191957 shutdown_connections 2024-03-08T06:01:07.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.880+0000 7f377fa76640 1 --2- 172.21.15.76:0/4250191957 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f375c06b500 0x7f375c06d9c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.880+0000 7f377fa76640 1 --2- 172.21.15.76:0/4250191957 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f377815e1e0 0x7f3778157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.880+0000 7f377fa76640 1 --2- 172.21.15.76:0/4250191957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f377815de00 0x7f37780c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.880+0000 7f377fa76640 1 --2- 172.21.15.76:0/4250191957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37780b0ef0 0x7f37780b6220 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.880+0000 7f377fa76640 1 -- 172.21.15.76:0/4250191957 >> 172.21.15.76:0/4250191957 conn(0x7f3778099a60 msgr2=0x7f37780cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:07.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.880+0000 7f377fa76640 1 -- 172.21.15.76:0/4250191957 shutdown_connections 2024-03-08T06:01:07.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.880+0000 7f377fa76640 1 -- 172.21.15.76:0/4250191957 wait complete. 2024-03-08T06:01:07.891 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:07.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.991+0000 7f74f616c640 1 Processor -- start 2024-03-08T06:01:07.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.991+0000 7f74f616c640 1 -- start start 2024-03-08T06:01:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.992+0000 7f74f616c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f74f0157c70 0x7f74f015a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.992+0000 7f74f616c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74f006eb70 0x7f74f006efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.992+0000 7f74f616c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f0158050 con 0x7f74f015abb0 2024-03-08T06:01:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.992+0000 7f74f616c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f015b160 con 0x7f74f0157c70 2024-03-08T06:01:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.992+0000 7f74f616c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f015b440 con 0x7f74f006eb70 2024-03-08T06:01:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.992+0000 7f74ef7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f74f0157c70 0x7f74f015a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.992+0000 7f74ef7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f74f0157c70 0x7f74f015a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51898/0 (socket says 172.21.15.76:51898) 2024-03-08T06:01:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.992+0000 7f74ef7fe640 1 -- 172.21.15.76:0/1694705853 learned_addr learned my addr 172.21.15.76:0/1694705853 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.992+0000 7f74effff640 1 --2- 172.21.15.76:0/1694705853 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74f006eb70 0x7f74f006efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.992+0000 7f74ef7fe640 1 -- 172.21.15.76:0/1694705853 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74f006eb70 msgr2=0x7f74f006efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.992+0000 7f74ef7fe640 1 --2- 172.21.15.76:0/1694705853 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74f006eb70 0x7f74f006efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.992+0000 7f74ef7fe640 1 -- 172.21.15.76:0/1694705853 >> v1:172.21.15.81:6789/0 conn(0x7f74f015abb0 legacy=0x7f74f00cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.992+0000 7f74ef7fe640 1 -- 172.21.15.76:0/1694705853 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f006e370 con 0x7f74f0157c70 2024-03-08T06:01:07.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.993+0000 7f74ef7fe640 1 --2- 172.21.15.76:0/1694705853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f0157c70 0x7f74f015a670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009870 tx=0x7f74e002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ec2036eabf361a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:07.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.993+0000 7f74ee7fc640 1 -- 172.21.15.76:0/1694705853 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0034070 con 0x7f74f0157c70 2024-03-08T06:01:07.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.993+0000 7f74ee7fc640 1 -- 172.21.15.76:0/1694705853 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e0030840 con 0x7f74f0157c70 2024-03-08T06:01:07.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.993+0000 7f74ee7fc640 1 -- 172.21.15.76:0/1694705853 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0030c10 con 0x7f74f0157c70 2024-03-08T06:01:07.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.993+0000 7f74f616c640 1 -- 172.21.15.76:0/1694705853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f0157c70 msgr2=0x7f74f015a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.993+0000 7f74f616c640 1 --2- 172.21.15.76:0/1694705853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f0157c70 0x7f74f015a670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009870 tx=0x7f74e002b980 comp rx=0 tx=0).stop 2024-03-08T06:01:07.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.994+0000 7f74f616c640 1 -- 172.21.15.76:0/1694705853 shutdown_connections 2024-03-08T06:01:07.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.994+0000 7f74f616c640 1 --2- 172.21.15.76:0/1694705853 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74f006eb70 0x7f74f006efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.994+0000 7f74f616c640 1 --2- 172.21.15.76:0/1694705853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f0157c70 0x7f74f015a670 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.994+0000 7f74f616c640 1 -- 172.21.15.76:0/1694705853 >> 172.21.15.76:0/1694705853 conn(0x7f74f00cf740 msgr2=0x7f74f00cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:07.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.994+0000 7f74f616c640 1 -- 172.21.15.76:0/1694705853 shutdown_connections 2024-03-08T06:01:07.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.994+0000 7f74f616c640 1 -- 172.21.15.76:0/1694705853 wait complete. 2024-03-08T06:01:07.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.995+0000 7f74f616c640 1 Processor -- start 2024-03-08T06:01:07.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.995+0000 7f74f616c640 1 -- start start 2024-03-08T06:01:07.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74f616c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74f006eb70 0x7f74f0146260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74f616c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f0157c70 0x7f74f00afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74f616c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74f015abb0 0x7f74f00c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:07.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74f616c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00cacf0 con 0x7f74f0157c70 2024-03-08T06:01:07.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74f616c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f006e020 con 0x7f74f015abb0 2024-03-08T06:01:07.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74f616c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00cda20 con 0x7f74f006eb70 2024-03-08T06:01:07.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74eeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f0157c70 0x7f74f00afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74eeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f0157c70 0x7f74f00afaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51912/0 (socket says 172.21.15.76:51912) 2024-03-08T06:01:07.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74eeffd640 1 -- 172.21.15.76:0/2579925402 learned_addr learned my addr 172.21.15.76:0/2579925402 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:07.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74ef7fe640 1 --2- 172.21.15.76:0/2579925402 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74f006eb70 0x7f74f0146260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74effff640 1 --2- 172.21.15.76:0/2579925402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74f015abb0 0x7f74f00c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:07.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74eeffd640 1 -- 172.21.15.76:0/2579925402 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74f006eb70 msgr2=0x7f74f0146260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74eeffd640 1 --2- 172.21.15.76:0/2579925402 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74f006eb70 0x7f74f0146260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74eeffd640 1 -- 172.21.15.76:0/2579925402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74f015abb0 msgr2=0x7f74f00c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:07.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74eeffd640 1 --2- 172.21.15.76:0/2579925402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74f015abb0 0x7f74f00c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:07.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74eeffd640 1 -- 172.21.15.76:0/2579925402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f014d9f0 con 0x7f74f0157c70 2024-03-08T06:01:07.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74eeffd640 1 --2- 172.21.15.76:0/2579925402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f0157c70 0x7f74f00afaf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f74e400c3c0 tx=0x7f74e400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:07.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74d7fff640 1 -- 172.21.15.76:0/2579925402 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e40052d0 con 0x7f74f0157c70 2024-03-08T06:01:07.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74d7fff640 1 -- 172.21.15.76:0/2579925402 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e4005490 con 0x7f74f0157c70 2024-03-08T06:01:07.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74d7fff640 1 -- 172.21.15.76:0/2579925402 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4007260 con 0x7f74f0157c70 2024-03-08T06:01:08.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74f616c640 1 -- 172.21.15.76:0/2579925402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f00b0440 con 0x7f74f0157c70 2024-03-08T06:01:08.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.996+0000 7f74f616c640 1 -- 172.21.15.76:0/2579925402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f00b0a00 con 0x7f74f0157c70 2024-03-08T06:01:08.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.997+0000 7f74f616c640 1 -- 172.21.15.76:0/2579925402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f00c9d40 con 0x7f74f0157c70 2024-03-08T06:01:08.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.998+0000 7f74d7fff640 1 -- 172.21.15.76:0/2579925402 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f74e4007400 con 0x7f74f0157c70 2024-03-08T06:01:08.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.998+0000 7f74d7fff640 1 --2- 172.21.15.76:0/2579925402 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f74dc06b500 0x7f74dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:08.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.998+0000 7f74ef7fe640 1 --2- 172.21.15.76:0/2579925402 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f74dc06b500 0x7f74dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:08.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.998+0000 7f74d7fff640 1 -- 172.21.15.76:0/2579925402 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f74e40898e0 con 0x7f74f0157c70 2024-03-08T06:01:08.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:07.999+0000 7f74ef7fe640 1 --2- 172.21.15.76:0/2579925402 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f74dc06b500 0x7f74dc06d9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f74e0036000 tx=0x7f74e0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:08.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.003+0000 7f74d7fff640 1 -- 172.21.15.76:0/2579925402 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f74f00c9d40 con 0x7f74f0157c70 2024-03-08T06:01:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.174+0000 7f74f616c640 1 -- 172.21.15.76:0/2579925402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74f00aea80 con 0x7f74f0157c70 2024-03-08T06:01:08.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.175+0000 7f74d7fff640 1 -- 172.21.15.76:0/2579925402 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f74f00aea80 con 0x7f74f0157c70 2024-03-08T06:01:08.176 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:08.176 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":26,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:00:42.202011+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.177+0000 7f74f616c640 1 -- 172.21.15.76:0/2579925402 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f74dc06b500 msgr2=0x7f74dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.177+0000 7f74f616c640 1 --2- 172.21.15.76:0/2579925402 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f74dc06b500 0x7f74dc06d9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f74e0036000 tx=0x7f74e0008030 comp rx=0 tx=0).stop 2024-03-08T06:01:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.177+0000 7f74f616c640 1 -- 172.21.15.76:0/2579925402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f0157c70 msgr2=0x7f74f00afaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.177+0000 7f74f616c640 1 --2- 172.21.15.76:0/2579925402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f0157c70 0x7f74f00afaf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f74e400c3c0 tx=0x7f74e400c880 comp rx=0 tx=0).stop 2024-03-08T06:01:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.177+0000 7f74f616c640 1 -- 172.21.15.76:0/2579925402 shutdown_connections 2024-03-08T06:01:08.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.177+0000 7f74f616c640 1 --2- 172.21.15.76:0/2579925402 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f74dc06b500 0x7f74dc06d9c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:08.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.177+0000 7f74f616c640 1 --2- 172.21.15.76:0/2579925402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74f015abb0 0x7f74f00c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:08.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.177+0000 7f74f616c640 1 --2- 172.21.15.76:0/2579925402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f0157c70 0x7f74f00afaf0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:08.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.177+0000 7f74f616c640 1 --2- 172.21.15.76:0/2579925402 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74f006eb70 0x7f74f0146260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:08.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.177+0000 7f74f616c640 1 -- 172.21.15.76:0/2579925402 >> 172.21.15.76:0/2579925402 conn(0x7f74f00cf740 msgr2=0x7f74f006efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:08.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.178+0000 7f74f616c640 1 -- 172.21.15.76:0/2579925402 shutdown_connections 2024-03-08T06:01:08.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.178+0000 7f74f616c640 1 -- 172.21.15.76:0/2579925402 wait complete. 2024-03-08T06:01:08.190 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:01:08.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.291+0000 7f31bf3e9640 1 Processor -- start 2024-03-08T06:01:08.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.291+0000 7f31bf3e9640 1 -- start start 2024-03-08T06:01:08.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.292+0000 7f31bf3e9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f31b806c7a0 0x7f31b806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:08.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.292+0000 7f31bf3e9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b815e140 con 0x7f31b80d0250 2024-03-08T06:01:08.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.292+0000 7f31bf3e9640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b80b1f90 con 0x7f31b815dd60 2024-03-08T06:01:08.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.292+0000 7f31bf3e9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b80b2270 con 0x7f31b806c7a0 2024-03-08T06:01:08.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.292+0000 7f31bc984640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f31b806c7a0 0x7f31b806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:08.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.292+0000 7f31bd986640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f31b80d0250 0x7f31b80d0630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57692/0 (socket says 172.21.15.76:57692) 2024-03-08T06:01:08.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.292+0000 7f31bd986640 1 -- 172.21.15.76:0/2180691435 learned_addr learned my addr 172.21.15.76:0/2180691435 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:08.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.292+0000 7f31bc984640 1 -- 172.21.15.76:0/2180691435 >> v1:172.21.15.81:6789/0 conn(0x7f31b815dd60 legacy=0x7f31b80b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:08.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.292+0000 7f31bc984640 1 -- 172.21.15.76:0/2180691435 >> v1:172.21.15.76:6789/0 conn(0x7f31b80d0250 legacy=0x7f31b80d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:08.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.292+0000 7f31bc984640 1 -- 172.21.15.76:0/2180691435 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b80b2550 con 0x7f31b806c7a0 2024-03-08T06:01:08.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.293+0000 7f31bc984640 1 --2- 172.21.15.76:0/2180691435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b806c7a0 0x7f31b806cc30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f31a80097a0 tx=0x7f31a802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06fad1905de85fc server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:08.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.293+0000 7f31affff640 1 -- 172.21.15.76:0/2180691435 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8034070 con 0x7f31b806c7a0 2024-03-08T06:01:08.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.293+0000 7f31affff640 1 -- 172.21.15.76:0/2180691435 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a8030610 con 0x7f31b806c7a0 2024-03-08T06:01:08.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.293+0000 7f31affff640 1 -- 172.21.15.76:0/2180691435 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a80309e0 con 0x7f31b806c7a0 2024-03-08T06:01:08.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.293+0000 7f31bf3e9640 1 -- 172.21.15.76:0/2180691435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b806c7a0 msgr2=0x7f31b806cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:08.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.293+0000 7f31bf3e9640 1 --2- 172.21.15.76:0/2180691435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b806c7a0 0x7f31b806cc30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f31a80097a0 tx=0x7f31a802b960 comp rx=0 tx=0).stop 2024-03-08T06:01:08.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.294+0000 7f31bf3e9640 1 -- 172.21.15.76:0/2180691435 shutdown_connections 2024-03-08T06:01:08.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.294+0000 7f31bf3e9640 1 --2- 172.21.15.76:0/2180691435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b806c7a0 0x7f31b806cc30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:08.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.294+0000 7f31bf3e9640 1 -- 172.21.15.76:0/2180691435 >> 172.21.15.76:0/2180691435 conn(0x7f31b80d1950 msgr2=0x7f31b80d1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:08.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.294+0000 7f31bf3e9640 1 -- 172.21.15.76:0/2180691435 shutdown_connections 2024-03-08T06:01:08.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.294+0000 7f31bf3e9640 1 -- 172.21.15.76:0/2180691435 wait complete. 2024-03-08T06:01:08.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.295+0000 7f31bf3e9640 1 Processor -- start 2024-03-08T06:01:08.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.295+0000 7f31bf3e9640 1 -- start start 2024-03-08T06:01:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.295+0000 7f31bf3e9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b806c7a0 0x7f31b81f7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.295+0000 7f31bf3e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31b80d0250 0x7f31b80c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.295+0000 7f31bf3e9640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f31b815dd60 0x7f31b81f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.295+0000 7f31bf3e9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b806c1b0 con 0x7f31b806c7a0 2024-03-08T06:01:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.295+0000 7f31bf3e9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b80b2830 con 0x7f31b80d0250 2024-03-08T06:01:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.295+0000 7f31bf3e9640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b806c330 con 0x7f31b815dd60 2024-03-08T06:01:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.296+0000 7f31bd185640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b806c7a0 0x7f31b81f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.296+0000 7f31bd185640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b806c7a0 0x7f31b81f7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51924/0 (socket says 172.21.15.76:51924) 2024-03-08T06:01:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.296+0000 7f31bd185640 1 -- 172.21.15.76:0/2328534545 learned_addr learned my addr 172.21.15.76:0/2328534545 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.296+0000 7f31bc984640 1 --2- 172.21.15.76:0/2328534545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31b80d0250 0x7f31b80c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.296+0000 7f31bd986640 1 --2- 172.21.15.76:0/2328534545 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f31b815dd60 0x7f31b81f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.296+0000 7f31bd185640 1 -- 172.21.15.76:0/2328534545 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f31b815dd60 msgr2=0x7f31b81f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.296+0000 7f31bd185640 1 --2- 172.21.15.76:0/2328534545 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f31b815dd60 0x7f31b81f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.296+0000 7f31bd185640 1 -- 172.21.15.76:0/2328534545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31b80d0250 msgr2=0x7f31b80c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.296+0000 7f31bd185640 1 --2- 172.21.15.76:0/2328534545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31b80d0250 0x7f31b80c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.296+0000 7f31bd185640 1 -- 172.21.15.76:0/2328534545 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b81f7bf0 con 0x7f31b806c7a0 2024-03-08T06:01:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.296+0000 7f31bd185640 1 --2- 172.21.15.76:0/2328534545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b806c7a0 0x7f31b81f7560 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f31b400c3c0 tx=0x7f31b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.296+0000 7f31adffb640 1 -- 172.21.15.76:0/2328534545 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b40052d0 con 0x7f31b806c7a0 2024-03-08T06:01:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.296+0000 7f31adffb640 1 -- 172.21.15.76:0/2328534545 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b4005490 con 0x7f31b806c7a0 2024-03-08T06:01:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.296+0000 7f31bf3e9640 1 -- 172.21.15.76:0/2328534545 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b81f7ed0 con 0x7f31b806c7a0 2024-03-08T06:01:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.296+0000 7f31adffb640 1 -- 172.21.15.76:0/2328534545 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4007260 con 0x7f31b806c7a0 2024-03-08T06:01:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.296+0000 7f31bf3e9640 1 -- 172.21.15.76:0/2328534545 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b80c6bb0 con 0x7f31b806c7a0 2024-03-08T06:01:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.298+0000 7f31bf3e9640 1 -- 172.21.15.76:0/2328534545 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b81f83f0 con 0x7f31b806c7a0 2024-03-08T06:01:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.298+0000 7f31adffb640 1 -- 172.21.15.76:0/2328534545 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f31b4007400 con 0x7f31b806c7a0 2024-03-08T06:01:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.298+0000 7f31adffb640 1 --2- 172.21.15.76:0/2328534545 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f319c06b500 0x7f319c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.298+0000 7f31adffb640 1 -- 172.21.15.76:0/2328534545 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f31b40898d0 con 0x7f31b806c7a0 2024-03-08T06:01:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.298+0000 7f31bc984640 1 --2- 172.21.15.76:0/2328534545 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f319c06b500 0x7f319c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.299+0000 7f31bc984640 1 --2- 172.21.15.76:0/2328534545 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f319c06b500 0x7f319c06d9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f31a8036000 tx=0x7f31a803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:08.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.302+0000 7f31adffb640 1 -- 172.21.15.76:0/2328534545 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f31b81f83f0 con 0x7f31b806c7a0 2024-03-08T06:01:08.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.511+0000 7f31bf3e9640 1 -- 172.21.15.76:0/2328534545 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31b804f030 con 0x7f31b806c7a0 2024-03-08T06:01:08.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.511+0000 7f31adffb640 1 -- 172.21.15.76:0/2328534545 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f31b4057e50 con 0x7f31b806c7a0 2024-03-08T06:01:08.512 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:08.512 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4249563566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4249563566},{"type":"v1","addr":"172.21.15.76:6835","nonce":4249563566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1514641617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1514641617},{"type":"v1","addr":"172.21.15.76:6837","nonce":1514641617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6834/1332442802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1332442802},{"type":"v1","addr":"172.21.15.81:6834","nonce":1332442802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:00:35.504231+0000","modified":"2024-03-08T06:00:57.594023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6835/2357927814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":2357927814},{"type":"v1","addr":"172.21.15.81:6835","nonce":2357927814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T06:01:08.512 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-03-08T06:01:08.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.513+0000 7f31bf3e9640 1 -- 172.21.15.76:0/2328534545 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f319c06b500 msgr2=0x7f319c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:08.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.513+0000 7f31bf3e9640 1 --2- 172.21.15.76:0/2328534545 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f319c06b500 0x7f319c06d9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f31a8036000 tx=0x7f31a803b040 comp rx=0 tx=0).stop 2024-03-08T06:01:08.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.513+0000 7f31bf3e9640 1 -- 172.21.15.76:0/2328534545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b806c7a0 msgr2=0x7f31b81f7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:08.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.513+0000 7f31bf3e9640 1 --2- 172.21.15.76:0/2328534545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b806c7a0 0x7f31b81f7560 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f31b400c3c0 tx=0x7f31b400c880 comp rx=0 tx=0).stop 2024-03-08T06:01:08.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.513+0000 7f31bf3e9640 1 -- 172.21.15.76:0/2328534545 shutdown_connections 2024-03-08T06:01:08.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.513+0000 7f31bf3e9640 1 --2- 172.21.15.76:0/2328534545 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f319c06b500 0x7f319c06d9c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:08.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.513+0000 7f31bf3e9640 1 --2- 172.21.15.76:0/2328534545 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f31b815dd60 0x7f31b81f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:08.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.513+0000 7f31bf3e9640 1 --2- 172.21.15.76:0/2328534545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31b80d0250 0x7f31b80c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:08.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.513+0000 7f31bf3e9640 1 --2- 172.21.15.76:0/2328534545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b806c7a0 0x7f31b81f7560 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:08.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.513+0000 7f31bf3e9640 1 -- 172.21.15.76:0/2328534545 >> 172.21.15.76:0/2328534545 conn(0x7f31b80d1950 msgr2=0x7f31b8099b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:08.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.513+0000 7f31bf3e9640 1 -- 172.21.15.76:0/2328534545 shutdown_connections 2024-03-08T06:01:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:08.513+0000 7f31bf3e9640 1 -- 172.21.15.76:0/2328534545 wait complete. 2024-03-08T06:01:08.926 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-08T06:00:35.504231+0000', 'modified': '2024-03-08T06:00:57.594023+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.81:6835/2357927814', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6833', 'nonce': 2357927814}, {'type': 'v1', 'addr': '172.21.15.81:6835', 'nonce': 2357927814}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-08T06:01:08.926 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-08T06:01:08.927 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-08T06:01:09.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.029+0000 7f65ae6ce640 1 Processor -- start 2024-03-08T06:01:09.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.029+0000 7f65ae6ce640 1 -- start start 2024-03-08T06:01:09.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.030+0000 7f65ae6ce640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65a806d8a0 0x7f65a8162020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.030+0000 7f65ae6ce640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a8161840 con 0x7f65a8162560 2024-03-08T06:01:09.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.030+0000 7f65ae6ce640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a815eb30 con 0x7f65a80ce5e0 2024-03-08T06:01:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.030+0000 7f65ae6ce640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8161b90 con 0x7f65a806d8a0 2024-03-08T06:01:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.030+0000 7f65acc6b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f65a8162560 0x7f65a815dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49724/0 (socket says 172.21.15.76:49724) 2024-03-08T06:01:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.030+0000 7f65acc6b640 1 -- 172.21.15.76:0/4066186544 learned_addr learned my addr 172.21.15.76:0/4066186544 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.030+0000 7f65a77fe640 1 --2- 172.21.15.76:0/4066186544 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65a806d8a0 0x7f65a8162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.030+0000 7f65a6ffd640 1 -- 172.21.15.76:0/4066186544 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1522123099 0 0) 0x7f65a8161840 con 0x7f65a8162560 2024-03-08T06:01:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.030+0000 7f65a6ffd640 1 -- 172.21.15.76:0/4066186544 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6590003520 con 0x7f65a8162560 2024-03-08T06:01:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.030+0000 7f65a6ffd640 1 -- 172.21.15.76:0/4066186544 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3094750299 0 0) 0x7f6590003520 con 0x7f65a8162560 2024-03-08T06:01:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.030+0000 7f65a6ffd640 1 -- 172.21.15.76:0/4066186544 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65a806d8a0 msgr2=0x7f65a8162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.030+0000 7f65a6ffd640 1 --2- 172.21.15.76:0/4066186544 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65a806d8a0 0x7f65a8162020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.030+0000 7f65a6ffd640 1 -- 172.21.15.76:0/4066186544 >> v1:172.21.15.81:6789/0 conn(0x7f65a80ce5e0 legacy=0x7f65a80ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.030+0000 7f65a6ffd640 1 -- 172.21.15.76:0/4066186544 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a815ee10 con 0x7f65a8162560 2024-03-08T06:01:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.030+0000 7f65a6ffd640 1 -- 172.21.15.76:0/4066186544 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f65a00019f0 con 0x7f65a8162560 2024-03-08T06:01:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.030+0000 7f65a6ffd640 1 -- 172.21.15.76:0/4066186544 >> v1:172.21.15.76:6789/0 conn(0x7f65a8162560 legacy=0x7f65a815dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.031+0000 7f65a6ffd640 1 --2- 172.21.15.76:0/4066186544 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65a806d8a0 0x7f6590003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.031+0000 7f65a6ffd640 1 --2- 172.21.15.76:0/4066186544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6590003fd0 0x7f6590004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.031+0000 7f65a6ffd640 1 --2- 172.21.15.76:0/4066186544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6590004d40 0x7f65900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.031+0000 7f65a6ffd640 1 -- 172.21.15.76:0/4066186544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8161b90 con 0x7f6590004d40 2024-03-08T06:01:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.031+0000 7f65a6ffd640 1 -- 172.21.15.76:0/4066186544 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659000bbb0 con 0x7f6590003fd0 2024-03-08T06:01:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.031+0000 7f65a6ffd640 1 -- 172.21.15.76:0/4066186544 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659000bdb0 con 0x7f65a806d8a0 2024-03-08T06:01:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.031+0000 7f65acc6b640 1 --2- 172.21.15.76:0/4066186544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6590004d40 0x7f65900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.031+0000 7f65a77fe640 1 --2- 172.21.15.76:0/4066186544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6590003fd0 0x7f6590004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.031+0000 7f65a7fff640 1 --2- 172.21.15.76:0/4066186544 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65a806d8a0 0x7f6590003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.031+0000 7f65acc6b640 1 -- 172.21.15.76:0/4066186544 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65a806d8a0 msgr2=0x7f6590003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.031+0000 7f65acc6b640 1 --2- 172.21.15.76:0/4066186544 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65a806d8a0 0x7f6590003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.031+0000 7f65acc6b640 1 -- 172.21.15.76:0/4066186544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6590003fd0 msgr2=0x7f6590004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.031+0000 7f65acc6b640 1 --2- 172.21.15.76:0/4066186544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6590003fd0 0x7f6590004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.031+0000 7f65acc6b640 1 -- 172.21.15.76:0/4066186544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6590003520 con 0x7f6590004d40 2024-03-08T06:01:09.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.032+0000 7f65acc6b640 1 --2- 172.21.15.76:0/4066186544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6590004d40 0x7f65900051c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f65a000a850 tx=0x7f6590005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d122edc155f0d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:09.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.032+0000 7f65a6ffd640 1 -- 172.21.15.76:0/4066186544 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a000a4a0 con 0x7f6590004d40 2024-03-08T06:01:09.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.032+0000 7f65a6ffd640 1 -- 172.21.15.76:0/4066186544 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a0034650 con 0x7f6590004d40 2024-03-08T06:01:09.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.032+0000 7f65a6ffd640 1 -- 172.21.15.76:0/4066186544 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0034a80 con 0x7f6590004d40 2024-03-08T06:01:09.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.032+0000 7f65ae6ce640 1 -- 172.21.15.76:0/4066186544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6590004d40 msgr2=0x7f65900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.032+0000 7f65ae6ce640 1 --2- 172.21.15.76:0/4066186544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6590004d40 0x7f65900051c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f65a000a850 tx=0x7f6590005830 comp rx=0 tx=0).stop 2024-03-08T06:01:09.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.032+0000 7f65acc6b640 1 -- 172.21.15.76:0/4066186544 reap_dead start 2024-03-08T06:01:09.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.032+0000 7f65ae6ce640 1 -- 172.21.15.76:0/4066186544 shutdown_connections 2024-03-08T06:01:09.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.032+0000 7f65ae6ce640 1 -- 172.21.15.76:0/4066186544 >> 172.21.15.76:0/4066186544 conn(0x7f65a806ae60 msgr2=0x7f65a806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:09.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.032+0000 7f65ae6ce640 1 -- 172.21.15.76:0/4066186544 shutdown_connections 2024-03-08T06:01:09.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.032+0000 7f65ae6ce640 1 -- 172.21.15.76:0/4066186544 wait complete. 2024-03-08T06:01:09.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.034+0000 7f65ae6ce640 1 Processor -- start 2024-03-08T06:01:09.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.034+0000 7f65ae6ce640 1 -- start start 2024-03-08T06:01:09.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.034+0000 7f65ae6ce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6590004d40 0x7f65a80b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.034+0000 7f65ae6ce640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65a8158d70 0x7f65a8158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.034+0000 7f65ae6ce640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65a80c55f0 0x7f65a80c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.034+0000 7f65ae6ce640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a815e840 con 0x7f6590004d40 2024-03-08T06:01:09.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.034+0000 7f65ae6ce640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a815e6c0 con 0x7f65a80c55f0 2024-03-08T06:01:09.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65ae6ce640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8162020 con 0x7f65a8158d70 2024-03-08T06:01:09.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65a7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6590004d40 0x7f65a80b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65a7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6590004d40 0x7f65a80b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55164/0 (socket says 172.21.15.76:55164) 2024-03-08T06:01:09.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65a7fff640 1 -- 172.21.15.76:0/800088217 learned_addr learned my addr 172.21.15.76:0/800088217 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:09.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65a7fff640 1 -- 172.21.15.76:0/800088217 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65a8158d70 msgr2=0x7f65a8158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65acc6b640 1 --2- 172.21.15.76:0/800088217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65a80c55f0 0x7f65a80c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65a77fe640 1 --2- 172.21.15.76:0/800088217 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65a8158d70 0x7f65a8158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65a7fff640 1 --2- 172.21.15.76:0/800088217 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65a8158d70 0x7f65a8158530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65a7fff640 1 -- 172.21.15.76:0/800088217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65a80c55f0 msgr2=0x7f65a80c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65a7fff640 1 --2- 172.21.15.76:0/800088217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65a80c55f0 0x7f65a80c5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65a7fff640 1 -- 172.21.15.76:0/800088217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a81e9440 con 0x7f6590004d40 2024-03-08T06:01:09.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65a7fff640 1 --2- 172.21.15.76:0/800088217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6590004d40 0x7f65a80b4f60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f659800f850 tx=0x7f659800fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:09.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65a4ff9640 1 -- 172.21.15.76:0/800088217 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659800fe40 con 0x7f6590004d40 2024-03-08T06:01:09.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65a4ff9640 1 -- 172.21.15.76:0/800088217 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6598018070 con 0x7f6590004d40 2024-03-08T06:01:09.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65a4ff9640 1 -- 172.21.15.76:0/800088217 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659800d6d0 con 0x7f6590004d40 2024-03-08T06:01:09.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65ae6ce640 1 -- 172.21.15.76:0/800088217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a80b56b0 con 0x7f6590004d40 2024-03-08T06:01:09.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.035+0000 7f65ae6ce640 1 -- 172.21.15.76:0/800088217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a80b5c70 con 0x7f6590004d40 2024-03-08T06:01:09.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.037+0000 7f65a4ff9640 1 -- 172.21.15.76:0/800088217 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f659800d870 con 0x7f6590004d40 2024-03-08T06:01:09.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.037+0000 7f65a4ff9640 1 --2- 172.21.15.76:0/800088217 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f659406b500 0x7f659406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.037+0000 7f65a4ff9640 1 -- 172.21.15.76:0/800088217 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f659808a310 con 0x7f6590004d40 2024-03-08T06:01:09.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.037+0000 7f65a77fe640 1 --2- 172.21.15.76:0/800088217 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f659406b500 0x7f659406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.037+0000 7f65a77fe640 1 --2- 172.21.15.76:0/800088217 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f659406b500 0x7f659406d9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f65a815f720 tx=0x7f659c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:09.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.038+0000 7f65b1383740 1 Processor -- start 2024-03-08T06:01:09.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.038+0000 7f65b1383740 1 -- start start 2024-03-08T06:01:09.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.038+0000 7f65b1383740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79ce870 0x5563b79cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.038+0000 7f65b1383740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79d28c0 0x5563b79d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65b1383740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79fe600 0x5563b79fea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65b1383740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563b79386b0 con 0x5563b79d28c0 2024-03-08T06:01:09.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65b1383740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563b78e1a60 con 0x5563b79ce870 2024-03-08T06:01:09.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65b1383740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563b7934750 con 0x5563b79fe600 2024-03-08T06:01:09.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65a77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79d28c0 0x5563b79d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65a77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79d28c0 0x5563b79d4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55180/0 (socket says 172.21.15.76:55180) 2024-03-08T06:01:09.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65a77fe640 1 -- 172.21.15.76:0/3405973738 learned_addr learned my addr 172.21.15.76:0/3405973738 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:09.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65a7fff640 1 --2- 172.21.15.76:0/3405973738 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79ce870 0x5563b79cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65acc6b640 1 --2- 172.21.15.76:0/3405973738 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79fe600 0x5563b79fea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65a77fe640 1 -- 172.21.15.76:0/3405973738 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79fe600 msgr2=0x5563b79fea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:09.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65a77fe640 1 --2- 172.21.15.76:0/3405973738 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79fe600 0x5563b79fea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65a77fe640 1 -- 172.21.15.76:0/3405973738 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79ce870 msgr2=0x5563b79cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:09.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65a77fe640 1 --2- 172.21.15.76:0/3405973738 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79ce870 0x5563b79cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65a77fe640 1 -- 172.21.15.76:0/3405973738 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563b78fff30 con 0x5563b79d28c0 2024-03-08T06:01:09.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65a7fff640 1 --2- 172.21.15.76:0/3405973738 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79ce870 0x5563b79cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:09.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65acc6b640 1 --2- 172.21.15.76:0/3405973738 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79fe600 0x5563b79fea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:09.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f65a77fe640 1 --2- 172.21.15.76:0/3405973738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79d28c0 0x5563b79d4360 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f659c013d80 tx=0x7f659c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c3e69101a092a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:09.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f658dffb640 1 -- 172.21.15.76:0/3405973738 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c004030 con 0x5563b79d28c0 2024-03-08T06:01:09.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f658dffb640 1 -- 172.21.15.76:0/3405973738 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c028070 con 0x5563b79d28c0 2024-03-08T06:01:09.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.039+0000 7f658dffb640 1 -- 172.21.15.76:0/3405973738 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c0043d0 con 0x5563b79d28c0 2024-03-08T06:01:09.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.040+0000 7f65b1383740 1 -- 172.21.15.76:0/3405973738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79d28c0 msgr2=0x5563b79d4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.040+0000 7f65b1383740 1 --2- 172.21.15.76:0/3405973738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79d28c0 0x5563b79d4360 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f659c013d80 tx=0x7f659c016000 comp rx=0 tx=0).stop 2024-03-08T06:01:09.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.040+0000 7f65b1383740 1 -- 172.21.15.76:0/3405973738 shutdown_connections 2024-03-08T06:01:09.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.040+0000 7f65b1383740 1 --2- 172.21.15.76:0/3405973738 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79fe600 0x5563b79fea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.040+0000 7f65b1383740 1 --2- 172.21.15.76:0/3405973738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79d28c0 0x5563b79d4360 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.040+0000 7f65b1383740 1 --2- 172.21.15.76:0/3405973738 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79ce870 0x5563b79cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.040+0000 7f65b1383740 1 -- 172.21.15.76:0/3405973738 >> 172.21.15.76:0/3405973738 conn(0x5563b79d08f0 msgr2=0x5563b79d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:09.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.040+0000 7f65b1383740 1 -- 172.21.15.76:0/3405973738 shutdown_connections 2024-03-08T06:01:09.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.040+0000 7f65b1383740 1 -- 172.21.15.76:0/3405973738 wait complete. 2024-03-08T06:01:09.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.040+0000 7f65b1383740 1 -- start start 2024-03-08T06:01:09.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.041+0000 7f65b1383740 1 Processor -- start 2024-03-08T06:01:09.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.041+0000 7f65b1383740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79fe600 0x5563b7a13d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.041+0000 7f65b1383740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79f9940 0x5563b7a01f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.041+0000 7f65b1383740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79f8d90 0x5563b79f9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.041+0000 7f65b1383740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563b79d2d50 con 0x5563b79fe600 2024-03-08T06:01:09.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.041+0000 7f65b1383740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563b78f4ea0 con 0x5563b79f8d90 2024-03-08T06:01:09.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.041+0000 7f65b1383740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563b79d3030 con 0x5563b79f9940 2024-03-08T06:01:09.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f65a7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79fe600 0x5563b7a13d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f65a7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79fe600 0x5563b7a13d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55194/0 (socket says 172.21.15.76:55194) 2024-03-08T06:01:09.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f65a7fff640 1 -- 172.21.15.76:0/2910338252 learned_addr learned my addr 172.21.15.76:0/2910338252 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:09.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f65b1383740 10 client.0 _ll_register_callbacks cb 0x5563b78f4870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:01:09.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f65acc6b640 1 --2- 172.21.15.76:0/2910338252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79f8d90 0x5563b79f9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f65a77fe640 1 --2- 172.21.15.76:0/2910338252 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79f9940 0x5563b7a01f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f65a7fff640 1 -- 172.21.15.76:0/2910338252 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79f9940 msgr2=0x5563b7a01f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:09.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f65a7fff640 1 --2- 172.21.15.76:0/2910338252 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79f9940 0x5563b7a01f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f65a7fff640 1 -- 172.21.15.76:0/2910338252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79f8d90 msgr2=0x5563b79f9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:09.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f65a7fff640 1 --2- 172.21.15.76:0/2910338252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79f8d90 0x5563b79f9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f65a7fff640 1 -- 172.21.15.76:0/2910338252 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563b78f5270 con 0x5563b79fe600 2024-03-08T06:01:09.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f65a77fe640 1 --2- 172.21.15.76:0/2910338252 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79f9940 0x5563b7a01f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:09.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f65a7fff640 1 --2- 172.21.15.76:0/2910338252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79fe600 0x5563b7a13d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f659800d520 tx=0x7f6598058e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869b9fa70bb0f0e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:09.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f6586ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-08T06:01:09.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f6586ffd640 1 -- 172.21.15.76:0/2910338252 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6598086060 con 0x5563b79fe600 2024-03-08T06:01:09.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f6586ffd640 1 -- 172.21.15.76:0/2910338252 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f659808a070 con 0x5563b79fe600 2024-03-08T06:01:09.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f65b1383740 1 -- 172.21.15.76:0/2910338252 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f659c028070 con 0x5563b79fe600 2024-03-08T06:01:09.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f6586ffd640 1 -- 172.21.15.76:0/2910338252 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6598075050 con 0x5563b79fe600 2024-03-08T06:01:09.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f65b1383740 10 client.4830 fetch_fsmap learned FSMap version 8 2024-03-08T06:01:09.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f65b1383740 1 -- 172.21.15.76:0/2910338252 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5563b764d2d0 con 0x5563b79fe600 2024-03-08T06:01:09.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f6586ffd640 1 -- 172.21.15.76:0/2910338252 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c028070 con 0x5563b79fe600 2024-03-08T06:01:09.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.042+0000 7f6586ffd640 1 -- 172.21.15.76:0/2910338252 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 8) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f6598070100 con 0x5563b79fe600 2024-03-08T06:01:09.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.043+0000 7f65b1383740 10 client.4830 fetch_fsmap finished waiting for FSMap version 8 2024-03-08T06:01:09.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.043+0000 7f65b1383740 10 client.4830 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-08T06:01:09.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.043+0000 7f65b1383740 20 client.4830 populate_metadata read hostname 'smithi076' 2024-03-08T06:01:09.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.043+0000 7f65b1383740 1 --2- 172.21.15.76:0/2910338252 >> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] conn(0x5563b79ffe30 0x5563b7a00210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.043+0000 7f65b1383740 4 client.4830 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T06:01:09.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.043+0000 7f65b1383740 1 -- 172.21.15.76:0/2910338252 --> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563b7945bf0 con 0x5563b79ffe30 2024-03-08T06:01:09.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.043+0000 7f65a77fe640 1 --2- 172.21.15.76:0/2910338252 >> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] conn(0x5563b79ffe30 0x5563b7a00210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.043+0000 7f65a77fe640 1 --2- 172.21.15.76:0/2910338252 >> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] conn(0x5563b79ffe30 0x5563b7a00210 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee27f76ecb2f1983 server_cookie=868c3fefa9445f49 in_seq=0 out_seq=0 2024-03-08T06:01:09.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.043+0000 7f6586ffd640 10 client.4830 ms_handle_connect on v2:172.21.15.81:6833/2357927814 2024-03-08T06:01:09.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.044+0000 7f6586ffd640 1 -- 172.21.15.76:0/2910338252 <== mds.0 v2:172.21.15.81:6833/2357927814 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5563b7945bf0 con 0x5563b79ffe30 2024-03-08T06:01:09.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.044+0000 7f6586ffd640 10 client.4830 handle_command_reply: tid=0 2024-03-08T06:01:09.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.044+0000 7f65b1383740 1 client.4830 shutdown 2024-03-08T06:01:09.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.044+0000 7f65b1383740 1 -- 172.21.15.76:0/2910338252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79fe600 msgr2=0x5563b7a13d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.044+0000 7f65b1383740 1 --2- 172.21.15.76:0/2910338252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79fe600 0x5563b7a13d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f659800d520 tx=0x7f6598058e50 comp rx=0 tx=0).stop 2024-03-08T06:01:09.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.044+0000 7f65b1383740 1 -- 172.21.15.76:0/2910338252 shutdown_connections 2024-03-08T06:01:09.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.044+0000 7f65b1383740 1 --2- 172.21.15.76:0/2910338252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79f8d90 0x5563b79f9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.044+0000 7f65b1383740 1 --2- 172.21.15.76:0/2910338252 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79f9940 0x5563b7a01f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.044+0000 7f65b1383740 1 --2- 172.21.15.76:0/2910338252 >> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] conn(0x5563b79ffe30 0x5563b7a00210 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.044+0000 7f65b1383740 1 --2- 172.21.15.76:0/2910338252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79fe600 0x5563b7a13d10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.044+0000 7f6586ffd640 0 client.4830 ms_handle_reset on v2:172.21.15.81:6833/2357927814 2024-03-08T06:01:09.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.044+0000 7f65b1383740 1 -- 172.21.15.76:0/2910338252 >> 172.21.15.76:0/2910338252 conn(0x5563b79ce870 msgr2=0x5563b79cd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:09.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.044+0000 7f65b1383740 1 -- 172.21.15.76:0/2910338252 shutdown_connections 2024-03-08T06:01:09.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.044+0000 7f65b1383740 1 -- 172.21.15.76:0/2910338252 wait complete. 2024-03-08T06:01:09.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.044+0000 7f65b1383740 20 client.4830 trim_cache size 0 max 16384 2024-03-08T06:01:09.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.060+0000 7f65b1383740 1 Processor -- start 2024-03-08T06:01:09.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.060+0000 7f65b1383740 1 -- start start 2024-03-08T06:01:09.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.060+0000 7f65b1383740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79f8d90 0x5563b79f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.060+0000 7f65b1383740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79f9940 0x5563b79c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.060+0000 7f65b1383740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79fe600 0x5563b79fe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f65b1383740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8201be0 con 0x5563b79fe600 2024-03-08T06:01:09.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f65b1383740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8200d70 con 0x5563b79f9940 2024-03-08T06:01:09.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f65b1383740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8200210 con 0x5563b79f8d90 2024-03-08T06:01:09.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f65acc6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79fe600 0x5563b79fe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f65acc6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79fe600 0x5563b79fe9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55202/0 (socket says 172.21.15.76:55202) 2024-03-08T06:01:09.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f65acc6b640 1 -- 172.21.15.76:0/2239689268 learned_addr learned my addr 172.21.15.76:0/2239689268 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:09.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f65acc6b640 1 -- 172.21.15.76:0/2239689268 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79f8d90 msgr2=0x5563b79f86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:09.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f65a77fe640 1 --2- 172.21.15.76:0/2239689268 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79f9940 0x5563b79c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f65a7fff640 1 --2- 172.21.15.76:0/2239689268 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79f8d90 0x5563b79f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f65acc6b640 1 --2- 172.21.15.76:0/2239689268 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79f8d90 0x5563b79f86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f65acc6b640 1 -- 172.21.15.76:0/2239689268 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79f9940 msgr2=0x5563b79c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:09.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f65acc6b640 1 --2- 172.21.15.76:0/2239689268 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79f9940 0x5563b79c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f65acc6b640 1 -- 172.21.15.76:0/2239689268 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563b77176b0 con 0x5563b79fe600 2024-03-08T06:01:09.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f65acc6b640 1 --2- 172.21.15.76:0/2239689268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79fe600 0x5563b79fe9e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f65a002cc70 tx=0x7f65a0009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6be955999a8fd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:09.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f658d7fa640 1 -- 172.21.15.76:0/2239689268 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0001250 con 0x5563b79fe600 2024-03-08T06:01:09.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f658d7fa640 1 -- 172.21.15.76:0/2239689268 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a000a480 con 0x5563b79fe600 2024-03-08T06:01:09.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.061+0000 7f658d7fa640 1 -- 172.21.15.76:0/2239689268 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a00019f0 con 0x5563b79fe600 2024-03-08T06:01:09.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.062+0000 7f65b1383740 1 -- 172.21.15.76:0/2239689268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79fe600 msgr2=0x5563b79fe9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.062+0000 7f65b1383740 1 --2- 172.21.15.76:0/2239689268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79fe600 0x5563b79fe9e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f65a002cc70 tx=0x7f65a0009030 comp rx=0 tx=0).stop 2024-03-08T06:01:09.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.062+0000 7f65b1383740 1 -- 172.21.15.76:0/2239689268 shutdown_connections 2024-03-08T06:01:09.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.062+0000 7f65b1383740 1 --2- 172.21.15.76:0/2239689268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79fe600 0x5563b79fe9e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.062+0000 7f65b1383740 1 --2- 172.21.15.76:0/2239689268 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79f9940 0x5563b79c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.062+0000 7f65b1383740 1 --2- 172.21.15.76:0/2239689268 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79f8d90 0x5563b79f86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.062+0000 7f65b1383740 1 -- 172.21.15.76:0/2239689268 >> 172.21.15.76:0/2239689268 conn(0x5563b79ce870 msgr2=0x5563b7a16bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:09.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.062+0000 7f65b1383740 1 -- 172.21.15.76:0/2239689268 shutdown_connections 2024-03-08T06:01:09.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.062+0000 7f65b1383740 1 -- 172.21.15.76:0/2239689268 wait complete. 2024-03-08T06:01:09.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.062+0000 7f65b1383740 1 -- start start 2024-03-08T06:01:09.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.063+0000 7f65b1383740 1 Processor -- start 2024-03-08T06:01:09.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.063+0000 7f65b1383740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79f8d90 0x5563b7a12fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.063+0000 7f65b1383740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79f9940 0x5563b79fede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.063+0000 7f65b1383740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79fe600 0x5563b7a16a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.063+0000 7f65b1383740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563b7a14840 con 0x5563b79f9940 2024-03-08T06:01:09.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.063+0000 7f65b1383740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8200770 con 0x5563b79f8d90 2024-03-08T06:01:09.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.063+0000 7f65b1383740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563b7a14e00 con 0x5563b79fe600 2024-03-08T06:01:09.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.063+0000 7f65b1383740 10 client.0 _ll_register_callbacks cb 0x7f6598001120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:01:09.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.063+0000 7f65a77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79f9940 0x5563b79fede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.063+0000 7f65a77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79f9940 0x5563b79fede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55214/0 (socket says 172.21.15.76:55214) 2024-03-08T06:01:09.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.063+0000 7f65a77fe640 1 -- 172.21.15.76:0/875840267 learned_addr learned my addr 172.21.15.76:0/875840267 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:09.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65acc6b640 1 --2- 172.21.15.76:0/875840267 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79fe600 0x5563b7a16a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65a7fff640 1 --2- 172.21.15.76:0/875840267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79f8d90 0x5563b7a12fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65a77fe640 1 -- 172.21.15.76:0/875840267 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79fe600 msgr2=0x5563b7a16a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:09.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65a77fe640 1 --2- 172.21.15.76:0/875840267 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79fe600 0x5563b7a16a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65a77fe640 1 -- 172.21.15.76:0/875840267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79f8d90 msgr2=0x5563b7a12fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:09.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65a77fe640 1 --2- 172.21.15.76:0/875840267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79f8d90 0x5563b7a12fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65a77fe640 1 -- 172.21.15.76:0/875840267 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563b79ff320 con 0x5563b79f9940 2024-03-08T06:01:09.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65a7fff640 1 --2- 172.21.15.76:0/875840267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79f8d90 0x5563b7a12fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:09.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65acc6b640 1 --2- 172.21.15.76:0/875840267 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79fe600 0x5563b7a16a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:09.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65a77fe640 1 --2- 172.21.15.76:0/875840267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79f9940 0x5563b79fede0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f659c013d80 tx=0x7f659c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d3071315a02426 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:09.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f6586ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-08T06:01:09.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f6586ffd640 1 -- 172.21.15.76:0/875840267 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c03c070 con 0x5563b79f9940 2024-03-08T06:01:09.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f6586ffd640 1 -- 172.21.15.76:0/875840267 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f659c0383a0 con 0x5563b79f9940 2024-03-08T06:01:09.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65b1383740 1 -- 172.21.15.76:0/875840267 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65a000a480 con 0x5563b79f9940 2024-03-08T06:01:09.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f6586ffd640 1 -- 172.21.15.76:0/875840267 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f659c047050 con 0x5563b79f9940 2024-03-08T06:01:09.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65b1383740 10 client.4836 fetch_fsmap learned FSMap version 8 2024-03-08T06:01:09.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65b1383740 1 -- 172.21.15.76:0/875840267 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5563b7a16f40 con 0x5563b79f9940 2024-03-08T06:01:09.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f6586ffd640 1 -- 172.21.15.76:0/875840267 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a000a480 con 0x5563b79f9940 2024-03-08T06:01:09.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f6586ffd640 1 -- 172.21.15.76:0/875840267 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 8) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f659c04a020 con 0x5563b79f9940 2024-03-08T06:01:09.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65b1383740 10 client.4836 fetch_fsmap finished waiting for FSMap version 8 2024-03-08T06:01:09.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65b1383740 10 client.4836 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-08T06:01:09.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65b1383740 20 client.4836 populate_metadata read hostname 'smithi076' 2024-03-08T06:01:09.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65b1383740 1 --2- 172.21.15.76:0/875840267 >> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] conn(0x5563b7a6aa70 0x5563b7a6ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65b1383740 4 client.4836 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T06:01:09.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.064+0000 7f65b1383740 1 -- 172.21.15.76:0/875840267 --> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5563b793ff40 con 0x5563b7a6aa70 2024-03-08T06:01:09.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.065+0000 7f65a7fff640 1 --2- 172.21.15.76:0/875840267 >> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] conn(0x5563b7a6aa70 0x5563b7a6ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.065+0000 7f6586ffd640 10 client.4836 ms_handle_connect on v2:172.21.15.81:6833/2357927814 2024-03-08T06:01:09.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.065+0000 7f65a7fff640 1 --2- 172.21.15.76:0/875840267 >> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] conn(0x5563b7a6aa70 0x5563b7a6ae50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1c84583bf7a1c3e server_cookie=c438bb1fc02a8f5e in_seq=0 out_seq=0 2024-03-08T06:01:09.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.065+0000 7f6586ffd640 1 -- 172.21.15.76:0/875840267 <== mds.0 v2:172.21.15.81:6833/2357927814 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x5563b793ff40 con 0x5563b7a6aa70 2024-03-08T06:01:09.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.065+0000 7f6586ffd640 10 client.4836 handle_command_reply: tid=0 2024-03-08T06:01:09.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.065+0000 7f65b1383740 1 client.4836 shutdown 2024-03-08T06:01:09.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.065+0000 7f65b1383740 1 -- 172.21.15.76:0/875840267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79f9940 msgr2=0x5563b79fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.065+0000 7f65b1383740 1 --2- 172.21.15.76:0/875840267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79f9940 0x5563b79fede0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f659c013d80 tx=0x7f659c004730 comp rx=0 tx=0).stop 2024-03-08T06:01:09.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65b1383740 1 -- 172.21.15.76:0/875840267 shutdown_connections 2024-03-08T06:01:09.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65b1383740 1 --2- 172.21.15.76:0/875840267 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x5563b79fe600 0x5563b7a16a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65b1383740 1 --2- 172.21.15.76:0/875840267 >> [v2:172.21.15.81:6833/2357927814,v1:172.21.15.81:6835/2357927814] conn(0x5563b7a6aa70 0x5563b7a6ae50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65b1383740 1 --2- 172.21.15.76:0/875840267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b79f9940 0x5563b79fede0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65b1383740 1 --2- 172.21.15.76:0/875840267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5563b79f8d90 0x5563b7a12fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f6586ffd640 0 client.4836 ms_handle_reset on v2:172.21.15.81:6833/2357927814 2024-03-08T06:01:09.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65b1383740 1 -- 172.21.15.76:0/875840267 >> 172.21.15.76:0/875840267 conn(0x5563b79ce870 msgr2=0x5563b79f8520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:09.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65b1383740 1 -- 172.21.15.76:0/875840267 shutdown_connections 2024-03-08T06:01:09.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65b1383740 1 -- 172.21.15.76:0/875840267 wait complete. 2024-03-08T06:01:09.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65b1383740 20 client.4836 trim_cache size 0 max 16384 2024-03-08T06:01:09.072 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-08T06:01:09.073 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "8a42235b-b7c3-45b2-aa98-336e6b1846d1", 2024-03-08T06:01:09.073 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-03-08T06:01:09.073 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4285, 2024-03-08T06:01:09.073 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-03-08T06:01:09.073 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-03-08T06:01:09.073 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-03-08T06:01:09.073 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 32.888111350000003, 2024-03-08T06:01:09.073 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 8, 2024-03-08T06:01:09.073 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 26, 2024-03-08T06:01:09.073 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 26, 2024-03-08T06:01:09.073 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 39.536738079999999 2024-03-08T06:01:09.073 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-08T06:01:09.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65ae6ce640 1 -- 172.21.15.76:0/800088217 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f659406b500 msgr2=0x7f659406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65ae6ce640 1 --2- 172.21.15.76:0/800088217 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f659406b500 0x7f659406d9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f65a815f720 tx=0x7f659c008040 comp rx=0 tx=0).stop 2024-03-08T06:01:09.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65ae6ce640 1 -- 172.21.15.76:0/800088217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6590004d40 msgr2=0x7f65a80b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65ae6ce640 1 --2- 172.21.15.76:0/800088217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6590004d40 0x7f65a80b4f60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f659800f850 tx=0x7f659800fd10 comp rx=0 tx=0).stop 2024-03-08T06:01:09.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65ae6ce640 1 -- 172.21.15.76:0/800088217 shutdown_connections 2024-03-08T06:01:09.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65ae6ce640 1 --2- 172.21.15.76:0/800088217 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f659406b500 0x7f659406d9c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65ae6ce640 1 --2- 172.21.15.76:0/800088217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65a80c55f0 0x7f65a80c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65ae6ce640 1 --2- 172.21.15.76:0/800088217 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65a8158d70 0x7f65a8158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65ae6ce640 1 --2- 172.21.15.76:0/800088217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6590004d40 0x7f65a80b4f60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65ae6ce640 1 -- 172.21.15.76:0/800088217 >> 172.21.15.76:0/800088217 conn(0x7f65a806ae60 msgr2=0x7f65a806f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:09.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65ae6ce640 1 -- 172.21.15.76:0/800088217 shutdown_connections 2024-03-08T06:01:09.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.066+0000 7f65ae6ce640 1 -- 172.21.15.76:0/800088217 wait complete. 2024-03-08T06:01:09.078 INFO:teuthology.run_tasks:Running task kclient... 2024-03-08T06:01:09.091 INFO:tasks.kclient:Mounting kernel clients... 2024-03-08T06:01:09.091 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-08T06:01:09.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:09.091 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-03-08T06:01:09.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-08T06:01:09.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:09.107 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-08T06:01:09.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:09.173 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2024-03-08T06:01:09.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:09.274 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-03-08T06:01:09.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:09.289 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2024-03-08T06:01:09.359 INFO:teuthology.orchestra.run.smithi076.stderr:Cannot find device "ceph-brx" 2024-03-08T06:01:09.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:01:09.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:09.361 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-03-08T06:01:09.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:09.380 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link delete ceph-brx 2024-03-08T06:01:09.454 INFO:teuthology.orchestra.run.smithi081.stderr:Cannot find device "ceph-brx" 2024-03-08T06:01:09.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:01:09.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:09.455 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-03-08T06:01:09.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:09.512 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link delete ceph-brx 2024-03-08T06:01:09.582 INFO:teuthology.orchestra.run.smithi081.stderr:Cannot find device "ceph-brx" 2024-03-08T06:01:09.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:01:09.584 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-03-08T06:01:09.584 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-08T06:01:09.584 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-08T06:01:09.584 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T06:01:09.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.685+0000 7f1b2465c640 1 Processor -- start 2024-03-08T06:01:09.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.685+0000 7f1b2465c640 1 -- start start 2024-03-08T06:01:09.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.686+0000 7f1b2465c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1b1c15a9f0 0x7f1b1c071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.686+0000 7f1b2465c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1c0722e0 con 0x7f1b1c0cec20 2024-03-08T06:01:09.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.686+0000 7f1b2465c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1c0ae8d0 con 0x7f1b1c0c98e0 2024-03-08T06:01:09.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.686+0000 7f1b2465c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b1c0ce790 con 0x7f1b1c15a9f0 2024-03-08T06:01:09.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.686+0000 7f1b223f8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1b1c15a9f0 0x7f1b1c071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.686+0000 7f1b22bf9640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1b1c0cec20 0x7f1b1c06e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49728/0 (socket says 172.21.15.76:49728) 2024-03-08T06:01:09.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.686+0000 7f1b22bf9640 1 -- 172.21.15.76:0/3059319573 learned_addr learned my addr 172.21.15.76:0/3059319573 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:09.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.686+0000 7f1b223f8640 1 -- 172.21.15.76:0/3059319573 >> v1:172.21.15.81:6789/0 conn(0x7f1b1c0c98e0 legacy=0x7f1b1c0c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:09.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.686+0000 7f1b223f8640 1 -- 172.21.15.76:0/3059319573 >> v1:172.21.15.76:6789/0 conn(0x7f1b1c0cec20 legacy=0x7f1b1c06e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.686+0000 7f1b223f8640 1 -- 172.21.15.76:0/3059319573 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c0aebb0 con 0x7f1b1c15a9f0 2024-03-08T06:01:09.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.686+0000 7f1b213f6640 1 -- 172.21.15.76:0/3059319573 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738350125 0 0) 0x7f1b1c0722e0 con 0x7f1b1c0cec20 2024-03-08T06:01:09.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.687+0000 7f1b223f8640 1 --2- 172.21.15.76:0/3059319573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c15a9f0 0x7f1b1c071da0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009d90 tx=0x7f1b1802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb346c23708039fa server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:09.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.687+0000 7f1b213f6640 1 -- 172.21.15.76:0/3059319573 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b18039070 con 0x7f1b1c15a9f0 2024-03-08T06:01:09.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.687+0000 7f1b213f6640 1 -- 172.21.15.76:0/3059319573 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1802cd00 con 0x7f1b1c15a9f0 2024-03-08T06:01:09.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.687+0000 7f1b213f6640 1 -- 172.21.15.76:0/3059319573 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b18034660 con 0x7f1b1c15a9f0 2024-03-08T06:01:09.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.688+0000 7f1b2465c640 1 -- 172.21.15.76:0/3059319573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c15a9f0 msgr2=0x7f1b1c071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.688+0000 7f1b2465c640 1 --2- 172.21.15.76:0/3059319573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c15a9f0 0x7f1b1c071da0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009d90 tx=0x7f1b1802bf50 comp rx=0 tx=0).stop 2024-03-08T06:01:09.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.688+0000 7f1b2465c640 1 -- 172.21.15.76:0/3059319573 shutdown_connections 2024-03-08T06:01:09.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.688+0000 7f1b2465c640 1 --2- 172.21.15.76:0/3059319573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c15a9f0 0x7f1b1c071da0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.688+0000 7f1b2465c640 1 -- 172.21.15.76:0/3059319573 >> 172.21.15.76:0/3059319573 conn(0x7f1b1c0cf740 msgr2=0x7f1b1c0cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:09.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.688+0000 7f1b2465c640 1 -- 172.21.15.76:0/3059319573 shutdown_connections 2024-03-08T06:01:09.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.688+0000 7f1b2465c640 1 -- 172.21.15.76:0/3059319573 wait complete. 2024-03-08T06:01:09.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.689+0000 7f1b2465c640 1 Processor -- start 2024-03-08T06:01:09.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.689+0000 7f1b2465c640 1 -- start start 2024-03-08T06:01:09.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b2465c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b1c0c98e0 0x7f1b1c158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b2465c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c0cec20 0x7f1b1c15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b2465c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b1c15a9f0 0x7f1b1c158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b2465c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0ca4a0 con 0x7f1b1c0cec20 2024-03-08T06:01:09.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b2465c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0ce420 con 0x7f1b1c0c98e0 2024-03-08T06:01:09.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b2465c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0ca620 con 0x7f1b1c15a9f0 2024-03-08T06:01:09.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b21bf7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c0cec20 0x7f1b1c15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b21bf7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c0cec20 0x7f1b1c15b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55220/0 (socket says 172.21.15.76:55220) 2024-03-08T06:01:09.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b21bf7640 1 -- 172.21.15.76:0/783829870 learned_addr learned my addr 172.21.15.76:0/783829870 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:09.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b223f8640 1 --2- 172.21.15.76:0/783829870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b1c0c98e0 0x7f1b1c158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b22bf9640 1 --2- 172.21.15.76:0/783829870 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b1c15a9f0 0x7f1b1c158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b21bf7640 1 -- 172.21.15.76:0/783829870 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b1c15a9f0 msgr2=0x7f1b1c158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b21bf7640 1 --2- 172.21.15.76:0/783829870 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b1c15a9f0 0x7f1b1c158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b21bf7640 1 -- 172.21.15.76:0/783829870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b1c0c98e0 msgr2=0x7f1b1c158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b21bf7640 1 --2- 172.21.15.76:0/783829870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b1c0c98e0 0x7f1b1c158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b21bf7640 1 -- 172.21.15.76:0/783829870 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c15bc40 con 0x7f1b1c0cec20 2024-03-08T06:01:09.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b21bf7640 1 --2- 172.21.15.76:0/783829870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c0cec20 0x7f1b1c15b210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00c430 tx=0x7f1b0c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:09.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.690+0000 7f1b12ffd640 1 -- 172.21.15.76:0/783829870 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c017070 con 0x7f1b1c0cec20 2024-03-08T06:01:09.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.691+0000 7f1b12ffd640 1 -- 172.21.15.76:0/783829870 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c0052d0 con 0x7f1b1c0cec20 2024-03-08T06:01:09.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.691+0000 7f1b12ffd640 1 -- 172.21.15.76:0/783829870 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c003830 con 0x7f1b1c0cec20 2024-03-08T06:01:09.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.691+0000 7f1b2465c640 1 -- 172.21.15.76:0/783829870 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1c15bf20 con 0x7f1b1c0cec20 2024-03-08T06:01:09.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.691+0000 7f1b2465c640 1 -- 172.21.15.76:0/783829870 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1c15c4e0 con 0x7f1b1c0cec20 2024-03-08T06:01:09.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.691+0000 7f1b2465c640 1 -- 172.21.15.76:0/783829870 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1c04f030 con 0x7f1b1c0cec20 2024-03-08T06:01:09.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.692+0000 7f1b12ffd640 1 -- 172.21.15.76:0/783829870 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b0c01a070 con 0x7f1b1c0cec20 2024-03-08T06:01:09.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.692+0000 7f1b12ffd640 1 --2- 172.21.15.76:0/783829870 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b0006b500 0x7f1b0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:09.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.692+0000 7f1b12ffd640 1 -- 172.21.15.76:0/783829870 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1b0c089880 con 0x7f1b1c0cec20 2024-03-08T06:01:09.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.692+0000 7f1b223f8640 1 --2- 172.21.15.76:0/783829870 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b0006b500 0x7f1b0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:09.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.693+0000 7f1b223f8640 1 --2- 172.21.15.76:0/783829870 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b0006b500 0x7f1b0006d9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1b18036000 tx=0x7f1b18030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:09.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.697+0000 7f1b12ffd640 1 -- 172.21.15.76:0/783829870 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b0c057d70 con 0x7f1b1c0cec20 2024-03-08T06:01:09.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.903+0000 7f1b2465c640 1 -- 172.21.15.76:0/783829870 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1b1c0725a0 con 0x7f1b1c0cec20 2024-03-08T06:01:09.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.904+0000 7f1b12ffd640 1 -- 172.21.15.76:0/783829870 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1b0c05be10 con 0x7f1b1c0cec20 2024-03-08T06:01:09.904 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-08T06:01:09.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.905+0000 7f1b2465c640 1 -- 172.21.15.76:0/783829870 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b0006b500 msgr2=0x7f1b0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.905+0000 7f1b2465c640 1 --2- 172.21.15.76:0/783829870 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b0006b500 0x7f1b0006d9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1b18036000 tx=0x7f1b18030000 comp rx=0 tx=0).stop 2024-03-08T06:01:09.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.905+0000 7f1b2465c640 1 -- 172.21.15.76:0/783829870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c0cec20 msgr2=0x7f1b1c15b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:09.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.905+0000 7f1b2465c640 1 --2- 172.21.15.76:0/783829870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c0cec20 0x7f1b1c15b210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00c430 tx=0x7f1b0c00c8f0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.906+0000 7f1b2465c640 1 -- 172.21.15.76:0/783829870 shutdown_connections 2024-03-08T06:01:09.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.906+0000 7f1b2465c640 1 --2- 172.21.15.76:0/783829870 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b0006b500 0x7f1b0006d9c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.906+0000 7f1b2465c640 1 --2- 172.21.15.76:0/783829870 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1b1c15a9f0 0x7f1b1c158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.906+0000 7f1b2465c640 1 --2- 172.21.15.76:0/783829870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c0cec20 0x7f1b1c15b210 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.906+0000 7f1b2465c640 1 --2- 172.21.15.76:0/783829870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b1c0c98e0 0x7f1b1c158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:09.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.906+0000 7f1b2465c640 1 -- 172.21.15.76:0/783829870 >> 172.21.15.76:0/783829870 conn(0x7f1b1c0cf740 msgr2=0x7f1b1c0cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:09.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.906+0000 7f1b2465c640 1 -- 172.21.15.76:0/783829870 shutdown_connections 2024-03-08T06:01:09.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:09.906+0000 7f1b2465c640 1 -- 172.21.15.76:0/783829870 wait complete. 2024-03-08T06:01:09.917 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-08T06:01:09.917 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T06:01:09.917 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-03-08T06:01:09.917 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-08T06:01:09.917 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-08T06:01:09.917 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-08T06:01:09.917 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-08T06:01:09.917 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T06:01:09.917 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-08T06:01:09.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:09.918 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-03-08T06:01:09.933 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:01:09.933 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:01:09.933 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:01:09.933 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:09.933 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-03-08T06:01:09.933 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:09.933 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:09.933 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:09.933 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-03-08T06:01:09.933 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:09.933 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:09.934 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-03-08T06:01:09.934 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:09.934 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:09.934 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T06:01:09.934 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:09.934 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-08T06:01:09.934 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42737sec preferred_lft 42737sec 2024-03-08T06:01:09.934 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-03-08T06:01:09.934 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:09.934 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-08T06:01:09.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:09.935 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2024-03-08T06:01:09.935 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-08T06:01:09.935 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add name ceph-brx type bridge 2024-03-08T06:01:09.935 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr flush dev ceph-brx 2024-03-08T06:01:09.935 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set ceph-brx up 2024-03-08T06:01:09.935 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-08T06:01:09.935 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-08T06:01:10.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:10.041 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T06:01:10.066 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-03-08T06:01:10.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:10.068 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-03-08T06:01:10.124 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-03-08T06:01:10.124 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T06:01:10.124 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-08T06:01:10.124 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-08T06:01:10.124 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T06:01:10.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:10.124 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2024-03-08T06:01:10.124 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-08T06:01:10.124 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-08T06:01:10.124 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-08T06:01:10.125 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-08T06:01:10.125 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-08T06:01:10.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:10.219 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-03-08T06:01:10.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:10.234 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list-id 2024-03-08T06:01:10.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:10.289 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2024-03-08T06:01:10.289 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-08T06:01:10.289 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T06:01:10.289 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-08T06:01:10.289 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-08T06:01:10.372 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-08T06:01:10.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:10.372 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2024-03-08T06:01:10.372 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-08T06:01:10.372 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-08T06:01:10.372 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-08T06:01:10.372 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-08T06:01:10.372 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-08T06:01:10.372 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-08T06:01:10.372 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-08T06:01:10.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:10.497 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2024-03-08T06:01:10.497 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-08T06:01:10.497 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2024-03-08T06:01:10.497 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set dev brx.0 master ceph-brx 2024-03-08T06:01:10.497 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-08T06:01:10.538 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:01:10.538 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-08T06:01:10.593 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-08T06:01:10.593 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:01:10.593 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-08T06:01:10.647 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-08T06:01:10.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:10.647 DEBUG:teuthology.orchestra.run.smithi076:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:10.731 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:10.731 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T06:01:10.837 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:10.837 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T06:01:10.837 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.76:3300,172.21.15.81:3300,172.21.15.81:3301" 2024-03-08T06:01:10.838 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 0@8a42235b-b7c3-45b2-aa98-336e6b1846d1.cephfs=/ 2024-03-08T06:01:10.838 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.76:3300/172.21.15.81:3300/172.21.15.81:3301" will pass to kernel 2024-03-08T06:01:10.838 INFO:teuthology.orchestra.run.smithi076.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-08T06:01:10.838 INFO:teuthology.orchestra.run.smithi076.stdout: You just mounted a file system that supports labels which does not 2024-03-08T06:01:10.838 INFO:teuthology.orchestra.run.smithi076.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T06:01:10.838 INFO:teuthology.orchestra.run.smithi076.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T06:01:10.838 INFO:teuthology.orchestra.run.smithi076.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T06:01:10.840 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T06:01:10.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:10.840 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-08T06:01:10.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:10.867 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:01:10.922 INFO:teuthology.orchestra.run.smithi076.stdout:client4723 2024-03-08T06:01:10.923 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:01:10.923 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:10.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:10.977 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:01:11.033 INFO:teuthology.orchestra.run.smithi076.stdout:client4723 2024-03-08T06:01:11.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:11.033 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:01:11.089 INFO:teuthology.orchestra.run.smithi076.stdout:client4723 2024-03-08T06:01:11.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:11.090 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8a42235b-b7c3-45b2-aa98-336e6b1846d1.client4723/status') 2024-03-08T06:01:11.156 INFO:teuthology.orchestra.run.smithi076.stdout:instance: client.4723 (3)172.21.15.76:0/1992318094 2024-03-08T06:01:11.157 INFO:teuthology.orchestra.run.smithi076.stdout:blocklisted: false 2024-03-08T06:01:11.157 INFO:teuthology.orchestra.run.smithi076.stderr:0+1 records in 2024-03-08T06:01:11.157 INFO:teuthology.orchestra.run.smithi076.stderr:0+1 records out 2024-03-08T06:01:11.157 INFO:teuthology.orchestra.run.smithi076.stderr:70 bytes copied, 2.0611e-05 s, 3.4 MB/s 2024-03-08T06:01:11.158 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-03-08T06:01:11.158 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-08T06:01:11.158 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-08T06:01:11.159 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T06:01:11.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.298+0000 7f36c090c640 1 Processor -- start 2024-03-08T06:01:11.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.298+0000 7f36c090c640 1 -- start start 2024-03-08T06:01:11.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.299+0000 7f36c090c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f36bc15dc60 0x7f36bc0d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:11.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.299+0000 7f36c090c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f36bc15e040 0x7f36bc099b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:11.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.299+0000 7f36c090c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36bc0cfcc0 con 0x7f36bc06bfa0 2024-03-08T06:01:11.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.299+0000 7f36c090c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36bc0b4360 con 0x7f36bc15e040 2024-03-08T06:01:11.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.299+0000 7f36c090c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36bc0cff60 con 0x7f36bc15dc60 2024-03-08T06:01:11.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.299+0000 7f36b3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f36bc15e040 0x7f36bc099b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:11.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.299+0000 7f36b3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f36bc15e040 0x7f36bc099b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55234/0 (socket says 172.21.15.76:55234) 2024-03-08T06:01:11.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.299+0000 7f36b3fff640 1 -- 172.21.15.76:0/3260324358 learned_addr learned my addr 172.21.15.76:0/3260324358 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:11.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.299+0000 7f36ba59c640 1 --2- 172.21.15.76:0/3260324358 >> v2:172.21.15.81:3300/0 conn(0x7f36bc15dc60 0x7f36bc0d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:11.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.299+0000 7f36b3fff640 1 -- 172.21.15.76:0/3260324358 >> v2:172.21.15.81:3300/0 conn(0x7f36bc15dc60 msgr2=0x7f36bc0d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:11.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.299+0000 7f36b3fff640 1 --2- 172.21.15.76:0/3260324358 >> v2:172.21.15.81:3300/0 conn(0x7f36bc15dc60 0x7f36bc0d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:11.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.299+0000 7f36b3fff640 1 -- 172.21.15.76:0/3260324358 >> v1:172.21.15.81:6789/0 conn(0x7f36bc06bfa0 legacy=0x7f36bc0cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:11.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.299+0000 7f36b3fff640 1 -- 172.21.15.76:0/3260324358 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36bc0d0240 con 0x7f36bc15e040 2024-03-08T06:01:11.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.300+0000 7f36b3fff640 1 --2- 172.21.15.76:0/3260324358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc15e040 0x7f36bc099b40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f36bc072ac0 tx=0x7f36a402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee328102b2c74cb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:11.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.304+0000 7f36b9d9b640 1 -- 172.21.15.76:0/3260324358 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4034070 con 0x7f36bc15e040 2024-03-08T06:01:11.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.304+0000 7f36b9d9b640 1 -- 172.21.15.76:0/3260324358 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a4030860 con 0x7f36bc15e040 2024-03-08T06:01:11.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.304+0000 7f36b9d9b640 1 -- 172.21.15.76:0/3260324358 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4030c30 con 0x7f36bc15e040 2024-03-08T06:01:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.304+0000 7f36c090c640 1 -- 172.21.15.76:0/3260324358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc15e040 msgr2=0x7f36bc099b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.304+0000 7f36c090c640 1 --2- 172.21.15.76:0/3260324358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc15e040 0x7f36bc099b40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f36bc072ac0 tx=0x7f36a402b9a0 comp rx=0 tx=0).stop 2024-03-08T06:01:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.304+0000 7f36c090c640 1 -- 172.21.15.76:0/3260324358 shutdown_connections 2024-03-08T06:01:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.304+0000 7f36c090c640 1 --2- 172.21.15.76:0/3260324358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc15e040 0x7f36bc099b40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.304+0000 7f36c090c640 1 --2- 172.21.15.76:0/3260324358 >> v2:172.21.15.81:3300/0 conn(0x7f36bc15dc60 0x7f36bc0d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.304+0000 7f36c090c640 1 -- 172.21.15.76:0/3260324358 >> 172.21.15.76:0/3260324358 conn(0x7f36bc0b3440 msgr2=0x7f36bc0b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:11.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.304+0000 7f36c090c640 1 -- 172.21.15.76:0/3260324358 shutdown_connections 2024-03-08T06:01:11.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.305+0000 7f36c090c640 1 -- 172.21.15.76:0/3260324358 wait complete. 2024-03-08T06:01:11.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.306+0000 7f36c090c640 1 Processor -- start 2024-03-08T06:01:11.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.306+0000 7f36c090c640 1 -- start start 2024-03-08T06:01:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36c090c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc06bfa0 0x7f36bc1f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36c090c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f36bc15dc60 0x7f36bc15fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36c090c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36bc15e040 0x7f36bc070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36c090c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc0d0ae0 con 0x7f36bc06bfa0 2024-03-08T06:01:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36c090c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc0d0800 con 0x7f36bc15e040 2024-03-08T06:01:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36c090c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc0d1540 con 0x7f36bc15dc60 2024-03-08T06:01:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36ba59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc06bfa0 0x7f36bc1f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36ba59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc06bfa0 0x7f36bc1f3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55250/0 (socket says 172.21.15.76:55250) 2024-03-08T06:01:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36ba59c640 1 -- 172.21.15.76:0/1180887374 learned_addr learned my addr 172.21.15.76:0/1180887374 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36b3fff640 1 --2- 172.21.15.76:0/1180887374 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f36bc15dc60 0x7f36bc15fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36bad9d640 1 --2- 172.21.15.76:0/1180887374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36bc15e040 0x7f36bc070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36ba59c640 1 -- 172.21.15.76:0/1180887374 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f36bc15dc60 msgr2=0x7f36bc15fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36ba59c640 1 --2- 172.21.15.76:0/1180887374 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f36bc15dc60 0x7f36bc15fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36ba59c640 1 -- 172.21.15.76:0/1180887374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36bc15e040 msgr2=0x7f36bc070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36ba59c640 1 --2- 172.21.15.76:0/1180887374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36bc15e040 0x7f36bc070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36ba59c640 1 -- 172.21.15.76:0/1180887374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36bc1605a0 con 0x7f36bc06bfa0 2024-03-08T06:01:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36bad9d640 1 --2- 172.21.15.76:0/1180887374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36bc15e040 0x7f36bc070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36ba59c640 1 --2- 172.21.15.76:0/1180887374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc06bfa0 0x7f36bc1f3080 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00d830 tx=0x7f36ac00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36b2ffd640 1 -- 172.21.15.76:0/1180887374 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00b6c0 con 0x7f36bc06bfa0 2024-03-08T06:01:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36b2ffd640 1 -- 172.21.15.76:0/1180887374 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac019070 con 0x7f36bc06bfa0 2024-03-08T06:01:11.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36b2ffd640 1 -- 172.21.15.76:0/1180887374 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00ba80 con 0x7f36bc06bfa0 2024-03-08T06:01:11.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36c090c640 1 -- 172.21.15.76:0/1180887374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36bc1f6ee0 con 0x7f36bc06bfa0 2024-03-08T06:01:11.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.307+0000 7f36c090c640 1 -- 172.21.15.76:0/1180887374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36bc160170 con 0x7f36bc06bfa0 2024-03-08T06:01:11.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.308+0000 7f36c090c640 1 -- 172.21.15.76:0/1180887374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36bc04f030 con 0x7f36bc06bfa0 2024-03-08T06:01:11.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.309+0000 7f36b2ffd640 1 -- 172.21.15.76:0/1180887374 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36ac01e460 con 0x7f36bc06bfa0 2024-03-08T06:01:11.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.309+0000 7f36b2ffd640 1 --2- 172.21.15.76:0/1180887374 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f369c06b500 0x7f369c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:11.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.309+0000 7f36b3fff640 1 --2- 172.21.15.76:0/1180887374 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f369c06b500 0x7f369c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:11.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.309+0000 7f36b2ffd640 1 -- 172.21.15.76:0/1180887374 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f36ac08aa90 con 0x7f36bc06bfa0 2024-03-08T06:01:11.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.309+0000 7f36b3fff640 1 --2- 172.21.15.76:0/1180887374 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f369c06b500 0x7f369c06d9c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f36a4036000 tx=0x7f36a403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:11.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.313+0000 7f36b2ffd640 1 -- 172.21.15.76:0/1180887374 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f36ac059660 con 0x7f36bc06bfa0 2024-03-08T06:01:11.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.522+0000 7f36c090c640 1 -- 172.21.15.76:0/1180887374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f36bc09ada0 con 0x7f36bc06bfa0 2024-03-08T06:01:11.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.522+0000 7f36b2ffd640 1 -- 172.21.15.76:0/1180887374 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f36ac058f60 con 0x7f36bc06bfa0 2024-03-08T06:01:11.523 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-08T06:01:11.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.524+0000 7f36c090c640 1 -- 172.21.15.76:0/1180887374 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f369c06b500 msgr2=0x7f369c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:11.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.524+0000 7f36c090c640 1 --2- 172.21.15.76:0/1180887374 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f369c06b500 0x7f369c06d9c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f36a4036000 tx=0x7f36a403b040 comp rx=0 tx=0).stop 2024-03-08T06:01:11.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.524+0000 7f36c090c640 1 -- 172.21.15.76:0/1180887374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc06bfa0 msgr2=0x7f36bc1f3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:11.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.524+0000 7f36c090c640 1 --2- 172.21.15.76:0/1180887374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc06bfa0 0x7f36bc1f3080 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00d830 tx=0x7f36ac00dcf0 comp rx=0 tx=0).stop 2024-03-08T06:01:11.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.524+0000 7f36c090c640 1 -- 172.21.15.76:0/1180887374 shutdown_connections 2024-03-08T06:01:11.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.524+0000 7f36c090c640 1 --2- 172.21.15.76:0/1180887374 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f369c06b500 0x7f369c06d9c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:11.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.524+0000 7f36c090c640 1 --2- 172.21.15.76:0/1180887374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36bc15e040 0x7f36bc070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:11.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.524+0000 7f36c090c640 1 --2- 172.21.15.76:0/1180887374 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f36bc15dc60 0x7f36bc15fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:11.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.524+0000 7f36c090c640 1 --2- 172.21.15.76:0/1180887374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc06bfa0 0x7f36bc1f3080 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:11.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.524+0000 7f36c090c640 1 -- 172.21.15.76:0/1180887374 >> 172.21.15.76:0/1180887374 conn(0x7f36bc0b3440 msgr2=0x7f36bc0b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:11.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.524+0000 7f36c090c640 1 -- 172.21.15.76:0/1180887374 shutdown_connections 2024-03-08T06:01:11.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:11.524+0000 7f36c090c640 1 -- 172.21.15.76:0/1180887374 wait complete. 2024-03-08T06:01:11.535 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-08T06:01:11.536 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T06:01:11.536 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-03-08T06:01:11.536 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-08T06:01:11.536 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-08T06:01:11.536 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-08T06:01:11.536 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-08T06:01:11.536 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T06:01:11.536 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-08T06:01:11.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:11.536 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-03-08T06:01:11.551 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:01:11.551 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:01:11.551 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:01:11.551 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:11.551 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-03-08T06:01:11.551 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:11.551 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:11.551 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:11.552 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-03-08T06:01:11.552 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:11.552 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:11.552 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-03-08T06:01:11.552 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:11.552 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:11.552 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T06:01:11.552 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:11.552 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-08T06:01:11.552 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42736sec preferred_lft 42736sec 2024-03-08T06:01:11.552 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-03-08T06:01:11.552 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:11.552 INFO:teuthology.orchestra.run.smithi076.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T06:01:11.552 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 16:af:2b:28:6e:f4 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:11.552 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-08T06:01:11.553 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:11.553 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::b48c:6dff:fea8:e46d/64 scope link 2024-03-08T06:01:11.553 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:11.553 INFO:teuthology.orchestra.run.smithi076.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T06:01:11.553 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 16:af:2b:28:6e:f4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T06:01:11.553 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::14af:2bff:fe28:6ef4/64 scope link tentative 2024-03-08T06:01:11.553 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:11.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:11.553 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T06:01:11.617 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-03-08T06:01:11.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:11.618 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-03-08T06:01:11.674 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-03-08T06:01:11.674 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T06:01:11.674 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-08T06:01:11.674 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-08T06:01:11.674 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T06:01:11.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:11.674 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2024-03-08T06:01:11.675 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-08T06:01:11.675 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-08T06:01:11.675 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-08T06:01:11.675 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-08T06:01:11.675 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-08T06:01:11.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:11.770 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-03-08T06:01:11.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-08T06:01:11.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:11.785 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list-id 2024-03-08T06:01:11.839 INFO:teuthology.orchestra.run.smithi076.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-08T06:01:11.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:11.839 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2024-03-08T06:01:11.839 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-08T06:01:11.839 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-08T06:01:11.839 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-08T06:01:11.839 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-08T06:01:11.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:11.922 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-08T06:01:11.989 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:01:11.989 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:01:11.989 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:01:11.989 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:11.989 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-03-08T06:01:11.989 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:11.989 INFO:teuthology.orchestra.run.smithi076.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T06:01:11.989 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 46:de:0e:bf:a0:b9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-08T06:01:11.989 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-08T06:01:11.989 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:11.989 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::44de:eff:febf:a0b9/64 scope link tentative 2024-03-08T06:01:11.989 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:11.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:11.991 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-08T06:01:12.056 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:01:12.057 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:01:12.057 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:01:12.057 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:12.057 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-03-08T06:01:12.057 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:12.057 INFO:teuthology.orchestra.run.smithi076.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T06:01:12.057 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 46:de:0e:bf:a0:b9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-08T06:01:12.057 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-08T06:01:12.057 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:12.057 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::44de:eff:febf:a0b9/64 scope link tentative 2024-03-08T06:01:12.057 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:12.058 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-08T06:01:12.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:12.058 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2024-03-08T06:01:12.058 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-08T06:01:12.058 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-08T06:01:12.059 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-08T06:01:12.059 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-08T06:01:12.059 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-08T06:01:12.059 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-08T06:01:12.059 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-08T06:01:12.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:12.183 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2024-03-08T06:01:12.183 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-08T06:01:12.183 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.1 up 2024-03-08T06:01:12.183 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set dev brx.1 master ceph-brx 2024-03-08T06:01:12.183 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-08T06:01:12.225 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:01:12.225 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-08T06:01:12.282 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-08T06:01:12.282 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:01:12.283 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-08T06:01:12.337 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-03-08T06:01:12.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:12.337 DEBUG:teuthology.orchestra.run.smithi076:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:12.422 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:12.422 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T06:01:12.430 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:12.430 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T06:01:12.430 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.76:3300,172.21.15.81:3300,172.21.15.81:3301" 2024-03-08T06:01:12.430 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 2@8a42235b-b7c3-45b2-aa98-336e6b1846d1.cephfs=/ 2024-03-08T06:01:12.430 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.76:3300/172.21.15.81:3300/172.21.15.81:3301" will pass to kernel 2024-03-08T06:01:12.431 INFO:teuthology.orchestra.run.smithi076.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-03-08T06:01:12.431 INFO:teuthology.orchestra.run.smithi076.stdout: You just mounted a file system that supports labels which does not 2024-03-08T06:01:12.431 INFO:teuthology.orchestra.run.smithi076.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T06:01:12.431 INFO:teuthology.orchestra.run.smithi076.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T06:01:12.431 INFO:teuthology.orchestra.run.smithi076.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T06:01:12.433 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T06:01:12.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:12.433 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-08T06:01:12.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:12.459 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:01:12.515 INFO:teuthology.orchestra.run.smithi076.stdout:client4562 2024-03-08T06:01:12.515 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:01:12.515 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:12.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:12.571 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:01:12.627 INFO:teuthology.orchestra.run.smithi076.stdout:client4562 2024-03-08T06:01:12.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:12.627 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:01:12.683 INFO:teuthology.orchestra.run.smithi076.stdout:client4562 2024-03-08T06:01:12.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:12.684 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8a42235b-b7c3-45b2-aa98-336e6b1846d1.client4562/status') 2024-03-08T06:01:12.750 INFO:teuthology.orchestra.run.smithi076.stdout:instance: client.4562 (3)172.21.15.76:0/3730714726 2024-03-08T06:01:12.750 INFO:teuthology.orchestra.run.smithi076.stdout:blocklisted: false 2024-03-08T06:01:12.751 INFO:teuthology.orchestra.run.smithi076.stderr:0+1 records in 2024-03-08T06:01:12.751 INFO:teuthology.orchestra.run.smithi076.stderr:0+1 records out 2024-03-08T06:01:12.751 INFO:teuthology.orchestra.run.smithi076.stderr:70 bytes copied, 2.0316e-05 s, 3.4 MB/s 2024-03-08T06:01:12.752 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-03-08T06:01:12.752 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-08T06:01:12.752 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-08T06:01:12.752 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T06:01:12.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.892+0000 7faf0a7c2640 1 Processor -- start 2024-03-08T06:01:12.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.892+0000 7faf0a7c2640 1 -- start start 2024-03-08T06:01:12.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.892+0000 7faf0a7c2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faf040cfa90 0x7faf0415ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:12.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.892+0000 7faf0a7c2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf04158670 con 0x7faf0406c2e0 2024-03-08T06:01:12.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.892+0000 7faf0a7c2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf0406b390 con 0x7faf04071c70 2024-03-08T06:01:12.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.892+0000 7faf0a7c2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf040cfe70 con 0x7faf040cfa90 2024-03-08T06:01:12.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.893+0000 7faf037fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7faf0406c2e0 0x7faf0406ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49734/0 (socket says 172.21.15.76:49734) 2024-03-08T06:01:12.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.893+0000 7faf037fe640 1 -- 172.21.15.76:0/3977073263 learned_addr learned my addr 172.21.15.76:0/3977073263 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:12.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.893+0000 7faf03fff640 1 --2- 172.21.15.76:0/3977073263 >> v2:172.21.15.76:3300/0 conn(0x7faf040cfa90 0x7faf0415ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:12.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.893+0000 7faf03fff640 1 -- 172.21.15.76:0/3977073263 >> v1:172.21.15.81:6789/0 conn(0x7faf04071c70 legacy=0x7faf04072090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:12.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.893+0000 7faf03fff640 1 -- 172.21.15.76:0/3977073263 >> v1:172.21.15.76:6789/0 conn(0x7faf0406c2e0 legacy=0x7faf0406ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:12.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.893+0000 7faf03fff640 1 -- 172.21.15.76:0/3977073263 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf040ad090 con 0x7faf040cfa90 2024-03-08T06:01:12.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.893+0000 7faf02ffd640 1 -- 172.21.15.76:0/3977073263 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333143461 0 0) 0x7faef8001110 con 0x7faf0406c2e0 2024-03-08T06:01:12.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.894+0000 7faf03fff640 1 --2- 172.21.15.76:0/3977073263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf040cfa90 0x7faf0415ebd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faef40097a0 tx=0x7faef402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6e0579d89bc69a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:12.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.894+0000 7faf02ffd640 1 -- 172.21.15.76:0/3977073263 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4035020 con 0x7faf040cfa90 2024-03-08T06:01:12.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.894+0000 7faf02ffd640 1 -- 172.21.15.76:0/3977073263 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef4031830 con 0x7faf040cfa90 2024-03-08T06:01:12.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.894+0000 7faf02ffd640 1 -- 172.21.15.76:0/3977073263 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4031c00 con 0x7faf040cfa90 2024-03-08T06:01:12.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.894+0000 7faf0a7c2640 1 -- 172.21.15.76:0/3977073263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf040cfa90 msgr2=0x7faf0415ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:12.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.894+0000 7faf0a7c2640 1 --2- 172.21.15.76:0/3977073263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf040cfa90 0x7faf0415ebd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faef40097a0 tx=0x7faef402b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:12.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.894+0000 7faf0a7c2640 1 -- 172.21.15.76:0/3977073263 shutdown_connections 2024-03-08T06:01:12.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.894+0000 7faf0a7c2640 1 --2- 172.21.15.76:0/3977073263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf040cfa90 0x7faf0415ebd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:12.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.894+0000 7faf0a7c2640 1 -- 172.21.15.76:0/3977073263 >> 172.21.15.76:0/3977073263 conn(0x7faf040af650 msgr2=0x7faf040b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:12.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.894+0000 7faf0a7c2640 1 -- 172.21.15.76:0/3977073263 shutdown_connections 2024-03-08T06:01:12.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.894+0000 7faf0a7c2640 1 -- 172.21.15.76:0/3977073263 wait complete. 2024-03-08T06:01:12.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.895+0000 7faf0a7c2640 1 Processor -- start 2024-03-08T06:01:12.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.896+0000 7faf0a7c2640 1 -- start start 2024-03-08T06:01:12.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.896+0000 7faf0a7c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf0406c2e0 0x7faf04144730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:12.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.896+0000 7faf0a7c2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf04071c70 0x7faf04160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:12.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.896+0000 7faf0a7c2640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf040cfa90 0x7faf041612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:12.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.896+0000 7faf0a7c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0406e020 con 0x7faf0406c2e0 2024-03-08T06:01:12.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.896+0000 7faf0a7c2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0406dd40 con 0x7faf04071c70 2024-03-08T06:01:12.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.896+0000 7faf0a7c2640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0406fa70 con 0x7faf040cfa90 2024-03-08T06:01:12.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.896+0000 7faf03fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf0406c2e0 0x7faf04144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:12.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.896+0000 7faf03fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf0406c2e0 0x7faf04144730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55270/0 (socket says 172.21.15.76:55270) 2024-03-08T06:01:12.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.896+0000 7faf03fff640 1 -- 172.21.15.76:0/510080078 learned_addr learned my addr 172.21.15.76:0/510080078 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:12.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.897+0000 7faf037fe640 1 --2- 172.21.15.76:0/510080078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf04071c70 0x7faf04160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:12.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.897+0000 7faf08d5f640 1 --2- 172.21.15.76:0/510080078 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf040cfa90 0x7faf041612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:12.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.897+0000 7faf03fff640 1 -- 172.21.15.76:0/510080078 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf040cfa90 msgr2=0x7faf041612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:12.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.897+0000 7faf03fff640 1 --2- 172.21.15.76:0/510080078 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf040cfa90 0x7faf041612b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:12.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.897+0000 7faf03fff640 1 -- 172.21.15.76:0/510080078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf04071c70 msgr2=0x7faf04160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:12.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.897+0000 7faf03fff640 1 --2- 172.21.15.76:0/510080078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf04071c70 0x7faf04160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:12.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.897+0000 7faf03fff640 1 -- 172.21.15.76:0/510080078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf041452d0 con 0x7faf0406c2e0 2024-03-08T06:01:12.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.897+0000 7faf03fff640 1 --2- 172.21.15.76:0/510080078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf0406c2e0 0x7faf04144730 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faef4000c00 tx=0x7faef40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:12.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.897+0000 7faf00ff9640 1 -- 172.21.15.76:0/510080078 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4004230 con 0x7faf0406c2e0 2024-03-08T06:01:12.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.897+0000 7faf00ff9640 1 -- 172.21.15.76:0/510080078 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef40043d0 con 0x7faf0406c2e0 2024-03-08T06:01:12.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.897+0000 7faf0a7c2640 1 -- 172.21.15.76:0/510080078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf041455b0 con 0x7faf0406c2e0 2024-03-08T06:01:12.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.897+0000 7faf00ff9640 1 -- 172.21.15.76:0/510080078 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4003850 con 0x7faf0406c2e0 2024-03-08T06:01:12.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.897+0000 7faf0a7c2640 1 -- 172.21.15.76:0/510080078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf04144c70 con 0x7faf0406c2e0 2024-03-08T06:01:12.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.898+0000 7faf0a7c2640 1 -- 172.21.15.76:0/510080078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf0404f030 con 0x7faf0406c2e0 2024-03-08T06:01:12.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.898+0000 7faf00ff9640 1 -- 172.21.15.76:0/510080078 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faef404c060 con 0x7faf0406c2e0 2024-03-08T06:01:12.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.899+0000 7faf00ff9640 1 --2- 172.21.15.76:0/510080078 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faef006b4d0 0x7faef006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:12.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.899+0000 7faf00ff9640 1 -- 172.21.15.76:0/510080078 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7faef40ac010 con 0x7faf0406c2e0 2024-03-08T06:01:12.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.899+0000 7faf037fe640 1 --2- 172.21.15.76:0/510080078 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faef006b4d0 0x7faef006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:12.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.899+0000 7faf037fe640 1 --2- 172.21.15.76:0/510080078 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faef006b4d0 0x7faef006d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faf0415ddc0 tx=0x7faef8007970 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:12.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:12.903+0000 7faf00ff9640 1 -- 172.21.15.76:0/510080078 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faef4040070 con 0x7faf0406c2e0 2024-03-08T06:01:13.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:13.109+0000 7faf0a7c2640 1 -- 172.21.15.76:0/510080078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faf040ae140 con 0x7faf0406c2e0 2024-03-08T06:01:13.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:13.109+0000 7faf00ff9640 1 -- 172.21.15.76:0/510080078 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7faef407ac70 con 0x7faf0406c2e0 2024-03-08T06:01:13.109 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-08T06:01:13.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:13.111+0000 7faf0a7c2640 1 -- 172.21.15.76:0/510080078 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faef006b4d0 msgr2=0x7faef006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:13.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:13.111+0000 7faf0a7c2640 1 --2- 172.21.15.76:0/510080078 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faef006b4d0 0x7faef006d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faf0415ddc0 tx=0x7faef8007970 comp rx=0 tx=0).stop 2024-03-08T06:01:13.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:13.111+0000 7faf0a7c2640 1 -- 172.21.15.76:0/510080078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf0406c2e0 msgr2=0x7faf04144730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:13.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:13.111+0000 7faf0a7c2640 1 --2- 172.21.15.76:0/510080078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf0406c2e0 0x7faf04144730 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faef4000c00 tx=0x7faef40046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:13.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:13.111+0000 7faf0a7c2640 1 -- 172.21.15.76:0/510080078 shutdown_connections 2024-03-08T06:01:13.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:13.111+0000 7faf0a7c2640 1 --2- 172.21.15.76:0/510080078 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faef006b4d0 0x7faef006d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:13.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:13.111+0000 7faf0a7c2640 1 --2- 172.21.15.76:0/510080078 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf040cfa90 0x7faf041612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:13.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:13.111+0000 7faf0a7c2640 1 --2- 172.21.15.76:0/510080078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf04071c70 0x7faf04160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:13.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:13.111+0000 7faf0a7c2640 1 --2- 172.21.15.76:0/510080078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf0406c2e0 0x7faf04144730 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:13.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:13.111+0000 7faf0a7c2640 1 -- 172.21.15.76:0/510080078 >> 172.21.15.76:0/510080078 conn(0x7faf040af650 msgr2=0x7faf040b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:13.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:13.111+0000 7faf0a7c2640 1 -- 172.21.15.76:0/510080078 shutdown_connections 2024-03-08T06:01:13.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:13.111+0000 7faf0a7c2640 1 -- 172.21.15.76:0/510080078 wait complete. 2024-03-08T06:01:13.122 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-08T06:01:13.122 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T06:01:13.122 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2024-03-08T06:01:13.122 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-08T06:01:13.122 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-08T06:01:13.123 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-08T06:01:13.123 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-08T06:01:13.123 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T06:01:13.123 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-08T06:01:13.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:13.123 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2024-03-08T06:01:13.141 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:01:13.141 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:01:13.141 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:01:13.142 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:13.142 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-03-08T06:01:13.142 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:13.142 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:13.142 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:13.142 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2024-03-08T06:01:13.142 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:13.142 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:13.142 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2024-03-08T06:01:13.142 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:13.142 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:13.142 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T06:01:13.142 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:13.142 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-08T06:01:13.142 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42674sec preferred_lft 42674sec 2024-03-08T06:01:13.143 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2024-03-08T06:01:13.143 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:13.143 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-08T06:01:13.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:13.143 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-08T06:01:13.143 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-08T06:01:13.143 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link add name ceph-brx type bridge 2024-03-08T06:01:13.143 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip addr flush dev ceph-brx 2024-03-08T06:01:13.143 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set ceph-brx up 2024-03-08T06:01:13.143 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-08T06:01:13.143 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-08T06:01:13.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:13.401 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T06:01:13.428 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-03-08T06:01:13.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:13.430 DEBUG:teuthology.orchestra.run.smithi081:> route 2024-03-08T06:01:13.498 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2024-03-08T06:01:13.498 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T06:01:13.499 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-08T06:01:13.499 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-08T06:01:13.499 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T06:01:13.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:13.499 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-08T06:01:13.499 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-08T06:01:13.499 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-08T06:01:13.499 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-08T06:01:13.499 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-08T06:01:13.499 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-08T06:01:13.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:13.729 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-03-08T06:01:13.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:13.743 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list-id 2024-03-08T06:01:13.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:13.798 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-08T06:01:13.798 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-08T06:01:13.798 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-08T06:01:13.798 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-08T06:01:13.798 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-08T06:01:13.880 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-08T06:01:13.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:13.880 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-08T06:01:13.880 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-08T06:01:13.880 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-08T06:01:13.880 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-08T06:01:13.880 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-08T06:01:13.881 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-08T06:01:13.881 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-08T06:01:13.881 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-08T06:01:14.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:14.066 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-08T06:01:14.066 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-08T06:01:14.066 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2024-03-08T06:01:14.066 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set dev brx.0 master ceph-brx 2024-03-08T06:01:14.066 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-08T06:01:14.106 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:01:14.106 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-08T06:01:14.161 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-08T06:01:14.161 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:01:14.161 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-08T06:01:14.215 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-08T06:01:14.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:14.216 DEBUG:teuthology.orchestra.run.smithi081:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:14.300 INFO:teuthology.orchestra.run.smithi081.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:14.300 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T06:01:14.414 INFO:teuthology.orchestra.run.smithi081.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:14.414 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T06:01:14.414 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: resolved to: "172.21.15.76:3300,172.21.15.81:3300,172.21.15.81:3301" 2024-03-08T06:01:14.415 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: trying mount with new device syntax: 1@8a42235b-b7c3-45b2-aa98-336e6b1846d1.cephfs=/ 2024-03-08T06:01:14.415 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.76:3300/172.21.15.81:3300/172.21.15.81:3301" will pass to kernel 2024-03-08T06:01:14.415 INFO:teuthology.orchestra.run.smithi081.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-08T06:01:14.415 INFO:teuthology.orchestra.run.smithi081.stdout: You just mounted a file system that supports labels which does not 2024-03-08T06:01:14.415 INFO:teuthology.orchestra.run.smithi081.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T06:01:14.415 INFO:teuthology.orchestra.run.smithi081.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T06:01:14.415 INFO:teuthology.orchestra.run.smithi081.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T06:01:14.417 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T06:01:14.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:14.417 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-08T06:01:14.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:14.443 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:01:14.499 INFO:teuthology.orchestra.run.smithi081.stdout:client4863 2024-03-08T06:01:14.499 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:01:14.499 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:14.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:14.554 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:01:14.610 INFO:teuthology.orchestra.run.smithi081.stdout:client4863 2024-03-08T06:01:14.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:14.610 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:01:14.666 INFO:teuthology.orchestra.run.smithi081.stdout:client4863 2024-03-08T06:01:14.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:14.666 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8a42235b-b7c3-45b2-aa98-336e6b1846d1.client4863/status') 2024-03-08T06:01:14.733 INFO:teuthology.orchestra.run.smithi081.stdout:instance: client.4863 (3)172.21.15.81:0/746727914 2024-03-08T06:01:14.733 INFO:teuthology.orchestra.run.smithi081.stdout:blocklisted: false 2024-03-08T06:01:14.733 INFO:teuthology.orchestra.run.smithi081.stderr:0+1 records in 2024-03-08T06:01:14.733 INFO:teuthology.orchestra.run.smithi081.stderr:0+1 records out 2024-03-08T06:01:14.733 INFO:teuthology.orchestra.run.smithi081.stderr:69 bytes copied, 2.1557e-05 s, 3.2 MB/s 2024-03-08T06:01:14.734 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-03-08T06:01:14.734 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-08T06:01:14.734 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-08T06:01:14.734 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T06:01:14.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.843+0000 7fdba9064640 1 Processor -- start 2024-03-08T06:01:14.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.843+0000 7fdba9064640 1 -- start start 2024-03-08T06:01:14.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.844+0000 7fdba9064640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdba40b6d00 0x7fdba40d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:14.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.844+0000 7fdba9064640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba40cf180 con 0x7fdba415dde0 2024-03-08T06:01:14.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.844+0000 7fdba9064640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba40d1ed0 con 0x7fdba40cec10 2024-03-08T06:01:14.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.844+0000 7fdba9064640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba40d21b0 con 0x7fdba40b6d00 2024-03-08T06:01:14.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.844+0000 7fdba2d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdba415dde0 0x7fdba4162250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49746/0 (socket says 172.21.15.76:49746) 2024-03-08T06:01:14.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.844+0000 7fdba2d9d640 1 -- 172.21.15.76:0/1006386965 learned_addr learned my addr 172.21.15.76:0/1006386965 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:14.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.844+0000 7fdba259c640 1 --2- 172.21.15.76:0/1006386965 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdba40b6d00 0x7fdba40d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:14.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.845+0000 7fdba1d9b640 1 -- 172.21.15.76:0/1006386965 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80097271 0 0) 0x7fdba40cf180 con 0x7fdba415dde0 2024-03-08T06:01:14.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.845+0000 7fdba1d9b640 1 -- 172.21.15.76:0/1006386965 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb8c003520 con 0x7fdba415dde0 2024-03-08T06:01:14.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.845+0000 7fdba1d9b640 1 -- 172.21.15.76:0/1006386965 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159293225 0 0) 0x7fdba40d1ed0 con 0x7fdba40cec10 2024-03-08T06:01:14.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.845+0000 7fdba1d9b640 1 -- 172.21.15.76:0/1006386965 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdba40cf180 con 0x7fdba40cec10 2024-03-08T06:01:14.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.845+0000 7fdba259c640 1 -- 172.21.15.76:0/1006386965 >> v1:172.21.15.81:6789/0 conn(0x7fdba40cec10 legacy=0x7fdba40d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:14.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.845+0000 7fdba259c640 1 -- 172.21.15.76:0/1006386965 >> v1:172.21.15.76:6789/0 conn(0x7fdba415dde0 legacy=0x7fdba4162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:14.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.845+0000 7fdba259c640 1 -- 172.21.15.76:0/1006386965 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba40d09c0 con 0x7fdba40b6d00 2024-03-08T06:01:14.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.846+0000 7fdba259c640 1 --2- 172.21.15.76:0/1006386965 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdba40b6d00 0x7fdba40d0480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdb980097d0 tx=0x7fdb9802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61774dcfb94f58b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:14.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.846+0000 7fdba1d9b640 1 -- 172.21.15.76:0/1006386965 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98035020 con 0x7fdba40b6d00 2024-03-08T06:01:14.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.846+0000 7fdba1d9b640 1 -- 172.21.15.76:0/1006386965 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb98031800 con 0x7fdba40b6d00 2024-03-08T06:01:14.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.846+0000 7fdba1d9b640 1 -- 172.21.15.76:0/1006386965 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98031bb0 con 0x7fdba40b6d00 2024-03-08T06:01:14.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.846+0000 7fdba9064640 1 -- 172.21.15.76:0/1006386965 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdba40b6d00 msgr2=0x7fdba40d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:14.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.846+0000 7fdba9064640 1 --2- 172.21.15.76:0/1006386965 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdba40b6d00 0x7fdba40d0480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdb980097d0 tx=0x7fdb9802b8a0 comp rx=0 tx=0).stop 2024-03-08T06:01:14.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.846+0000 7fdba9064640 1 -- 172.21.15.76:0/1006386965 shutdown_connections 2024-03-08T06:01:14.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.846+0000 7fdba9064640 1 --2- 172.21.15.76:0/1006386965 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdba40b6d00 0x7fdba40d0480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:14.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.846+0000 7fdba9064640 1 -- 172.21.15.76:0/1006386965 >> 172.21.15.76:0/1006386965 conn(0x7fdba40b56e0 msgr2=0x7fdba40b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:14.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.846+0000 7fdba9064640 1 -- 172.21.15.76:0/1006386965 shutdown_connections 2024-03-08T06:01:14.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.846+0000 7fdba9064640 1 -- 172.21.15.76:0/1006386965 wait complete. 2024-03-08T06:01:14.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.848+0000 7fdba9064640 1 Processor -- start 2024-03-08T06:01:14.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.848+0000 7fdba9064640 1 -- start start 2024-03-08T06:01:14.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba9064640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdba40b6d00 0x7fdba40cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:14.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba9064640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdba40cec10 0x7fdba41f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:14.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba9064640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba415dde0 0x7fdba41f7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:14.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba9064640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba40d3240 con 0x7fdba415dde0 2024-03-08T06:01:14.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba9064640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba40d30c0 con 0x7fdba40cec10 2024-03-08T06:01:14.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba9064640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba40d33c0 con 0x7fdba40b6d00 2024-03-08T06:01:14.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba415dde0 0x7fdba41f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:14.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdba40cec10 0x7fdba41f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:14.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba2d9d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdba40b6d00 0x7fdba40cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:14.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba415dde0 0x7fdba41f7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55280/0 (socket says 172.21.15.76:55280) 2024-03-08T06:01:14.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba359e640 1 -- 172.21.15.76:0/2450834457 learned_addr learned my addr 172.21.15.76:0/2450834457 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:14.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba359e640 1 -- 172.21.15.76:0/2450834457 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdba40b6d00 msgr2=0x7fdba40cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:14.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba359e640 1 --2- 172.21.15.76:0/2450834457 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdba40b6d00 0x7fdba40cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:14.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba359e640 1 -- 172.21.15.76:0/2450834457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdba40cec10 msgr2=0x7fdba41f6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:14.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba359e640 1 --2- 172.21.15.76:0/2450834457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdba40cec10 0x7fdba41f6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:14.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba359e640 1 -- 172.21.15.76:0/2450834457 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba40cc6b0 con 0x7fdba415dde0 2024-03-08T06:01:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba259c640 1 --2- 172.21.15.76:0/2450834457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdba40cec10 0x7fdba41f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba359e640 1 --2- 172.21.15.76:0/2450834457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba415dde0 0x7fdba41f7320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c00c400 tx=0x7fdb9c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdb8b7fe640 1 -- 172.21.15.76:0/2450834457 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c017070 con 0x7fdba415dde0 2024-03-08T06:01:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdba9064640 1 -- 172.21.15.76:0/2450834457 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba40cc990 con 0x7fdba415dde0 2024-03-08T06:01:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdb8b7fe640 1 -- 172.21.15.76:0/2450834457 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9c0052d0 con 0x7fdba415dde0 2024-03-08T06:01:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.849+0000 7fdb8b7fe640 1 -- 172.21.15.76:0/2450834457 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c0037f0 con 0x7fdba415dde0 2024-03-08T06:01:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.850+0000 7fdba9064640 1 -- 172.21.15.76:0/2450834457 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba40ccf50 con 0x7fdba415dde0 2024-03-08T06:01:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.851+0000 7fdba9064640 1 -- 172.21.15.76:0/2450834457 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba404f030 con 0x7fdba415dde0 2024-03-08T06:01:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.851+0000 7fdb8b7fe640 1 -- 172.21.15.76:0/2450834457 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb9c01a070 con 0x7fdba415dde0 2024-03-08T06:01:14.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.851+0000 7fdb8b7fe640 1 --2- 172.21.15.76:0/2450834457 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdb9006b500 0x7fdb9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:14.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.851+0000 7fdb8b7fe640 1 -- 172.21.15.76:0/2450834457 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdb9c089890 con 0x7fdba415dde0 2024-03-08T06:01:14.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.851+0000 7fdba2d9d640 1 --2- 172.21.15.76:0/2450834457 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdb9006b500 0x7fdb9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:14.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.852+0000 7fdba2d9d640 1 --2- 172.21.15.76:0/2450834457 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdb9006b500 0x7fdb9006d9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdb940012e0 tx=0x7fdb940019f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:14.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:14.856+0000 7fdb8b7fe640 1 -- 172.21.15.76:0/2450834457 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb9c057d00 con 0x7fdba415dde0 2024-03-08T06:01:15.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:15.063+0000 7fdba9064640 1 -- 172.21.15.76:0/2450834457 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdba40d1870 con 0x7fdba415dde0 2024-03-08T06:01:15.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:15.063+0000 7fdb8b7fe640 1 -- 172.21.15.76:0/2450834457 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fdb9c05bda0 con 0x7fdba415dde0 2024-03-08T06:01:15.064 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-08T06:01:15.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:15.065+0000 7fdba9064640 1 -- 172.21.15.76:0/2450834457 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdb9006b500 msgr2=0x7fdb9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:15.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:15.065+0000 7fdba9064640 1 --2- 172.21.15.76:0/2450834457 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdb9006b500 0x7fdb9006d9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdb940012e0 tx=0x7fdb940019f0 comp rx=0 tx=0).stop 2024-03-08T06:01:15.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:15.065+0000 7fdba9064640 1 -- 172.21.15.76:0/2450834457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba415dde0 msgr2=0x7fdba41f7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:15.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:15.065+0000 7fdba9064640 1 --2- 172.21.15.76:0/2450834457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba415dde0 0x7fdba41f7320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c00c400 tx=0x7fdb9c00c8c0 comp rx=0 tx=0).stop 2024-03-08T06:01:15.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:15.065+0000 7fdba9064640 1 -- 172.21.15.76:0/2450834457 shutdown_connections 2024-03-08T06:01:15.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:15.065+0000 7fdba9064640 1 --2- 172.21.15.76:0/2450834457 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fdb9006b500 0x7fdb9006d9c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:15.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:15.065+0000 7fdba9064640 1 --2- 172.21.15.76:0/2450834457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba415dde0 0x7fdba41f7320 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:15.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:15.065+0000 7fdba9064640 1 --2- 172.21.15.76:0/2450834457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdba40cec10 0x7fdba41f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:15.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:15.065+0000 7fdba9064640 1 --2- 172.21.15.76:0/2450834457 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdba40b6d00 0x7fdba40cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:15.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:15.065+0000 7fdba9064640 1 -- 172.21.15.76:0/2450834457 >> 172.21.15.76:0/2450834457 conn(0x7fdba40b56e0 msgr2=0x7fdba40c8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:15.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:15.065+0000 7fdba9064640 1 -- 172.21.15.76:0/2450834457 shutdown_connections 2024-03-08T06:01:15.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:15.065+0000 7fdba9064640 1 -- 172.21.15.76:0/2450834457 wait complete. 2024-03-08T06:01:15.077 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-08T06:01:15.077 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T06:01:15.077 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2024-03-08T06:01:15.077 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-08T06:01:15.077 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-08T06:01:15.077 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-08T06:01:15.077 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-08T06:01:15.077 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T06:01:15.077 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-08T06:01:15.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:15.077 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2024-03-08T06:01:15.093 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:01:15.093 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:01:15.093 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:01:15.093 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:15.093 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-03-08T06:01:15.093 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:15.093 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:15.093 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:15.093 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2024-03-08T06:01:15.093 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:15.094 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:15.094 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2024-03-08T06:01:15.094 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:15.094 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:15.094 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T06:01:15.094 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:15.094 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-08T06:01:15.094 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42672sec preferred_lft 42672sec 2024-03-08T06:01:15.094 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2024-03-08T06:01:15.094 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:15.094 INFO:teuthology.orchestra.run.smithi081.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T06:01:15.094 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 52:ef:1d:a5:da:97 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:15.094 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-08T06:01:15.094 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:15.094 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::f0d7:8dff:fe91:22e0/64 scope link 2024-03-08T06:01:15.095 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:15.095 INFO:teuthology.orchestra.run.smithi081.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T06:01:15.095 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 52:ef:1d:a5:da:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-08T06:01:15.095 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::50ef:1dff:fea5:da97/64 scope link tentative 2024-03-08T06:01:15.095 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:15.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:15.095 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T06:01:15.159 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-03-08T06:01:15.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:15.161 DEBUG:teuthology.orchestra.run.smithi081:> route 2024-03-08T06:01:15.216 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2024-03-08T06:01:15.217 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T06:01:15.217 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-08T06:01:15.217 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-08T06:01:15.217 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T06:01:15.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:15.217 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-08T06:01:15.217 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-08T06:01:15.217 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-08T06:01:15.217 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-08T06:01:15.217 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-08T06:01:15.217 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-08T06:01:15.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:15.314 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-03-08T06:01:15.329 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-08T06:01:15.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:15.329 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list-id 2024-03-08T06:01:15.383 INFO:teuthology.orchestra.run.smithi081.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-08T06:01:15.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:15.384 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-08T06:01:15.384 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-08T06:01:15.384 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-08T06:01:15.384 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-08T06:01:15.384 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-08T06:01:15.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:15.466 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-08T06:01:15.532 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:01:15.532 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:01:15.532 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:01:15.532 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:15.532 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-03-08T06:01:15.533 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:15.533 INFO:teuthology.orchestra.run.smithi081.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T06:01:15.533 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether ea:ea:60:09:d7:f0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-08T06:01:15.533 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-08T06:01:15.533 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:15.533 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::e8ea:60ff:fe09:d7f0/64 scope link tentative 2024-03-08T06:01:15.533 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:15.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:15.534 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-08T06:01:15.601 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:01:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:01:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:01:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-03-08T06:01:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:15.601 INFO:teuthology.orchestra.run.smithi081.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T06:01:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether ea:ea:60:09:d7:f0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-08T06:01:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-08T06:01:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::e8ea:60ff:fe09:d7f0/64 scope link tentative 2024-03-08T06:01:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:15.603 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-08T06:01:15.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:15.603 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-08T06:01:15.603 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-08T06:01:15.603 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-08T06:01:15.603 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-08T06:01:15.603 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-08T06:01:15.603 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-08T06:01:15.603 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-08T06:01:15.603 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-08T06:01:15.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:15.728 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-08T06:01:15.728 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-08T06:01:15.728 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.1 up 2024-03-08T06:01:15.728 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set dev brx.1 master ceph-brx 2024-03-08T06:01:15.728 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-08T06:01:15.769 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:01:15.769 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-08T06:01:15.824 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-08T06:01:15.824 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:01:15.824 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-08T06:01:15.878 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-03-08T06:01:15.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:15.878 DEBUG:teuthology.orchestra.run.smithi081:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:15.962 INFO:teuthology.orchestra.run.smithi081.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:15.962 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T06:01:15.970 INFO:teuthology.orchestra.run.smithi081.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:15.971 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T06:01:15.971 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: resolved to: "172.21.15.76:3300,172.21.15.81:3300,172.21.15.81:3301" 2024-03-08T06:01:15.971 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: trying mount with new device syntax: 3@8a42235b-b7c3-45b2-aa98-336e6b1846d1.cephfs=/ 2024-03-08T06:01:15.971 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.76:3300/172.21.15.81:3300/172.21.15.81:3301" will pass to kernel 2024-03-08T06:01:15.971 INFO:teuthology.orchestra.run.smithi081.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-03-08T06:01:15.971 INFO:teuthology.orchestra.run.smithi081.stdout: You just mounted a file system that supports labels which does not 2024-03-08T06:01:15.971 INFO:teuthology.orchestra.run.smithi081.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T06:01:15.971 INFO:teuthology.orchestra.run.smithi081.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T06:01:15.971 INFO:teuthology.orchestra.run.smithi081.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T06:01:15.972 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T06:01:15.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:15.973 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-08T06:01:16.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:16.004 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:01:16.060 INFO:teuthology.orchestra.run.smithi081.stdout:client4574 2024-03-08T06:01:16.060 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:01:16.060 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:16.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:16.115 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:01:16.171 INFO:teuthology.orchestra.run.smithi081.stdout:client4574 2024-03-08T06:01:16.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:16.171 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:01:16.227 INFO:teuthology.orchestra.run.smithi081.stdout:client4574 2024-03-08T06:01:16.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:16.227 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8a42235b-b7c3-45b2-aa98-336e6b1846d1.client4574/status') 2024-03-08T06:01:16.293 INFO:teuthology.orchestra.run.smithi081.stdout:instance: client.4574 (3)192.168.0.2:0/983965454 2024-03-08T06:01:16.293 INFO:teuthology.orchestra.run.smithi081.stdout:blocklisted: false 2024-03-08T06:01:16.293 INFO:teuthology.orchestra.run.smithi081.stderr:0+1 records in 2024-03-08T06:01:16.293 INFO:teuthology.orchestra.run.smithi081.stderr:0+1 records out 2024-03-08T06:01:16.294 INFO:teuthology.orchestra.run.smithi081.stderr:68 bytes copied, 2.0074e-05 s, 3.4 MB/s 2024-03-08T06:01:16.295 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-08T06:01:16.310 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:16.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.419+0000 7f308b247640 1 Processor -- start 2024-03-08T06:01:16.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.419+0000 7f308b247640 1 -- start start 2024-03-08T06:01:16.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.420+0000 7f308b247640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f308406d3f0 0x7f30840cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:16.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.420+0000 7f308b247640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f30840cd310 0x7f30840cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:16.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.420+0000 7f308b247640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308406d880 con 0x7f3084071390 2024-03-08T06:01:16.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.420+0000 7f308b247640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308406e010 con 0x7f30840cd310 2024-03-08T06:01:16.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.420+0000 7f308b247640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308406e2f0 con 0x7f308406d3f0 2024-03-08T06:01:16.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.420+0000 7f30897e4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f30840cd310 0x7f30840cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:16.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.420+0000 7f30897e4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f30840cd310 0x7f30840cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55294/0 (socket says 172.21.15.76:55294) 2024-03-08T06:01:16.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.420+0000 7f30897e4640 1 -- 172.21.15.76:0/147659480 learned_addr learned my addr 172.21.15.76:0/147659480 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:16.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.420+0000 7f3083fff640 1 --2- 172.21.15.76:0/147659480 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f308406d3f0 0x7f30840cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:16.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.420+0000 7f30897e4640 1 -- 172.21.15.76:0/147659480 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f308406d3f0 msgr2=0x7f30840cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:16.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.420+0000 7f30897e4640 1 --2- 172.21.15.76:0/147659480 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f308406d3f0 0x7f30840cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.420+0000 7f30897e4640 1 -- 172.21.15.76:0/147659480 >> v1:172.21.15.81:6789/0 conn(0x7f3084071390 legacy=0x7f3084071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:16.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.420+0000 7f30837fe640 1 -- 172.21.15.76:0/147659480 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2583167579 0 0) 0x7f308406d880 con 0x7f3084071390 2024-03-08T06:01:16.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.420+0000 7f30897e4640 1 -- 172.21.15.76:0/147659480 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30840ce7b0 con 0x7f30840cd310 2024-03-08T06:01:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.421+0000 7f30897e4640 1 --2- 172.21.15.76:0/147659480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30840cd310 0x7f30840cd7c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f307c009ed0 tx=0x7f307c030370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e486a2cef50fb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.421+0000 7f30837fe640 1 -- 172.21.15.76:0/147659480 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c03c070 con 0x7f30840cd310 2024-03-08T06:01:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.421+0000 7f30837fe640 1 -- 172.21.15.76:0/147659480 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307c037600 con 0x7f30840cd310 2024-03-08T06:01:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.421+0000 7f30837fe640 1 -- 172.21.15.76:0/147659480 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c0379d0 con 0x7f30840cd310 2024-03-08T06:01:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.421+0000 7f308b247640 1 -- 172.21.15.76:0/147659480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30840cd310 msgr2=0x7f30840cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.421+0000 7f308b247640 1 --2- 172.21.15.76:0/147659480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30840cd310 0x7f30840cd7c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f307c009ed0 tx=0x7f307c030370 comp rx=0 tx=0).stop 2024-03-08T06:01:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.422+0000 7f308b247640 1 -- 172.21.15.76:0/147659480 shutdown_connections 2024-03-08T06:01:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.422+0000 7f308b247640 1 --2- 172.21.15.76:0/147659480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30840cd310 0x7f30840cd7c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.422+0000 7f308b247640 1 --2- 172.21.15.76:0/147659480 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f308406d3f0 0x7f30840cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.422+0000 7f308b247640 1 -- 172.21.15.76:0/147659480 >> 172.21.15.76:0/147659480 conn(0x7f30840cf740 msgr2=0x7f30840cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.422+0000 7f308b247640 1 -- 172.21.15.76:0/147659480 shutdown_connections 2024-03-08T06:01:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.422+0000 7f308b247640 1 -- 172.21.15.76:0/147659480 wait complete. 2024-03-08T06:01:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.423+0000 7f308b247640 1 Processor -- start 2024-03-08T06:01:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.423+0000 7f308b247640 1 -- start start 2024-03-08T06:01:16.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f308b247640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3084071390 0x7f3084154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:16.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f308b247640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30840cd310 0x7f308415cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:16.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f308b247640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308415d0f0 0x7f3084156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:16.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f308b247640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308415c060 con 0x7f30840cd310 2024-03-08T06:01:16.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f308b247640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308415bee0 con 0x7f308415d0f0 2024-03-08T06:01:16.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f308b247640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30840cdd00 con 0x7f3084071390 2024-03-08T06:01:16.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f3083fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30840cd310 0x7f308415cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:16.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f3083fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30840cd310 0x7f308415cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55310/0 (socket says 172.21.15.76:55310) 2024-03-08T06:01:16.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f3088fe3640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3084071390 0x7f3084154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:16.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f3083fff640 1 -- 172.21.15.76:0/343818930 learned_addr learned my addr 172.21.15.76:0/343818930 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:16.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f30897e4640 1 --2- 172.21.15.76:0/343818930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308415d0f0 0x7f3084156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:16.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f3083fff640 1 -- 172.21.15.76:0/343818930 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3084071390 msgr2=0x7f3084154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:16.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f3083fff640 1 --2- 172.21.15.76:0/343818930 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3084071390 0x7f3084154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f3083fff640 1 -- 172.21.15.76:0/343818930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308415d0f0 msgr2=0x7f3084156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:16.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f3083fff640 1 --2- 172.21.15.76:0/343818930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308415d0f0 0x7f3084156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f3083fff640 1 -- 172.21.15.76:0/343818930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308415c650 con 0x7f30840cd310 2024-03-08T06:01:16.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f30897e4640 1 --2- 172.21.15.76:0/343818930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308415d0f0 0x7f3084156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:16.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f3083fff640 1 --2- 172.21.15.76:0/343818930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30840cd310 0x7f308415cbb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f306c00e9b0 tx=0x7f306c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:16.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f30817fa640 1 -- 172.21.15.76:0/343818930 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c017070 con 0x7f30840cd310 2024-03-08T06:01:16.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f30817fa640 1 -- 172.21.15.76:0/343818930 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c00c9a0 con 0x7f30840cd310 2024-03-08T06:01:16.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f308b247640 1 -- 172.21.15.76:0/343818930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3084155d80 con 0x7f30840cd310 2024-03-08T06:01:16.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f30817fa640 1 -- 172.21.15.76:0/343818930 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c00cd90 con 0x7f30840cd310 2024-03-08T06:01:16.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.424+0000 7f308b247640 1 -- 172.21.15.76:0/343818930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308415c1e0 con 0x7f30840cd310 2024-03-08T06:01:16.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.425+0000 7f308b247640 1 -- 172.21.15.76:0/343818930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308404f030 con 0x7f30840cd310 2024-03-08T06:01:16.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.426+0000 7f30817fa640 1 -- 172.21.15.76:0/343818930 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f306c01a070 con 0x7f30840cd310 2024-03-08T06:01:16.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.426+0000 7f30817fa640 1 --2- 172.21.15.76:0/343818930 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f306406b500 0x7f306406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:16.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.426+0000 7f30817fa640 1 -- 172.21.15.76:0/343818930 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f306c08a4b0 con 0x7f30840cd310 2024-03-08T06:01:16.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.426+0000 7f3088fe3640 1 --2- 172.21.15.76:0/343818930 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f306406b500 0x7f306406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:16.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.427+0000 7f3088fe3640 1 --2- 172.21.15.76:0/343818930 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f306406b500 0x7f306406d9c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3074000f60 tx=0x7f3074009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:16.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.430+0000 7f30817fa640 1 -- 172.21.15.76:0/343818930 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f306c058970 con 0x7f30840cd310 2024-03-08T06:01:16.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.602+0000 7f308b247640 1 -- 172.21.15.76:0/343818930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30840cbfa0 con 0x7f30840cd310 2024-03-08T06:01:16.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.603+0000 7f30817fa640 1 -- 172.21.15.76:0/343818930 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18757 (secure 0 0 0) 0x7f306c05ca10 con 0x7f30840cd310 2024-03-08T06:01:16.604 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:16.604 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:01:16.309309+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:16.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.605+0000 7f308b247640 1 -- 172.21.15.76:0/343818930 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f306406b500 msgr2=0x7f306406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:16.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.605+0000 7f308b247640 1 --2- 172.21.15.76:0/343818930 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f306406b500 0x7f306406d9c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3074000f60 tx=0x7f3074009040 comp rx=0 tx=0).stop 2024-03-08T06:01:16.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.605+0000 7f308b247640 1 -- 172.21.15.76:0/343818930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30840cd310 msgr2=0x7f308415cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:16.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.605+0000 7f308b247640 1 --2- 172.21.15.76:0/343818930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30840cd310 0x7f308415cbb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f306c00e9b0 tx=0x7f306c00ee70 comp rx=0 tx=0).stop 2024-03-08T06:01:16.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.605+0000 7f308b247640 1 -- 172.21.15.76:0/343818930 shutdown_connections 2024-03-08T06:01:16.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.605+0000 7f308b247640 1 --2- 172.21.15.76:0/343818930 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f306406b500 0x7f306406d9c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.605+0000 7f308b247640 1 --2- 172.21.15.76:0/343818930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308415d0f0 0x7f3084156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.605+0000 7f308b247640 1 --2- 172.21.15.76:0/343818930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30840cd310 0x7f308415cbb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.605+0000 7f308b247640 1 --2- 172.21.15.76:0/343818930 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3084071390 0x7f3084154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.605+0000 7f308b247640 1 -- 172.21.15.76:0/343818930 >> 172.21.15.76:0/343818930 conn(0x7f30840cf740 msgr2=0x7f30840c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:16.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.605+0000 7f308b247640 1 -- 172.21.15.76:0/343818930 shutdown_connections 2024-03-08T06:01:16.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.605+0000 7f308b247640 1 -- 172.21.15.76:0/343818930 wait complete. 2024-03-08T06:01:16.617 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:00:33.926651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:00:34.474030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T06:00:38.207853+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T06:01:16.617 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:01:16.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.722+0000 7f645cdd0640 1 Processor -- start 2024-03-08T06:01:16.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.722+0000 7f645cdd0640 1 -- start start 2024-03-08T06:01:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.723+0000 7f645cdd0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6458159770 0x7f645815f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.723+0000 7f645cdd0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6458071890 0x7f64580cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.723+0000 7f645cdd0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645815fc10 con 0x7f64580cf9e0 2024-03-08T06:01:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.723+0000 7f645cdd0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645815c170 con 0x7f6458159770 2024-03-08T06:01:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.723+0000 7f645cdd0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645815fef0 con 0x7f6458071890 2024-03-08T06:01:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.723+0000 7f645659c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6458159770 0x7f645815f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.723+0000 7f644ffff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6458071890 0x7f64580cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.723+0000 7f6456d9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f64580cf9e0 0x7f645815c600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:35922/0 (socket says 172.21.15.76:35922) 2024-03-08T06:01:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.723+0000 7f645659c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6458159770 0x7f645815f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.76:60598/0 (socket says 172.21.15.76:60598) 2024-03-08T06:01:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.723+0000 7f6456d9d640 1 -- 172.21.15.76:0/2397083090 learned_addr learned my addr 172.21.15.76:0/2397083090 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.723+0000 7f645659c640 1 -- 172.21.15.76:0/2397083090 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6458071890 msgr2=0x7f64580cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.723+0000 7f645659c640 1 --2- 172.21.15.76:0/2397083090 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6458071890 0x7f64580cf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.723+0000 7f645659c640 1 -- 172.21.15.76:0/2397083090 >> v1:172.21.15.81:6789/0 conn(0x7f64580cf9e0 legacy=0x7f645815c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.723+0000 7f645659c640 1 -- 172.21.15.76:0/2397083090 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64581601d0 con 0x7f6458159770 2024-03-08T06:01:16.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.724+0000 7f645659c640 1 --2- 172.21.15.76:0/2397083090 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6458159770 0x7f645815f5c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f64480097a0 tx=0x7f644802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7ff7b72cc0906b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:16.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.724+0000 7f6455d9b640 1 -- 172.21.15.76:0/2397083090 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6448035020 con 0x7f6458159770 2024-03-08T06:01:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.724+0000 7f6455d9b640 1 -- 172.21.15.76:0/2397083090 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6448031830 con 0x7f6458159770 2024-03-08T06:01:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.724+0000 7f6455d9b640 1 -- 172.21.15.76:0/2397083090 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6448031c00 con 0x7f6458159770 2024-03-08T06:01:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.725+0000 7f645cdd0640 1 -- 172.21.15.76:0/2397083090 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6458159770 msgr2=0x7f645815f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.725+0000 7f645cdd0640 1 --2- 172.21.15.76:0/2397083090 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6458159770 0x7f645815f5c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f64480097a0 tx=0x7f644802b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.725+0000 7f645cdd0640 1 -- 172.21.15.76:0/2397083090 shutdown_connections 2024-03-08T06:01:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.725+0000 7f645cdd0640 1 --2- 172.21.15.76:0/2397083090 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6458071890 0x7f64580cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.725+0000 7f645cdd0640 1 --2- 172.21.15.76:0/2397083090 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6458159770 0x7f645815f5c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.725+0000 7f645cdd0640 1 -- 172.21.15.76:0/2397083090 >> 172.21.15.76:0/2397083090 conn(0x7f645806f530 msgr2=0x7f645806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.725+0000 7f645cdd0640 1 -- 172.21.15.76:0/2397083090 shutdown_connections 2024-03-08T06:01:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.725+0000 7f645cdd0640 1 -- 172.21.15.76:0/2397083090 wait complete. 2024-03-08T06:01:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.726+0000 7f645cdd0640 1 Processor -- start 2024-03-08T06:01:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.726+0000 7f645cdd0640 1 -- start start 2024-03-08T06:01:16.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.727+0000 7f645cdd0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6458071890 0x7f64580b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:16.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.727+0000 7f645cdd0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64580cf9e0 0x7f64580b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:16.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.727+0000 7f645cdd0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6458159770 0x7f6458148820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:16.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.727+0000 7f645cdd0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645815cd10 con 0x7f64580cf9e0 2024-03-08T06:01:16.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.727+0000 7f645cdd0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6458071d10 con 0x7f6458159770 2024-03-08T06:01:16.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.727+0000 7f645cdd0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645815ce90 con 0x7f6458071890 2024-03-08T06:01:16.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.727+0000 7f644ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64580cf9e0 0x7f64580b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:16.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.727+0000 7f644ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64580cf9e0 0x7f64580b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55318/0 (socket says 172.21.15.76:55318) 2024-03-08T06:01:16.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.727+0000 7f644ffff640 1 -- 172.21.15.76:0/555078524 learned_addr learned my addr 172.21.15.76:0/555078524 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:16.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.727+0000 7f6456d9d640 1 --2- 172.21.15.76:0/555078524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6458159770 0x7f6458148820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:16.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.727+0000 7f645659c640 1 --2- 172.21.15.76:0/555078524 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6458071890 0x7f64580b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:16.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.727+0000 7f644ffff640 1 -- 172.21.15.76:0/555078524 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6458071890 msgr2=0x7f64580b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:16.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.728+0000 7f644ffff640 1 --2- 172.21.15.76:0/555078524 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6458071890 0x7f64580b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.728+0000 7f644ffff640 1 -- 172.21.15.76:0/555078524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6458159770 msgr2=0x7f6458148820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.728+0000 7f644ffff640 1 --2- 172.21.15.76:0/555078524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6458159770 0x7f6458148820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.728+0000 7f644ffff640 1 -- 172.21.15.76:0/555078524 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64580b1cf0 con 0x7f64580cf9e0 2024-03-08T06:01:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.728+0000 7f6456d9d640 1 --2- 172.21.15.76:0/555078524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6458159770 0x7f6458148820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.728+0000 7f645659c640 1 --2- 172.21.15.76:0/555078524 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6458071890 0x7f64580b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.728+0000 7f644ffff640 1 --2- 172.21.15.76:0/555078524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64580cf9e0 0x7f64580b14c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6440012ae0 tx=0x7f64400100c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.728+0000 7f644effd640 1 -- 172.21.15.76:0/555078524 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6440010a90 con 0x7f64580cf9e0 2024-03-08T06:01:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.728+0000 7f644effd640 1 -- 172.21.15.76:0/555078524 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644001d070 con 0x7f64580cf9e0 2024-03-08T06:01:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.728+0000 7f645cdd0640 1 -- 172.21.15.76:0/555078524 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64580b1fd0 con 0x7f64580cf9e0 2024-03-08T06:01:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.728+0000 7f644effd640 1 -- 172.21.15.76:0/555078524 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6440010e70 con 0x7f64580cf9e0 2024-03-08T06:01:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.728+0000 7f645cdd0640 1 -- 172.21.15.76:0/555078524 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64580b2590 con 0x7f64580cf9e0 2024-03-08T06:01:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.729+0000 7f645cdd0640 1 -- 172.21.15.76:0/555078524 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645804f030 con 0x7f64580cf9e0 2024-03-08T06:01:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.729+0000 7f644effd640 1 -- 172.21.15.76:0/555078524 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6440016050 con 0x7f64580cf9e0 2024-03-08T06:01:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.730+0000 7f644effd640 1 --2- 172.21.15.76:0/555078524 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f643806b4d0 0x7f643806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.730+0000 7f645659c640 1 --2- 172.21.15.76:0/555078524 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f643806b4d0 0x7f643806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.730+0000 7f644effd640 1 -- 172.21.15.76:0/555078524 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f644008e7c0 con 0x7f64580cf9e0 2024-03-08T06:01:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.730+0000 7f645659c640 1 --2- 172.21.15.76:0/555078524 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f643806b4d0 0x7f643806d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6448036000 tx=0x7f6448008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:16.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.734+0000 7f644effd640 1 -- 172.21.15.76:0/555078524 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6440027020 con 0x7f64580cf9e0 2024-03-08T06:01:16.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.929+0000 7f645cdd0640 1 -- 172.21.15.76:0/555078524 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f64580703a0 con 0x7f64580cf9e0 2024-03-08T06:01:16.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.929+0000 7f644effd640 1 -- 172.21.15.76:0/555078524 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6440014040 con 0x7f64580cf9e0 2024-03-08T06:01:16.930 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:01:16.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.932+0000 7f645cdd0640 1 -- 172.21.15.76:0/555078524 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f643806b4d0 msgr2=0x7f643806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:16.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.932+0000 7f645cdd0640 1 --2- 172.21.15.76:0/555078524 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f643806b4d0 0x7f643806d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6448036000 tx=0x7f6448008030 comp rx=0 tx=0).stop 2024-03-08T06:01:16.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.932+0000 7f645cdd0640 1 -- 172.21.15.76:0/555078524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64580cf9e0 msgr2=0x7f64580b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:16.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.932+0000 7f645cdd0640 1 --2- 172.21.15.76:0/555078524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64580cf9e0 0x7f64580b14c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6440012ae0 tx=0x7f64400100c0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.932+0000 7f645cdd0640 1 -- 172.21.15.76:0/555078524 shutdown_connections 2024-03-08T06:01:16.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.932+0000 7f645cdd0640 1 --2- 172.21.15.76:0/555078524 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f643806b4d0 0x7f643806d990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.932+0000 7f645cdd0640 1 --2- 172.21.15.76:0/555078524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6458159770 0x7f6458148820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.932+0000 7f645cdd0640 1 --2- 172.21.15.76:0/555078524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64580cf9e0 0x7f64580b14c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.932+0000 7f645cdd0640 1 --2- 172.21.15.76:0/555078524 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6458071890 0x7f64580b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:16.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.932+0000 7f645cdd0640 1 -- 172.21.15.76:0/555078524 >> 172.21.15.76:0/555078524 conn(0x7f645806f530 msgr2=0x7f64580ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:16.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.932+0000 7f645cdd0640 1 -- 172.21.15.76:0/555078524 shutdown_connections 2024-03-08T06:01:16.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:16.932+0000 7f645cdd0640 1 -- 172.21.15.76:0/555078524 wait complete. 2024-03-08T06:01:16.943 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:01:17.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.050+0000 7f024eeeb640 1 Processor -- start 2024-03-08T06:01:17.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.050+0000 7f024eeeb640 1 -- start start 2024-03-08T06:01:17.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.050+0000 7f024eeeb640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f024815dbc0 0x7f02480d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.050+0000 7f024eeeb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02480d27d0 con 0x7f02480b5cf0 2024-03-08T06:01:17.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.050+0000 7f024eeeb640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02480d2ab0 con 0x7f02480b9840 2024-03-08T06:01:17.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.050+0000 7f024eeeb640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02480d2d90 con 0x7f024815dbc0 2024-03-08T06:01:17.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f024d488640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f02480b5cf0 0x7f02480b6c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49760/0 (socket says 172.21.15.76:49760) 2024-03-08T06:01:17.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f024d488640 1 -- 172.21.15.76:0/1393222973 learned_addr learned my addr 172.21.15.76:0/1393222973 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:17.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f024cc87640 1 --2- 172.21.15.76:0/1393222973 >> v2:172.21.15.81:3300/0 conn(0x7f024815dbc0 0x7f02480d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f02477fe640 1 -- 172.21.15.76:0/1393222973 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441692856 0 0) 0x7f02480d27d0 con 0x7f02480b5cf0 2024-03-08T06:01:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f02477fe640 1 -- 172.21.15.76:0/1393222973 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0234003520 con 0x7f02480b5cf0 2024-03-08T06:01:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f02477fe640 1 -- 172.21.15.76:0/1393222973 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3741547495 0 0) 0x7f0234003520 con 0x7f02480b5cf0 2024-03-08T06:01:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f02477fe640 1 -- 172.21.15.76:0/1393222973 >> v2:172.21.15.81:3300/0 conn(0x7f024815dbc0 msgr2=0x7f02480d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f02477fe640 1 --2- 172.21.15.76:0/1393222973 >> v2:172.21.15.81:3300/0 conn(0x7f024815dbc0 0x7f02480d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f02477fe640 1 -- 172.21.15.76:0/1393222973 >> v1:172.21.15.81:6789/0 conn(0x7f02480b9840 legacy=0x7f02480b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f02477fe640 1 -- 172.21.15.76:0/1393222973 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02480d3070 con 0x7f02480b5cf0 2024-03-08T06:01:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f02477fe640 1 -- 172.21.15.76:0/1393222973 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f0230001670 con 0x7f02480b5cf0 2024-03-08T06:01:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f02477fe640 1 -- 172.21.15.76:0/1393222973 >> v1:172.21.15.76:6789/0 conn(0x7f02480b5cf0 legacy=0x7f02480b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f02477fe640 1 --2- 172.21.15.76:0/1393222973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0234003a20 0x7f0234003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f02477fe640 1 --2- 172.21.15.76:0/1393222973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0234004800 0x7f0234004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f02477fe640 1 --2- 172.21.15.76:0/1393222973 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f02340051c0 0x7f023400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f02477fe640 1 -- 172.21.15.76:0/1393222973 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02480d2d90 con 0x7f0234004800 2024-03-08T06:01:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f02477fe640 1 -- 172.21.15.76:0/1393222973 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023400c0f0 con 0x7f0234003a20 2024-03-08T06:01:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f02477fe640 1 -- 172.21.15.76:0/1393222973 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023400c2f0 con 0x7f02340051c0 2024-03-08T06:01:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f0247fff640 1 --2- 172.21.15.76:0/1393222973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0234004800 0x7f0234004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f024cc87640 1 --2- 172.21.15.76:0/1393222973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0234003a20 0x7f0234003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.051+0000 7f024d488640 1 --2- 172.21.15.76:0/1393222973 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f02340051c0 0x7f023400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.052+0000 7f0247fff640 1 -- 172.21.15.76:0/1393222973 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f02340051c0 msgr2=0x7f023400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.052+0000 7f0247fff640 1 --2- 172.21.15.76:0/1393222973 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f02340051c0 0x7f023400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.052+0000 7f0247fff640 1 -- 172.21.15.76:0/1393222973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0234003a20 msgr2=0x7f0234003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.052+0000 7f0247fff640 1 --2- 172.21.15.76:0/1393222973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0234003a20 0x7f0234003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.052+0000 7f024d488640 1 -- 172.21.15.76:0/1393222973 reap_dead start 2024-03-08T06:01:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.052+0000 7f0247fff640 1 -- 172.21.15.76:0/1393222973 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0234003520 con 0x7f0234004800 2024-03-08T06:01:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.052+0000 7f0247fff640 1 --2- 172.21.15.76:0/1393222973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0234004800 0x7f0234004c80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0238001e50 tx=0x7f0234005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d1319e66dde04d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.053+0000 7f02477fe640 1 -- 172.21.15.76:0/1393222973 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238001a70 con 0x7f0234004800 2024-03-08T06:01:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.053+0000 7f02477fe640 1 -- 172.21.15.76:0/1393222973 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0238003b70 con 0x7f0234004800 2024-03-08T06:01:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.053+0000 7f02477fe640 1 -- 172.21.15.76:0/1393222973 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02380050d0 con 0x7f0234004800 2024-03-08T06:01:17.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.053+0000 7f024eeeb640 1 -- 172.21.15.76:0/1393222973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0234004800 msgr2=0x7f0234004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.053+0000 7f024eeeb640 1 --2- 172.21.15.76:0/1393222973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0234004800 0x7f0234004c80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0238001e50 tx=0x7f0234005830 comp rx=0 tx=0).stop 2024-03-08T06:01:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.053+0000 7f024eeeb640 1 -- 172.21.15.76:0/1393222973 shutdown_connections 2024-03-08T06:01:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.053+0000 7f024eeeb640 1 --2- 172.21.15.76:0/1393222973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0234004800 0x7f0234004c80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.053+0000 7f024eeeb640 1 -- 172.21.15.76:0/1393222973 >> 172.21.15.76:0/1393222973 conn(0x7f02480670d0 msgr2=0x7f02480d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.053+0000 7f024eeeb640 1 -- 172.21.15.76:0/1393222973 shutdown_connections 2024-03-08T06:01:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.053+0000 7f024eeeb640 1 -- 172.21.15.76:0/1393222973 wait complete. 2024-03-08T06:01:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.055+0000 7f024eeeb640 1 Processor -- start 2024-03-08T06:01:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.055+0000 7f024eeeb640 1 -- start start 2024-03-08T06:01:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.055+0000 7f024eeeb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0234004800 0x7f0248158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.055+0000 7f024eeeb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02480cd540 0x7f02481f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.055+0000 7f024eeeb640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0248145770 0x7f0248145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.055+0000 7f024eeeb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024815e120 con 0x7f0234004800 2024-03-08T06:01:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.055+0000 7f024eeeb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024815dfa0 con 0x7f02480cd540 2024-03-08T06:01:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.055+0000 7f024eeeb640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02480b55e0 con 0x7f0248145770 2024-03-08T06:01:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f024cc87640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0234004800 0x7f0248158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f024cc87640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0234004800 0x7f0248158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55346/0 (socket says 172.21.15.76:55346) 2024-03-08T06:01:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f024cc87640 1 -- 172.21.15.76:0/148406682 learned_addr learned my addr 172.21.15.76:0/148406682 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f024d488640 1 --2- 172.21.15.76:0/148406682 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0248145770 0x7f0248145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f0247fff640 1 --2- 172.21.15.76:0/148406682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02480cd540 0x7f02481f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f024cc87640 1 -- 172.21.15.76:0/148406682 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0248145770 msgr2=0x7f0248145bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f024cc87640 1 --2- 172.21.15.76:0/148406682 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0248145770 0x7f0248145bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f024cc87640 1 -- 172.21.15.76:0/148406682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02480cd540 msgr2=0x7f02481f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f024cc87640 1 --2- 172.21.15.76:0/148406682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02480cd540 0x7f02481f31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f024cc87640 1 -- 172.21.15.76:0/148406682 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0248158e20 con 0x7f0234004800 2024-03-08T06:01:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f0247fff640 1 --2- 172.21.15.76:0/148406682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02480cd540 0x7f02481f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f024cc87640 1 --2- 172.21.15.76:0/148406682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0234004800 0x7f0248158830 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0240011890 tx=0x7f0240011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f02457fa640 1 -- 172.21.15.76:0/148406682 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240019070 con 0x7f0234004800 2024-03-08T06:01:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f02457fa640 1 -- 172.21.15.76:0/148406682 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0240004240 con 0x7f0234004800 2024-03-08T06:01:17.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f024eeeb640 1 -- 172.21.15.76:0/148406682 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0248159100 con 0x7f0234004800 2024-03-08T06:01:17.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f02457fa640 1 -- 172.21.15.76:0/148406682 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024000f820 con 0x7f0234004800 2024-03-08T06:01:17.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.056+0000 7f024eeeb640 1 -- 172.21.15.76:0/148406682 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02481596c0 con 0x7f0234004800 2024-03-08T06:01:17.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.058+0000 7f024eeeb640 1 -- 172.21.15.76:0/148406682 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f024804f030 con 0x7f0234004800 2024-03-08T06:01:17.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.058+0000 7f02457fa640 1 -- 172.21.15.76:0/148406682 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f024001c070 con 0x7f0234004800 2024-03-08T06:01:17.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.058+0000 7f02457fa640 1 --2- 172.21.15.76:0/148406682 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f022806b500 0x7f022806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.058+0000 7f02457fa640 1 -- 172.21.15.76:0/148406682 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f024008aa10 con 0x7f0234004800 2024-03-08T06:01:17.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.058+0000 7f0247fff640 1 --2- 172.21.15.76:0/148406682 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f022806b500 0x7f022806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.059+0000 7f0247fff640 1 --2- 172.21.15.76:0/148406682 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f022806b500 0x7f022806d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0238034000 tx=0x7f0238009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:17.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.063+0000 7f02457fa640 1 -- 172.21.15.76:0/148406682 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0240059490 con 0x7f0234004800 2024-03-08T06:01:17.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.242+0000 7f024eeeb640 1 -- 172.21.15.76:0/148406682 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f02480b77c0 con 0x7f0234004800 2024-03-08T06:01:17.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.243+0000 7f02457fa640 1 -- 172.21.15.76:0/148406682 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0240058d90 con 0x7f0234004800 2024-03-08T06:01:17.243 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:17.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.245+0000 7f024eeeb640 1 -- 172.21.15.76:0/148406682 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f022806b500 msgr2=0x7f022806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.245+0000 7f024eeeb640 1 --2- 172.21.15.76:0/148406682 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f022806b500 0x7f022806d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0238034000 tx=0x7f0238009030 comp rx=0 tx=0).stop 2024-03-08T06:01:17.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.245+0000 7f024eeeb640 1 -- 172.21.15.76:0/148406682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0234004800 msgr2=0x7f0248158830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.245+0000 7f024eeeb640 1 --2- 172.21.15.76:0/148406682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0234004800 0x7f0248158830 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0240011890 tx=0x7f0240011d50 comp rx=0 tx=0).stop 2024-03-08T06:01:17.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.245+0000 7f024eeeb640 1 -- 172.21.15.76:0/148406682 shutdown_connections 2024-03-08T06:01:17.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.245+0000 7f024eeeb640 1 --2- 172.21.15.76:0/148406682 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f022806b500 0x7f022806d9c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.245+0000 7f024eeeb640 1 --2- 172.21.15.76:0/148406682 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0248145770 0x7f0248145bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.245+0000 7f024eeeb640 1 --2- 172.21.15.76:0/148406682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02480cd540 0x7f02481f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.245+0000 7f024eeeb640 1 --2- 172.21.15.76:0/148406682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0234004800 0x7f0248158830 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.245+0000 7f024eeeb640 1 -- 172.21.15.76:0/148406682 >> 172.21.15.76:0/148406682 conn(0x7f02480670d0 msgr2=0x7f02480c7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:17.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.245+0000 7f024eeeb640 1 -- 172.21.15.76:0/148406682 shutdown_connections 2024-03-08T06:01:17.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.245+0000 7f024eeeb640 1 -- 172.21.15.76:0/148406682 wait complete. 2024-03-08T06:01:17.256 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:01:17.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.364+0000 7f823e2ff640 1 Processor -- start 2024-03-08T06:01:17.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.364+0000 7f823e2ff640 1 -- start start 2024-03-08T06:01:17.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.365+0000 7f823e2ff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8238154df0 0x7f82380b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.365+0000 7f823e2ff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f82380cc7e0 0x7f823815af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.365+0000 7f823e2ff640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823815f450 con 0x7f82380b3640 2024-03-08T06:01:17.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.365+0000 7f823e2ff640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82380af1e0 con 0x7f82380cc7e0 2024-03-08T06:01:17.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.365+0000 7f823e2ff640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82380b3c80 con 0x7f8238154df0 2024-03-08T06:01:17.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.365+0000 7f823c89c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f82380cc7e0 0x7f823815af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.365+0000 7f823c89c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f82380cc7e0 0x7f823815af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55360/0 (socket says 172.21.15.76:55360) 2024-03-08T06:01:17.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.365+0000 7f823c89c640 1 -- 172.21.15.76:0/983759580 learned_addr learned my addr 172.21.15.76:0/983759580 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:17.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.365+0000 7f82377fe640 1 --1- 172.21.15.76:0/983759580 >> v1:172.21.15.76:6789/0 conn(0x7f82380b3640 0x7f82380cc0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49770/0 (socket says 172.21.15.76:49770) 2024-03-08T06:01:17.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.365+0000 7f8237fff640 1 --2- 172.21.15.76:0/983759580 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8238154df0 0x7f82380b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.365+0000 7f823c89c640 1 -- 172.21.15.76:0/983759580 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8238154df0 msgr2=0x7f82380b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:17.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.365+0000 7f823c89c640 1 --2- 172.21.15.76:0/983759580 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8238154df0 0x7f82380b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.365+0000 7f823c89c640 1 -- 172.21.15.76:0/983759580 >> v1:172.21.15.76:6789/0 conn(0x7f82380b3640 legacy=0x7f82380cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.365+0000 7f8236ffd640 1 -- 172.21.15.76:0/983759580 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281649814 0 0) 0x7f823815f450 con 0x7f82380b3640 2024-03-08T06:01:17.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.365+0000 7f823c89c640 1 -- 172.21.15.76:0/983759580 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823815e690 con 0x7f82380cc7e0 2024-03-08T06:01:17.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.366+0000 7f823c89c640 1 --2- 172.21.15.76:0/983759580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82380cc7e0 0x7f823815af70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f823000d150 tx=0x7f823002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ca6954946c6fd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:17.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.366+0000 7f8236ffd640 1 -- 172.21.15.76:0/983759580 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823002ea80 con 0x7f82380cc7e0 2024-03-08T06:01:17.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.366+0000 7f8236ffd640 1 -- 172.21.15.76:0/983759580 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823002ec20 con 0x7f82380cc7e0 2024-03-08T06:01:17.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.366+0000 7f8236ffd640 1 -- 172.21.15.76:0/983759580 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230009600 con 0x7f82380cc7e0 2024-03-08T06:01:17.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.367+0000 7f823e2ff640 1 -- 172.21.15.76:0/983759580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82380cc7e0 msgr2=0x7f823815af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.367+0000 7f823e2ff640 1 --2- 172.21.15.76:0/983759580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82380cc7e0 0x7f823815af70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f823000d150 tx=0x7f823002e0e0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.367+0000 7f823e2ff640 1 -- 172.21.15.76:0/983759580 shutdown_connections 2024-03-08T06:01:17.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.367+0000 7f823e2ff640 1 --2- 172.21.15.76:0/983759580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82380cc7e0 0x7f823815af70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.367+0000 7f823e2ff640 1 --2- 172.21.15.76:0/983759580 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8238154df0 0x7f82380b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.367+0000 7f823e2ff640 1 -- 172.21.15.76:0/983759580 >> 172.21.15.76:0/983759580 conn(0x7f823806ac80 msgr2=0x7f823806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:17.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.367+0000 7f823e2ff640 1 -- 172.21.15.76:0/983759580 shutdown_connections 2024-03-08T06:01:17.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.367+0000 7f823e2ff640 1 -- 172.21.15.76:0/983759580 wait complete. 2024-03-08T06:01:17.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.368+0000 7f823e2ff640 1 Processor -- start 2024-03-08T06:01:17.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.368+0000 7f823e2ff640 1 -- start start 2024-03-08T06:01:17.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f823e2ff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82380b3640 0x7f8238147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f823e2ff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82380cc7e0 0x7f82380a8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f823e2ff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8238154df0 0x7f82381f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f823e2ff640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823815ef30 con 0x7f82380cc7e0 2024-03-08T06:01:17.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f823e2ff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823815ec50 con 0x7f82380b3640 2024-03-08T06:01:17.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f823e2ff640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82381551d0 con 0x7f8238154df0 2024-03-08T06:01:17.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f82377fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82380cc7e0 0x7f82380a8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f82377fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82380cc7e0 0x7f82380a8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55374/0 (socket says 172.21.15.76:55374) 2024-03-08T06:01:17.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f82377fe640 1 -- 172.21.15.76:0/4262723306 learned_addr learned my addr 172.21.15.76:0/4262723306 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:17.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f8237fff640 1 --2- 172.21.15.76:0/4262723306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82380b3640 0x7f8238147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f823c89c640 1 --2- 172.21.15.76:0/4262723306 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8238154df0 0x7f82381f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f82377fe640 1 -- 172.21.15.76:0/4262723306 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8238154df0 msgr2=0x7f82381f33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f82377fe640 1 --2- 172.21.15.76:0/4262723306 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8238154df0 0x7f82381f33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f82377fe640 1 -- 172.21.15.76:0/4262723306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82380b3640 msgr2=0x7f8238147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f82377fe640 1 --2- 172.21.15.76:0/4262723306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82380b3640 0x7f8238147c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f82377fe640 1 -- 172.21.15.76:0/4262723306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823815bc10 con 0x7f82380cc7e0 2024-03-08T06:01:17.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f82377fe640 1 --2- 172.21.15.76:0/4262723306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82380cc7e0 0x7f82380a8a40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f822c00c4d0 tx=0x7f822c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:17.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f8234ff9640 1 -- 172.21.15.76:0/4262723306 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c018070 con 0x7f82380cc7e0 2024-03-08T06:01:17.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f8234ff9640 1 -- 172.21.15.76:0/4262723306 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c007500 con 0x7f82380cc7e0 2024-03-08T06:01:17.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.369+0000 7f823e2ff640 1 -- 172.21.15.76:0/4262723306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823815bef0 con 0x7f82380cc7e0 2024-03-08T06:01:17.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.370+0000 7f8234ff9640 1 -- 172.21.15.76:0/4262723306 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c004a20 con 0x7f82380cc7e0 2024-03-08T06:01:17.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.370+0000 7f823e2ff640 1 -- 172.21.15.76:0/4262723306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823815c790 con 0x7f82380cc7e0 2024-03-08T06:01:17.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.371+0000 7f823e2ff640 1 -- 172.21.15.76:0/4262723306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823815c0c0 con 0x7f82380cc7e0 2024-03-08T06:01:17.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.371+0000 7f8234ff9640 1 -- 172.21.15.76:0/4262723306 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f822c01b070 con 0x7f82380cc7e0 2024-03-08T06:01:17.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.371+0000 7f8234ff9640 1 --2- 172.21.15.76:0/4262723306 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f822406b4d0 0x7f822406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.372+0000 7f8234ff9640 1 -- 172.21.15.76:0/4262723306 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f822c0898b0 con 0x7f82380cc7e0 2024-03-08T06:01:17.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.372+0000 7f8237fff640 1 --2- 172.21.15.76:0/4262723306 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f822406b4d0 0x7f822406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.372+0000 7f8237fff640 1 --2- 172.21.15.76:0/4262723306 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f822406b4d0 0x7f822406d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f82280079a0 tx=0x7f8228008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:17.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.380+0000 7f8234ff9640 1 -- 172.21.15.76:0/4262723306 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f822c057ce0 con 0x7f82380cc7e0 2024-03-08T06:01:17.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.558+0000 7f823e2ff640 1 -- 172.21.15.76:0/4262723306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f823815c960 con 0x7f82380cc7e0 2024-03-08T06:01:17.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.558+0000 7f8234ff9640 1 -- 172.21.15.76:0/4262723306 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f822c05bd80 con 0x7f82380cc7e0 2024-03-08T06:01:17.559 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:01:17.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.560+0000 7f823e2ff640 1 -- 172.21.15.76:0/4262723306 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f822406b4d0 msgr2=0x7f822406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.560+0000 7f823e2ff640 1 --2- 172.21.15.76:0/4262723306 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f822406b4d0 0x7f822406d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f82280079a0 tx=0x7f8228008040 comp rx=0 tx=0).stop 2024-03-08T06:01:17.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.560+0000 7f823e2ff640 1 -- 172.21.15.76:0/4262723306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82380cc7e0 msgr2=0x7f82380a8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.560+0000 7f823e2ff640 1 --2- 172.21.15.76:0/4262723306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82380cc7e0 0x7f82380a8a40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f822c00c4d0 tx=0x7f822c00c990 comp rx=0 tx=0).stop 2024-03-08T06:01:17.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.560+0000 7f823e2ff640 1 -- 172.21.15.76:0/4262723306 shutdown_connections 2024-03-08T06:01:17.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.560+0000 7f823e2ff640 1 --2- 172.21.15.76:0/4262723306 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f822406b4d0 0x7f822406d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.560+0000 7f823e2ff640 1 --2- 172.21.15.76:0/4262723306 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8238154df0 0x7f82381f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.560+0000 7f823e2ff640 1 --2- 172.21.15.76:0/4262723306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82380cc7e0 0x7f82380a8a40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.560+0000 7f823e2ff640 1 --2- 172.21.15.76:0/4262723306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82380b3640 0x7f8238147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.560+0000 7f823e2ff640 1 -- 172.21.15.76:0/4262723306 >> 172.21.15.76:0/4262723306 conn(0x7f823806ac80 msgr2=0x7f823806d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:17.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.560+0000 7f823e2ff640 1 -- 172.21.15.76:0/4262723306 shutdown_connections 2024-03-08T06:01:17.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.560+0000 7f823e2ff640 1 -- 172.21.15.76:0/4262723306 wait complete. 2024-03-08T06:01:17.571 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T06:01:17.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.675+0000 7fed6a44e640 1 Processor -- start 2024-03-08T06:01:17.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.675+0000 7fed6a44e640 1 -- start start 2024-03-08T06:01:17.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.675+0000 7fed6a44e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fed6415e120 0x7fed640cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.675+0000 7fed6a44e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fed6415e500 0x7fed640ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.675+0000 7fed6a44e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed640b1550 con 0x7fed640cf4a0 2024-03-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.675+0000 7fed6a44e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed640ae1d0 con 0x7fed6415e120 2024-03-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.675+0000 7fed6a44e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed640ae4b0 con 0x7fed6415e500 2024-03-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.676+0000 7fed63fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fed6415e120 0x7fed640cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.676+0000 7fed689eb640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fed640cf4a0 0x7fed640b01f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49772/0 (socket says 172.21.15.76:49772) 2024-03-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.676+0000 7fed63fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fed6415e120 0x7fed640cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55382/0 (socket says 172.21.15.76:55382) 2024-03-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.676+0000 7fed689eb640 1 -- 172.21.15.76:0/2411783773 learned_addr learned my addr 172.21.15.76:0/2411783773 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.676+0000 7fed637fe640 1 --2- 172.21.15.76:0/2411783773 >> v2:172.21.15.81:3300/0 conn(0x7fed6415e500 0x7fed640ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.676+0000 7fed63fff640 1 -- 172.21.15.76:0/2411783773 >> v2:172.21.15.81:3300/0 conn(0x7fed6415e500 msgr2=0x7fed640ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.676+0000 7fed63fff640 1 --2- 172.21.15.76:0/2411783773 >> v2:172.21.15.81:3300/0 conn(0x7fed6415e500 0x7fed640ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.676+0000 7fed63fff640 1 -- 172.21.15.76:0/2411783773 >> v1:172.21.15.76:6789/0 conn(0x7fed640cf4a0 legacy=0x7fed640b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.676+0000 7fed63fff640 1 -- 172.21.15.76:0/2411783773 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed640ae790 con 0x7fed6415e120 2024-03-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.676+0000 7fed62ffd640 1 -- 172.21.15.76:0/2411783773 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090253866 0 0) 0x7fed640b1550 con 0x7fed640cf4a0 2024-03-08T06:01:17.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.677+0000 7fed63fff640 1 --2- 172.21.15.76:0/2411783773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed6415e120 0x7fed640cb270 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fed640b3390 tx=0x7fed5402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9a0df785af1393 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:17.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.677+0000 7fed62ffd640 1 -- 172.21.15.76:0/2411783773 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54035020 con 0x7fed6415e120 2024-03-08T06:01:17.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.677+0000 7fed62ffd640 1 -- 172.21.15.76:0/2411783773 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed54031830 con 0x7fed6415e120 2024-03-08T06:01:17.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.677+0000 7fed62ffd640 1 -- 172.21.15.76:0/2411783773 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54031c00 con 0x7fed6415e120 2024-03-08T06:01:17.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.677+0000 7fed6a44e640 1 -- 172.21.15.76:0/2411783773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed6415e120 msgr2=0x7fed640cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.677+0000 7fed6a44e640 1 --2- 172.21.15.76:0/2411783773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed6415e120 0x7fed640cb270 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fed640b3390 tx=0x7fed5402b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.677+0000 7fed6a44e640 1 -- 172.21.15.76:0/2411783773 shutdown_connections 2024-03-08T06:01:17.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.677+0000 7fed6a44e640 1 --2- 172.21.15.76:0/2411783773 >> v2:172.21.15.81:3300/0 conn(0x7fed6415e500 0x7fed640ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.677+0000 7fed6a44e640 1 --2- 172.21.15.76:0/2411783773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed6415e120 0x7fed640cb270 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.677+0000 7fed6a44e640 1 -- 172.21.15.76:0/2411783773 >> 172.21.15.76:0/2411783773 conn(0x7fed640d0890 msgr2=0x7fed640d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:17.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.677+0000 7fed6a44e640 1 -- 172.21.15.76:0/2411783773 shutdown_connections 2024-03-08T06:01:17.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.678+0000 7fed6a44e640 1 -- 172.21.15.76:0/2411783773 wait complete. 2024-03-08T06:01:17.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.679+0000 7fed6a44e640 1 Processor -- start 2024-03-08T06:01:17.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.679+0000 7fed6a44e640 1 -- start start 2024-03-08T06:01:17.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.679+0000 7fed6a44e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed640cf4a0 0x7fed6414e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.679+0000 7fed6a44e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fed6415e120 0x7fed64155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.679+0000 7fed6a44e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed6415e500 0x7fed64155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.679+0000 7fed6a44e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed640b1da0 con 0x7fed640cf4a0 2024-03-08T06:01:17.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.679+0000 7fed6a44e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed640d1590 con 0x7fed6415e500 2024-03-08T06:01:17.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.679+0000 7fed6a44e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed640b1f20 con 0x7fed6415e120 2024-03-08T06:01:17.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed63fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed640cf4a0 0x7fed6414e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed63fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed640cf4a0 0x7fed6414e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55388/0 (socket says 172.21.15.76:55388) 2024-03-08T06:01:17.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed63fff640 1 -- 172.21.15.76:0/1230896021 learned_addr learned my addr 172.21.15.76:0/1230896021 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:17.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed689eb640 1 --2- 172.21.15.76:0/1230896021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed6415e500 0x7fed64155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed63fff640 1 -- 172.21.15.76:0/1230896021 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fed6415e120 msgr2=0x7fed64155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed637fe640 1 --2- 172.21.15.76:0/1230896021 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fed6415e120 0x7fed64155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed63fff640 1 --2- 172.21.15.76:0/1230896021 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fed6415e120 0x7fed64155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed63fff640 1 -- 172.21.15.76:0/1230896021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed6415e500 msgr2=0x7fed64155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed63fff640 1 --2- 172.21.15.76:0/1230896021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed6415e500 0x7fed64155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed63fff640 1 -- 172.21.15.76:0/1230896021 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed64157880 con 0x7fed640cf4a0 2024-03-08T06:01:17.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed689eb640 1 --2- 172.21.15.76:0/1230896021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed6415e500 0x7fed64155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:17.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed637fe640 1 --2- 172.21.15.76:0/1230896021 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fed6415e120 0x7fed64155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:17.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed63fff640 1 --2- 172.21.15.76:0/1230896021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed640cf4a0 0x7fed6414e0b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fed5402bee0 tx=0x7fed540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:17.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed60ff9640 1 -- 172.21.15.76:0/1230896021 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54041070 con 0x7fed640cf4a0 2024-03-08T06:01:17.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed60ff9640 1 -- 172.21.15.76:0/1230896021 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5403f070 con 0x7fed640cf4a0 2024-03-08T06:01:17.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed60ff9640 1 -- 172.21.15.76:0/1230896021 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54035040 con 0x7fed640cf4a0 2024-03-08T06:01:17.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed6a44e640 1 -- 172.21.15.76:0/1230896021 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6414e5f0 con 0x7fed640cf4a0 2024-03-08T06:01:17.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.680+0000 7fed6a44e640 1 -- 172.21.15.76:0/1230896021 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed64157dd0 con 0x7fed640cf4a0 2024-03-08T06:01:17.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.681+0000 7fed6a44e640 1 -- 172.21.15.76:0/1230896021 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6404f030 con 0x7fed640cf4a0 2024-03-08T06:01:17.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.682+0000 7fed60ff9640 1 -- 172.21.15.76:0/1230896021 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed54038020 con 0x7fed640cf4a0 2024-03-08T06:01:17.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.682+0000 7fed60ff9640 1 --2- 172.21.15.76:0/1230896021 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fed5006b4d0 0x7fed5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.682+0000 7fed60ff9640 1 -- 172.21.15.76:0/1230896021 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fed540abc60 con 0x7fed640cf4a0 2024-03-08T06:01:17.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.682+0000 7fed637fe640 1 --2- 172.21.15.76:0/1230896021 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fed5006b4d0 0x7fed5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.683+0000 7fed637fe640 1 --2- 172.21.15.76:0/1230896021 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fed5006b4d0 0x7fed5006d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fed58006fd0 tx=0x7fed58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:17.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.686+0000 7fed60ff9640 1 -- 172.21.15.76:0/1230896021 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed5407a770 con 0x7fed640cf4a0 2024-03-08T06:01:17.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.864+0000 7fed6a44e640 1 -- 172.21.15.76:0/1230896021 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fed640b0570 con 0x7fed640cf4a0 2024-03-08T06:01:17.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.864+0000 7fed60ff9640 1 -- 172.21.15.76:0/1230896021 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fed540ab070 con 0x7fed640cf4a0 2024-03-08T06:01:17.864 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.866+0000 7fed6a44e640 1 -- 172.21.15.76:0/1230896021 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fed5006b4d0 msgr2=0x7fed5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.866+0000 7fed6a44e640 1 --2- 172.21.15.76:0/1230896021 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fed5006b4d0 0x7fed5006d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fed58006fd0 tx=0x7fed58008040 comp rx=0 tx=0).stop 2024-03-08T06:01:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.866+0000 7fed6a44e640 1 -- 172.21.15.76:0/1230896021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed640cf4a0 msgr2=0x7fed6414e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.866+0000 7fed6a44e640 1 --2- 172.21.15.76:0/1230896021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed640cf4a0 0x7fed6414e0b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fed5402bee0 tx=0x7fed540046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.866+0000 7fed6a44e640 1 -- 172.21.15.76:0/1230896021 shutdown_connections 2024-03-08T06:01:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.866+0000 7fed6a44e640 1 --2- 172.21.15.76:0/1230896021 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fed5006b4d0 0x7fed5006d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.866+0000 7fed6a44e640 1 --2- 172.21.15.76:0/1230896021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed6415e500 0x7fed64155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.866+0000 7fed6a44e640 1 --2- 172.21.15.76:0/1230896021 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fed6415e120 0x7fed64155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.866+0000 7fed6a44e640 1 --2- 172.21.15.76:0/1230896021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed640cf4a0 0x7fed6414e0b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.866+0000 7fed6a44e640 1 -- 172.21.15.76:0/1230896021 >> 172.21.15.76:0/1230896021 conn(0x7fed640d0890 msgr2=0x7fed6406e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.866+0000 7fed6a44e640 1 -- 172.21.15.76:0/1230896021 shutdown_connections 2024-03-08T06:01:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.866+0000 7fed6a44e640 1 -- 172.21.15.76:0/1230896021 wait complete. 2024-03-08T06:01:17.877 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:17.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.977+0000 7f4b9fd4b640 1 Processor -- start 2024-03-08T06:01:17.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.977+0000 7f4b9fd4b640 1 -- start start 2024-03-08T06:01:17.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.978+0000 7f4b9fd4b640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4b98146b20 0x7f4b980aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.978+0000 7f4b9fd4b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4b98147760 0x7f4b9806f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.978+0000 7f4b9fd4b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b9806e730 con 0x7f4b980af020 2024-03-08T06:01:17.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.978+0000 7f4b9fd4b640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b9806e9b0 con 0x7f4b98147760 2024-03-08T06:01:17.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.978+0000 7f4b9fd4b640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9806ec90 con 0x7f4b98146b20 2024-03-08T06:01:17.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.978+0000 7f4b9e2e8640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4b98147760 0x7f4b9806f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.978+0000 7f4b9dae7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4b98146b20 0x7f4b980aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.978+0000 7f4b9dae7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4b98146b20 0x7f4b980aeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:47282/0 (socket says 172.21.15.76:47282) 2024-03-08T06:01:17.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.978+0000 7f4b9dae7640 1 -- 172.21.15.76:0/646281742 learned_addr learned my addr 172.21.15.76:0/646281742 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:17.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.978+0000 7f4b9e2e8640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4b98147760 0x7f4b9806f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.76:60686/0 (socket says 172.21.15.76:60686) 2024-03-08T06:01:17.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.978+0000 7f4b9d2e6640 1 --1- 172.21.15.76:0/646281742 >> v1:172.21.15.81:6789/0 conn(0x7f4b980af020 0x7f4b980b2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:35948/0 (socket says 172.21.15.76:35948) 2024-03-08T06:01:17.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.979+0000 7f4b9e2e8640 1 -- 172.21.15.76:0/646281742 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4b98146b20 msgr2=0x7f4b980aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:17.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.979+0000 7f4b9e2e8640 1 --2- 172.21.15.76:0/646281742 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4b98146b20 0x7f4b980aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.979+0000 7f4b9e2e8640 1 -- 172.21.15.76:0/646281742 >> v1:172.21.15.81:6789/0 conn(0x7f4b980af020 legacy=0x7f4b980b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.979+0000 7f4b9e2e8640 1 -- 172.21.15.76:0/646281742 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b980ae2e0 con 0x7f4b98147760 2024-03-08T06:01:17.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.979+0000 7f4b9cae5640 1 -- 172.21.15.76:0/646281742 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283663549 0 0) 0x7f4b8c001110 con 0x7f4b980af020 2024-03-08T06:01:17.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.979+0000 7f4b9dae7640 1 --2- 172.21.15.76:0/646281742 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4b98146b20 0x7f4b980aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:17.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.980+0000 7f4b9e2e8640 1 --2- 172.21.15.76:0/646281742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b98147760 0x7f4b9806f8c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b900097a0 tx=0x7f4b9002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaecf7e1ab5ac833 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:17.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.980+0000 7f4b9cae5640 1 -- 172.21.15.76:0/646281742 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90035020 con 0x7f4b98147760 2024-03-08T06:01:17.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.980+0000 7f4b9cae5640 1 -- 172.21.15.76:0/646281742 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b90031830 con 0x7f4b98147760 2024-03-08T06:01:17.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.980+0000 7f4b9cae5640 1 -- 172.21.15.76:0/646281742 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90031c00 con 0x7f4b98147760 2024-03-08T06:01:17.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.980+0000 7f4b9fd4b640 1 -- 172.21.15.76:0/646281742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b98147760 msgr2=0x7f4b9806f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.980+0000 7f4b9fd4b640 1 --2- 172.21.15.76:0/646281742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b98147760 0x7f4b9806f8c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b900097a0 tx=0x7f4b9002b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.980+0000 7f4b9fd4b640 1 -- 172.21.15.76:0/646281742 shutdown_connections 2024-03-08T06:01:17.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.980+0000 7f4b9fd4b640 1 --2- 172.21.15.76:0/646281742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b98147760 0x7f4b9806f8c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.980+0000 7f4b9fd4b640 1 --2- 172.21.15.76:0/646281742 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4b98146b20 0x7f4b980aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.980+0000 7f4b9fd4b640 1 -- 172.21.15.76:0/646281742 >> 172.21.15.76:0/646281742 conn(0x7f4b980b1aa0 msgr2=0x7f4b980b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:17.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.980+0000 7f4b9fd4b640 1 -- 172.21.15.76:0/646281742 shutdown_connections 2024-03-08T06:01:17.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.980+0000 7f4b9fd4b640 1 -- 172.21.15.76:0/646281742 wait complete. 2024-03-08T06:01:17.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.981+0000 7f4b9fd4b640 1 Processor -- start 2024-03-08T06:01:17.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.981+0000 7f4b9fd4b640 1 -- start start 2024-03-08T06:01:17.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.982+0000 7f4b9fd4b640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4b980af020 0x7f4b9815d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.982+0000 7f4b9fd4b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b98146b20 0x7f4b980c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.982+0000 7f4b9fd4b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b98147760 0x7f4b980c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.982+0000 7f4b9fd4b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b980adf00 con 0x7f4b98147760 2024-03-08T06:01:17.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.982+0000 7f4b9fd4b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b98146f00 con 0x7f4b98146b20 2024-03-08T06:01:17.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.982+0000 7f4b9fd4b640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b980ae080 con 0x7f4b980af020 2024-03-08T06:01:17.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.982+0000 7f4b9e2e8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b98147760 0x7f4b980c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.982+0000 7f4b9e2e8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b98147760 0x7f4b980c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55402/0 (socket says 172.21.15.76:55402) 2024-03-08T06:01:17.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.982+0000 7f4b9e2e8640 1 -- 172.21.15.76:0/2822229094 learned_addr learned my addr 172.21.15.76:0/2822229094 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:17.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.982+0000 7f4b9dae7640 1 --2- 172.21.15.76:0/2822229094 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4b980af020 0x7f4b9815d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.983+0000 7f4b9d2e6640 1 --2- 172.21.15.76:0/2822229094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b98146b20 0x7f4b980c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.983+0000 7f4b9e2e8640 1 -- 172.21.15.76:0/2822229094 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4b980af020 msgr2=0x7f4b9815d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.983+0000 7f4b9e2e8640 1 --2- 172.21.15.76:0/2822229094 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4b980af020 0x7f4b9815d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.983+0000 7f4b9e2e8640 1 -- 172.21.15.76:0/2822229094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b98146b20 msgr2=0x7f4b980c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:17.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.983+0000 7f4b9e2e8640 1 --2- 172.21.15.76:0/2822229094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b98146b20 0x7f4b980c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:17.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.983+0000 7f4b9e2e8640 1 -- 172.21.15.76:0/2822229094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b980c4eb0 con 0x7f4b98147760 2024-03-08T06:01:17.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.983+0000 7f4b9e2e8640 1 --2- 172.21.15.76:0/2822229094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b98147760 0x7f4b980c8860 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4b90034040 tx=0x7f4b900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:17.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.983+0000 7f4b967fc640 1 -- 172.21.15.76:0/2822229094 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90035040 con 0x7f4b98147760 2024-03-08T06:01:17.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.983+0000 7f4b967fc640 1 -- 172.21.15.76:0/2822229094 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b90004230 con 0x7f4b98147760 2024-03-08T06:01:17.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.983+0000 7f4b967fc640 1 -- 172.21.15.76:0/2822229094 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9003a640 con 0x7f4b98147760 2024-03-08T06:01:17.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.983+0000 7f4b9fd4b640 1 -- 172.21.15.76:0/2822229094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b980c5190 con 0x7f4b98147760 2024-03-08T06:01:17.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.983+0000 7f4b9fd4b640 1 -- 172.21.15.76:0/2822229094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b980c5750 con 0x7f4b98147760 2024-03-08T06:01:17.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.984+0000 7f4b9fd4b640 1 -- 172.21.15.76:0/2822229094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9804f030 con 0x7f4b98147760 2024-03-08T06:01:17.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.985+0000 7f4b967fc640 1 -- 172.21.15.76:0/2822229094 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b90038020 con 0x7f4b98147760 2024-03-08T06:01:17.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.985+0000 7f4b967fc640 1 --2- 172.21.15.76:0/2822229094 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4b8406b4d0 0x7f4b8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:17.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.985+0000 7f4b9dae7640 1 --2- 172.21.15.76:0/2822229094 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4b8406b4d0 0x7f4b8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:17.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.985+0000 7f4b967fc640 1 -- 172.21.15.76:0/2822229094 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f4b900ac150 con 0x7f4b98147760 2024-03-08T06:01:17.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.986+0000 7f4b9dae7640 1 --2- 172.21.15.76:0/2822229094 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4b8406b4d0 0x7f4b8406d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4b880072c0 tx=0x7f4b8800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:17.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:17.990+0000 7f4b967fc640 1 -- 172.21.15.76:0/2822229094 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b9007a580 con 0x7f4b98147760 2024-03-08T06:01:18.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.159+0000 7f4b9fd4b640 1 -- 172.21.15.76:0/2822229094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b9815dee0 con 0x7f4b98147760 2024-03-08T06:01:18.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.160+0000 7f4b967fc640 1 -- 172.21.15.76:0/2822229094 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18757 (secure 0 0 0) 0x7f4b9007e620 con 0x7f4b98147760 2024-03-08T06:01:18.161 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:18.161 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":28,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:01:17.313272+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:18.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.162+0000 7f4b9fd4b640 1 -- 172.21.15.76:0/2822229094 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4b8406b4d0 msgr2=0x7f4b8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.162+0000 7f4b9fd4b640 1 --2- 172.21.15.76:0/2822229094 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4b8406b4d0 0x7f4b8406d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4b880072c0 tx=0x7f4b8800b040 comp rx=0 tx=0).stop 2024-03-08T06:01:18.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.162+0000 7f4b9fd4b640 1 -- 172.21.15.76:0/2822229094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b98147760 msgr2=0x7f4b980c8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.162+0000 7f4b9fd4b640 1 --2- 172.21.15.76:0/2822229094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b98147760 0x7f4b980c8860 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4b90034040 tx=0x7f4b900046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.162+0000 7f4b9fd4b640 1 -- 172.21.15.76:0/2822229094 shutdown_connections 2024-03-08T06:01:18.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.162+0000 7f4b9fd4b640 1 --2- 172.21.15.76:0/2822229094 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4b8406b4d0 0x7f4b8406d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.162+0000 7f4b9fd4b640 1 --2- 172.21.15.76:0/2822229094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b98147760 0x7f4b980c8860 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.162+0000 7f4b9fd4b640 1 --2- 172.21.15.76:0/2822229094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b98146b20 0x7f4b980c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.162+0000 7f4b9fd4b640 1 --2- 172.21.15.76:0/2822229094 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4b980af020 0x7f4b9815d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.162+0000 7f4b9fd4b640 1 -- 172.21.15.76:0/2822229094 >> 172.21.15.76:0/2822229094 conn(0x7f4b980b1aa0 msgr2=0x7f4b980afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:18.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.162+0000 7f4b9fd4b640 1 -- 172.21.15.76:0/2822229094 shutdown_connections 2024-03-08T06:01:18.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.162+0000 7f4b9fd4b640 1 -- 172.21.15.76:0/2822229094 wait complete. 2024-03-08T06:01:18.173 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:00:33.926651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:00:34.474030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T06:00:38.207853+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T06:01:18.174 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:01:18.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.276+0000 7f79cd678640 1 Processor -- start 2024-03-08T06:01:18.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.276+0000 7f79cd678640 1 -- start start 2024-03-08T06:01:18.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.277+0000 7f79cd678640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f79c8154df0 0x7f79c80b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:18.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.277+0000 7f79cd678640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79c80cc7e0 0x7f79c815af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:18.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.277+0000 7f79cd678640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c815f450 con 0x7f79c80b3640 2024-03-08T06:01:18.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.277+0000 7f79cd678640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c80af1e0 con 0x7f79c8154df0 2024-03-08T06:01:18.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.277+0000 7f79cd678640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c80b3c80 con 0x7f79c80cc7e0 2024-03-08T06:01:18.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.277+0000 7f79c6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f79c8154df0 0x7f79c80b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:18.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.277+0000 7f79c6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f79c8154df0 0x7f79c80b3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55416/0 (socket says 172.21.15.76:55416) 2024-03-08T06:01:18.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.277+0000 7f79c6ffd640 1 -- 172.21.15.76:0/1590989478 learned_addr learned my addr 172.21.15.76:0/1590989478 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:18.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.277+0000 7f79c77fe640 1 --2- 172.21.15.76:0/1590989478 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79c80cc7e0 0x7f79c815af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:18.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.278+0000 7f79c6ffd640 1 -- 172.21.15.76:0/1590989478 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79c80cc7e0 msgr2=0x7f79c815af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:18.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.278+0000 7f79c6ffd640 1 --2- 172.21.15.76:0/1590989478 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79c80cc7e0 0x7f79c815af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.278+0000 7f79c6ffd640 1 -- 172.21.15.76:0/1590989478 >> v1:172.21.15.81:6789/0 conn(0x7f79c80b3640 legacy=0x7f79c80cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.278+0000 7f79c6ffd640 1 -- 172.21.15.76:0/1590989478 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c815e690 con 0x7f79c8154df0 2024-03-08T06:01:18.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.278+0000 7f79c6ffd640 1 --2- 172.21.15.76:0/1590989478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c8154df0 0x7f79c80b3100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009870 tx=0x7f79b802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ce2589ead035c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:18.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.278+0000 7f79c67fc640 1 -- 172.21.15.76:0/1590989478 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b8035020 con 0x7f79c8154df0 2024-03-08T06:01:18.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.278+0000 7f79c67fc640 1 -- 172.21.15.76:0/1590989478 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b8031640 con 0x7f79c8154df0 2024-03-08T06:01:18.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.278+0000 7f79c67fc640 1 -- 172.21.15.76:0/1590989478 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b802d040 con 0x7f79c8154df0 2024-03-08T06:01:18.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.279+0000 7f79cd678640 1 -- 172.21.15.76:0/1590989478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c8154df0 msgr2=0x7f79c80b3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.279+0000 7f79cd678640 1 --2- 172.21.15.76:0/1590989478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c8154df0 0x7f79c80b3100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009870 tx=0x7f79b802b8f0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.279+0000 7f79cd678640 1 -- 172.21.15.76:0/1590989478 shutdown_connections 2024-03-08T06:01:18.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.279+0000 7f79cd678640 1 --2- 172.21.15.76:0/1590989478 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79c80cc7e0 0x7f79c815af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.279+0000 7f79cd678640 1 --2- 172.21.15.76:0/1590989478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c8154df0 0x7f79c80b3100 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.279+0000 7f79cd678640 1 -- 172.21.15.76:0/1590989478 >> 172.21.15.76:0/1590989478 conn(0x7f79c806ac80 msgr2=0x7f79c806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:18.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.279+0000 7f79cd678640 1 -- 172.21.15.76:0/1590989478 shutdown_connections 2024-03-08T06:01:18.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.279+0000 7f79cd678640 1 -- 172.21.15.76:0/1590989478 wait complete. 2024-03-08T06:01:18.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.280+0000 7f79cd678640 1 Processor -- start 2024-03-08T06:01:18.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.280+0000 7f79cd678640 1 -- start start 2024-03-08T06:01:18.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79cd678640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79c80b3640 0x7f79c80abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:18.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79cd678640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c80cc7e0 0x7f79c81512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:18.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79cd678640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79c8154df0 0x7f79c80ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:18.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79cd678640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c815ef30 con 0x7f79c80cc7e0 2024-03-08T06:01:18.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79cd678640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c815ec50 con 0x7f79c8154df0 2024-03-08T06:01:18.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79cd678640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c81551d0 con 0x7f79c80b3640 2024-03-08T06:01:18.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79bffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c80cc7e0 0x7f79c81512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:18.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79bffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c80cc7e0 0x7f79c81512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55422/0 (socket says 172.21.15.76:55422) 2024-03-08T06:01:18.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79bffff640 1 -- 172.21.15.76:0/1335492348 learned_addr learned my addr 172.21.15.76:0/1335492348 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:18.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79c6ffd640 1 --2- 172.21.15.76:0/1335492348 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79c80b3640 0x7f79c80abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:18.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79c77fe640 1 --2- 172.21.15.76:0/1335492348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79c8154df0 0x7f79c80ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:18.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79bffff640 1 -- 172.21.15.76:0/1335492348 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79c80b3640 msgr2=0x7f79c80abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79bffff640 1 --2- 172.21.15.76:0/1335492348 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79c80b3640 0x7f79c80abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79bffff640 1 -- 172.21.15.76:0/1335492348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79c8154df0 msgr2=0x7f79c80ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79bffff640 1 --2- 172.21.15.76:0/1335492348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79c8154df0 0x7f79c80ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79bffff640 1 -- 172.21.15.76:0/1335492348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c80aad50 con 0x7f79c80cc7e0 2024-03-08T06:01:18.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79c77fe640 1 --2- 172.21.15.76:0/1335492348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79c8154df0 0x7f79c80ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:18.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.281+0000 7f79bffff640 1 --2- 172.21.15.76:0/1335492348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c80cc7e0 0x7f79c81512f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f79b000c400 tx=0x7f79b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:18.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.282+0000 7f79bf7fe640 1 -- 172.21.15.76:0/1335492348 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b00052d0 con 0x7f79c80cc7e0 2024-03-08T06:01:18.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.282+0000 7f79bf7fe640 1 -- 172.21.15.76:0/1335492348 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b0005490 con 0x7f79c80cc7e0 2024-03-08T06:01:18.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.282+0000 7f79cd678640 1 -- 172.21.15.76:0/1335492348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c80ab030 con 0x7f79c80cc7e0 2024-03-08T06:01:18.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.282+0000 7f79bf7fe640 1 -- 172.21.15.76:0/1335492348 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0007260 con 0x7f79c80cc7e0 2024-03-08T06:01:18.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.282+0000 7f79cd678640 1 -- 172.21.15.76:0/1335492348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c80ab5f0 con 0x7f79c80cc7e0 2024-03-08T06:01:18.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.283+0000 7f79cd678640 1 -- 172.21.15.76:0/1335492348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c80abae0 con 0x7f79c80cc7e0 2024-03-08T06:01:18.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.283+0000 7f79bf7fe640 1 -- 172.21.15.76:0/1335492348 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79b0007400 con 0x7f79c80cc7e0 2024-03-08T06:01:18.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.283+0000 7f79bf7fe640 1 --2- 172.21.15.76:0/1335492348 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f79a806b4d0 0x7f79a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:18.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.283+0000 7f79bf7fe640 1 -- 172.21.15.76:0/1335492348 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f79b0089a10 con 0x7f79c80cc7e0 2024-03-08T06:01:18.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.283+0000 7f79c6ffd640 1 --2- 172.21.15.76:0/1335492348 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f79a806b4d0 0x7f79a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:18.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.284+0000 7f79c6ffd640 1 --2- 172.21.15.76:0/1335492348 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f79a806b4d0 0x7f79a806d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f79b8036000 tx=0x7f79b803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:18.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.288+0000 7f79bf7fe640 1 -- 172.21.15.76:0/1335492348 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79b0057ed0 con 0x7f79c80cc7e0 2024-03-08T06:01:18.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.459+0000 7f79cd678640 1 -- 172.21.15.76:0/1335492348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79c804f030 con 0x7f79c80cc7e0 2024-03-08T06:01:18.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.459+0000 7f79bf7fe640 1 -- 172.21.15.76:0/1335492348 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79b005bf70 con 0x7f79c80cc7e0 2024-03-08T06:01:18.460 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:01:18.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.461+0000 7f79cd678640 1 -- 172.21.15.76:0/1335492348 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f79a806b4d0 msgr2=0x7f79a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.461+0000 7f79cd678640 1 --2- 172.21.15.76:0/1335492348 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f79a806b4d0 0x7f79a806d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f79b8036000 tx=0x7f79b803c040 comp rx=0 tx=0).stop 2024-03-08T06:01:18.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.461+0000 7f79cd678640 1 -- 172.21.15.76:0/1335492348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c80cc7e0 msgr2=0x7f79c81512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.461+0000 7f79cd678640 1 --2- 172.21.15.76:0/1335492348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c80cc7e0 0x7f79c81512f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f79b000c400 tx=0x7f79b000c8c0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.462+0000 7f79cd678640 1 -- 172.21.15.76:0/1335492348 shutdown_connections 2024-03-08T06:01:18.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.462+0000 7f79cd678640 1 --2- 172.21.15.76:0/1335492348 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f79a806b4d0 0x7f79a806d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.462+0000 7f79cd678640 1 --2- 172.21.15.76:0/1335492348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79c8154df0 0x7f79c80ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.462+0000 7f79cd678640 1 --2- 172.21.15.76:0/1335492348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c80cc7e0 0x7f79c81512f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.462+0000 7f79cd678640 1 --2- 172.21.15.76:0/1335492348 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79c80b3640 0x7f79c80abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.462+0000 7f79cd678640 1 -- 172.21.15.76:0/1335492348 >> 172.21.15.76:0/1335492348 conn(0x7f79c806ac80 msgr2=0x7f79c8153340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:18.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.462+0000 7f79cd678640 1 -- 172.21.15.76:0/1335492348 shutdown_connections 2024-03-08T06:01:18.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.462+0000 7f79cd678640 1 -- 172.21.15.76:0/1335492348 wait complete. 2024-03-08T06:01:18.473 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:01:18.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.575+0000 7fb979fb6640 1 Processor -- start 2024-03-08T06:01:18.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.576+0000 7fb979fb6640 1 -- start start 2024-03-08T06:01:18.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.576+0000 7fb979fb6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb97415dbc0 0x7fb9740d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:18.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.576+0000 7fb979fb6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9740d27d0 con 0x7fb9740b9840 2024-03-08T06:01:18.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.576+0000 7fb979fb6640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9740d2ab0 con 0x7fb9740b5cf0 2024-03-08T06:01:18.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.576+0000 7fb979fb6640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9740d2d90 con 0x7fb97415dbc0 2024-03-08T06:01:18.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.576+0000 7fb9737fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb97415dbc0 0x7fb9740d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:18.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.577+0000 7fb972ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb9740b9840 0x7fb9740b55e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39418/0 (socket says 172.21.15.76:39418) 2024-03-08T06:01:18.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.577+0000 7fb972ffd640 1 -- 172.21.15.76:0/300701147 learned_addr learned my addr 172.21.15.76:0/300701147 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:18.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.577+0000 7fb9737fe640 1 -- 172.21.15.76:0/300701147 >> v1:172.21.15.81:6789/0 conn(0x7fb9740b5cf0 legacy=0x7fb9740b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:18.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.577+0000 7fb9737fe640 1 -- 172.21.15.76:0/300701147 >> v1:172.21.15.76:6789/0 conn(0x7fb9740b9840 legacy=0x7fb9740b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.577+0000 7fb9737fe640 1 -- 172.21.15.76:0/300701147 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9740d3070 con 0x7fb97415dbc0 2024-03-08T06:01:18.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.577+0000 7fb9727fc640 1 -- 172.21.15.76:0/300701147 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675347020 0 0) 0x7fb9740d27d0 con 0x7fb9740b9840 2024-03-08T06:01:18.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.577+0000 7fb9737fe640 1 --2- 172.21.15.76:0/300701147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb97415dbc0 0x7fb9740d1ef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb9640097a0 tx=0x7fb96402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385b325e7523a7ec server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:18.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.578+0000 7fb9727fc640 1 -- 172.21.15.76:0/300701147 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964035020 con 0x7fb97415dbc0 2024-03-08T06:01:18.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.578+0000 7fb9727fc640 1 -- 172.21.15.76:0/300701147 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb964031830 con 0x7fb97415dbc0 2024-03-08T06:01:18.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.578+0000 7fb9727fc640 1 -- 172.21.15.76:0/300701147 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964031c00 con 0x7fb97415dbc0 2024-03-08T06:01:18.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.578+0000 7fb979fb6640 1 -- 172.21.15.76:0/300701147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb97415dbc0 msgr2=0x7fb9740d1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.578+0000 7fb979fb6640 1 --2- 172.21.15.76:0/300701147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb97415dbc0 0x7fb9740d1ef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb9640097a0 tx=0x7fb96402b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.578+0000 7fb979fb6640 1 -- 172.21.15.76:0/300701147 shutdown_connections 2024-03-08T06:01:18.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.578+0000 7fb979fb6640 1 --2- 172.21.15.76:0/300701147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb97415dbc0 0x7fb9740d1ef0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.578+0000 7fb979fb6640 1 -- 172.21.15.76:0/300701147 >> 172.21.15.76:0/300701147 conn(0x7fb9740670d0 msgr2=0x7fb9740d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:18.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.578+0000 7fb979fb6640 1 -- 172.21.15.76:0/300701147 shutdown_connections 2024-03-08T06:01:18.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.578+0000 7fb979fb6640 1 -- 172.21.15.76:0/300701147 wait complete. 2024-03-08T06:01:18.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.579+0000 7fb979fb6640 1 Processor -- start 2024-03-08T06:01:18.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.580+0000 7fb979fb6640 1 -- start start 2024-03-08T06:01:18.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.580+0000 7fb979fb6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9740b5cf0 0x7fb9741f3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:18.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.580+0000 7fb979fb6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9740b9840 0x7fb974158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:18.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.580+0000 7fb979fb6640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb97415dbc0 0x7fb9740d3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:18.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.580+0000 7fb979fb6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9740d25b0 con 0x7fb9740b5cf0 2024-03-08T06:01:18.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.580+0000 7fb979fb6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9740d2430 con 0x7fb9740b9840 2024-03-08T06:01:18.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.580+0000 7fb979fb6640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97415dfa0 con 0x7fb97415dbc0 2024-03-08T06:01:18.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.580+0000 7fb9737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9740b5cf0 0x7fb9741f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:18.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.580+0000 7fb9737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9740b5cf0 0x7fb9741f3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41558/0 (socket says 172.21.15.76:41558) 2024-03-08T06:01:18.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.580+0000 7fb9737fe640 1 -- 172.21.15.76:0/3953717889 learned_addr learned my addr 172.21.15.76:0/3953717889 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:18.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.581+0000 7fb9737fe640 1 -- 172.21.15.76:0/3953717889 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb97415dbc0 msgr2=0x7fb9740d3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.581+0000 7fb9737fe640 1 --2- 172.21.15.76:0/3953717889 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb97415dbc0 0x7fb9740d3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.581+0000 7fb9737fe640 1 -- 172.21.15.76:0/3953717889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9740b9840 msgr2=0x7fb974158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.581+0000 7fb9737fe640 1 --2- 172.21.15.76:0/3953717889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9740b9840 0x7fb974158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.581+0000 7fb9737fe640 1 -- 172.21.15.76:0/3953717889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb974157e30 con 0x7fb9740b5cf0 2024-03-08T06:01:18.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.581+0000 7fb9737fe640 1 --2- 172.21.15.76:0/3953717889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9740b5cf0 0x7fb9741f3020 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb9640097a0 tx=0x7fb9640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:18.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.581+0000 7fb95bfff640 1 -- 172.21.15.76:0/3953717889 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96403aa70 con 0x7fb9740b5cf0 2024-03-08T06:01:18.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.581+0000 7fb95bfff640 1 -- 172.21.15.76:0/3953717889 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96403ac10 con 0x7fb9740b5cf0 2024-03-08T06:01:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.581+0000 7fb979fb6640 1 -- 172.21.15.76:0/3953717889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb974158110 con 0x7fb9740b5cf0 2024-03-08T06:01:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.581+0000 7fb95bfff640 1 -- 172.21.15.76:0/3953717889 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9640443f0 con 0x7fb9740b5cf0 2024-03-08T06:01:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.581+0000 7fb979fb6640 1 -- 172.21.15.76:0/3953717889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9741586d0 con 0x7fb9740b5cf0 2024-03-08T06:01:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.582+0000 7fb979fb6640 1 -- 172.21.15.76:0/3953717889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97404f030 con 0x7fb9740b5cf0 2024-03-08T06:01:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.582+0000 7fb95bfff640 1 -- 172.21.15.76:0/3953717889 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb96404b060 con 0x7fb9740b5cf0 2024-03-08T06:01:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.583+0000 7fb95bfff640 1 --2- 172.21.15.76:0/3953717889 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb96006b4d0 0x7fb96006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:18.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.583+0000 7fb95bfff640 1 -- 172.21.15.76:0/3953717889 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb9640ac360 con 0x7fb9740b5cf0 2024-03-08T06:01:18.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.583+0000 7fb972ffd640 1 --2- 172.21.15.76:0/3953717889 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb96006b4d0 0x7fb96006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:18.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.583+0000 7fb972ffd640 1 --2- 172.21.15.76:0/3953717889 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb96006b4d0 0x7fb96006d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb968007fd0 tx=0x7fb968007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:18.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.587+0000 7fb95bfff640 1 -- 172.21.15.76:0/3953717889 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb96403f070 con 0x7fb9740b5cf0 2024-03-08T06:01:18.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.756+0000 7fb979fb6640 1 -- 172.21.15.76:0/3953717889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb97406b2b0 con 0x7fb9740b5cf0 2024-03-08T06:01:18.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.756+0000 7fb95bfff640 1 -- 172.21.15.76:0/3953717889 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb96402f2e0 con 0x7fb9740b5cf0 2024-03-08T06:01:18.757 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:18.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.758+0000 7fb979fb6640 1 -- 172.21.15.76:0/3953717889 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb96006b4d0 msgr2=0x7fb96006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.758+0000 7fb979fb6640 1 --2- 172.21.15.76:0/3953717889 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb96006b4d0 0x7fb96006d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb968007fd0 tx=0x7fb968007920 comp rx=0 tx=0).stop 2024-03-08T06:01:18.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.758+0000 7fb979fb6640 1 -- 172.21.15.76:0/3953717889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9740b5cf0 msgr2=0x7fb9741f3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.758+0000 7fb979fb6640 1 --2- 172.21.15.76:0/3953717889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9740b5cf0 0x7fb9741f3020 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb9640097a0 tx=0x7fb9640046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.758+0000 7fb979fb6640 1 -- 172.21.15.76:0/3953717889 shutdown_connections 2024-03-08T06:01:18.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.758+0000 7fb979fb6640 1 --2- 172.21.15.76:0/3953717889 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb96006b4d0 0x7fb96006d990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.758+0000 7fb979fb6640 1 --2- 172.21.15.76:0/3953717889 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb97415dbc0 0x7fb9740d3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.758+0000 7fb979fb6640 1 --2- 172.21.15.76:0/3953717889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9740b9840 0x7fb974158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.758+0000 7fb979fb6640 1 --2- 172.21.15.76:0/3953717889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9740b5cf0 0x7fb9741f3020 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.758+0000 7fb979fb6640 1 -- 172.21.15.76:0/3953717889 >> 172.21.15.76:0/3953717889 conn(0x7fb9740670d0 msgr2=0x7fb97415a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:18.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.758+0000 7fb979fb6640 1 -- 172.21.15.76:0/3953717889 shutdown_connections 2024-03-08T06:01:18.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.758+0000 7fb979fb6640 1 -- 172.21.15.76:0/3953717889 wait complete. 2024-03-08T06:01:18.769 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:01:18.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.872+0000 7fd9fa363640 1 Processor -- start 2024-03-08T06:01:18.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.872+0000 7fd9fa363640 1 -- start start 2024-03-08T06:01:18.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.873+0000 7fd9fa363640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd9f40cc0d0 0x7fd9f40cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:18.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.873+0000 7fd9fa363640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f415f450 con 0x7fd9f4154df0 2024-03-08T06:01:18.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.873+0000 7fd9fa363640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f415e690 con 0x7fd9f415af70 2024-03-08T06:01:18.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.873+0000 7fd9fa363640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9f40af1e0 con 0x7fd9f40cc0d0 2024-03-08T06:01:18.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.873+0000 7fd9f37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd9f40cc0d0 0x7fd9f40cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:18.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.873+0000 7fd9f3fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd9f4154df0 0x7fd9f40b3100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39422/0 (socket says 172.21.15.76:39422) 2024-03-08T06:01:18.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.873+0000 7fd9f37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd9f40cc0d0 0x7fd9f40cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41562/0 (socket says 172.21.15.76:41562) 2024-03-08T06:01:18.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.873+0000 7fd9f3fff640 1 -- 172.21.15.76:0/2577062765 learned_addr learned my addr 172.21.15.76:0/2577062765 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:18.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.874+0000 7fd9f37fe640 1 -- 172.21.15.76:0/2577062765 >> v1:172.21.15.81:6789/0 conn(0x7fd9f415af70 legacy=0x7fd9f415b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:18.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.874+0000 7fd9f37fe640 1 -- 172.21.15.76:0/2577062765 >> v1:172.21.15.76:6789/0 conn(0x7fd9f4154df0 legacy=0x7fd9f40b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.874+0000 7fd9f2ffd640 1 -- 172.21.15.76:0/2577062765 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030088840 0 0) 0x7fd9f415f450 con 0x7fd9f4154df0 2024-03-08T06:01:18.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.874+0000 7fd9f37fe640 1 -- 172.21.15.76:0/2577062765 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f415e970 con 0x7fd9f40cc0d0 2024-03-08T06:01:18.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.874+0000 7fd9f37fe640 1 --2- 172.21.15.76:0/2577062765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9f40cc0d0 0x7fd9f40cc580 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800ba40 tx=0x7fd9e802e010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3cf56aef351299 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:18.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.875+0000 7fd9f2ffd640 1 -- 172.21.15.76:0/2577062765 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e803b070 con 0x7fd9f40cc0d0 2024-03-08T06:01:18.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.875+0000 7fd9f2ffd640 1 -- 172.21.15.76:0/2577062765 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e802edc0 con 0x7fd9f40cc0d0 2024-03-08T06:01:18.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.875+0000 7fd9f2ffd640 1 -- 172.21.15.76:0/2577062765 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8036660 con 0x7fd9f40cc0d0 2024-03-08T06:01:18.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.875+0000 7fd9fa363640 1 -- 172.21.15.76:0/2577062765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9f40cc0d0 msgr2=0x7fd9f40cc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.875+0000 7fd9fa363640 1 --2- 172.21.15.76:0/2577062765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9f40cc0d0 0x7fd9f40cc580 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800ba40 tx=0x7fd9e802e010 comp rx=0 tx=0).stop 2024-03-08T06:01:18.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.875+0000 7fd9fa363640 1 -- 172.21.15.76:0/2577062765 shutdown_connections 2024-03-08T06:01:18.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.875+0000 7fd9fa363640 1 --2- 172.21.15.76:0/2577062765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9f40cc0d0 0x7fd9f40cc580 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.875+0000 7fd9fa363640 1 -- 172.21.15.76:0/2577062765 >> 172.21.15.76:0/2577062765 conn(0x7fd9f406ac80 msgr2=0x7fd9f406b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:18.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.875+0000 7fd9fa363640 1 -- 172.21.15.76:0/2577062765 shutdown_connections 2024-03-08T06:01:18.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.875+0000 7fd9fa363640 1 -- 172.21.15.76:0/2577062765 wait complete. 2024-03-08T06:01:18.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.876+0000 7fd9fa363640 1 Processor -- start 2024-03-08T06:01:18.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.876+0000 7fd9fa363640 1 -- start start 2024-03-08T06:01:18.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.877+0000 7fd9fa363640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd9f40cc0d0 0x7fd9f4149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:18.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.877+0000 7fd9fa363640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9f4154df0 0x7fd9f4151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:18.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.877+0000 7fd9fa363640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9f415af70 0x7fd9f4152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:18.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.877+0000 7fd9fa363640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f415ef30 con 0x7fd9f4154df0 2024-03-08T06:01:18.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.877+0000 7fd9fa363640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f40b3c80 con 0x7fd9f415af70 2024-03-08T06:01:18.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.877+0000 7fd9fa363640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f40ceb60 con 0x7fd9f40cc0d0 2024-03-08T06:01:18.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.877+0000 7fd9f37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9f4154df0 0x7fd9f4151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:18.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.877+0000 7fd9f37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9f4154df0 0x7fd9f4151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41572/0 (socket says 172.21.15.76:41572) 2024-03-08T06:01:18.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.877+0000 7fd9f37fe640 1 -- 172.21.15.76:0/662702670 learned_addr learned my addr 172.21.15.76:0/662702670 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:18.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.877+0000 7fd9f8900640 1 --2- 172.21.15.76:0/662702670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9f415af70 0x7fd9f4152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:18.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.877+0000 7fd9f3fff640 1 --2- 172.21.15.76:0/662702670 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd9f40cc0d0 0x7fd9f4149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:18.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.878+0000 7fd9f37fe640 1 -- 172.21.15.76:0/662702670 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd9f40cc0d0 msgr2=0x7fd9f4149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.878+0000 7fd9f37fe640 1 --2- 172.21.15.76:0/662702670 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd9f40cc0d0 0x7fd9f4149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.878+0000 7fd9f37fe640 1 -- 172.21.15.76:0/662702670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9f415af70 msgr2=0x7fd9f4152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:18.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.878+0000 7fd9f37fe640 1 --2- 172.21.15.76:0/662702670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9f415af70 0x7fd9f4152200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:18.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.878+0000 7fd9f37fe640 1 -- 172.21.15.76:0/662702670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f414a510 con 0x7fd9f4154df0 2024-03-08T06:01:18.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.878+0000 7fd9f37fe640 1 --2- 172.21.15.76:0/662702670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9f4154df0 0x7fd9f4151cc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8039040 tx=0x7fd9e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:18.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.878+0000 7fd9f0ff9640 1 -- 172.21.15.76:0/662702670 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8046070 con 0x7fd9f4154df0 2024-03-08T06:01:18.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.878+0000 7fd9f0ff9640 1 -- 172.21.15.76:0/662702670 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e8041650 con 0x7fd9f4154df0 2024-03-08T06:01:18.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.878+0000 7fd9fa363640 1 -- 172.21.15.76:0/662702670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f40a8680 con 0x7fd9f4154df0 2024-03-08T06:01:18.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.878+0000 7fd9f0ff9640 1 -- 172.21.15.76:0/662702670 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e803b040 con 0x7fd9f4154df0 2024-03-08T06:01:18.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.878+0000 7fd9fa363640 1 -- 172.21.15.76:0/662702670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f40a8c40 con 0x7fd9f4154df0 2024-03-08T06:01:18.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.879+0000 7fd9fa363640 1 -- 172.21.15.76:0/662702670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f404f030 con 0x7fd9f4154df0 2024-03-08T06:01:18.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.879+0000 7fd9f0ff9640 1 -- 172.21.15.76:0/662702670 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9e8049070 con 0x7fd9f4154df0 2024-03-08T06:01:18.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.880+0000 7fd9f0ff9640 1 --2- 172.21.15.76:0/662702670 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd9e006b4d0 0x7fd9e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:18.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.880+0000 7fd9f3fff640 1 --2- 172.21.15.76:0/662702670 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd9e006b4d0 0x7fd9e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:18.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.880+0000 7fd9f0ff9640 1 -- 172.21.15.76:0/662702670 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd9e80ac020 con 0x7fd9f4154df0 2024-03-08T06:01:18.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.880+0000 7fd9f3fff640 1 --2- 172.21.15.76:0/662702670 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd9e006b4d0 0x7fd9e006d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4000f60 tx=0x7fd9e4007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:18.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:18.884+0000 7fd9f0ff9640 1 -- 172.21.15.76:0/662702670 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9e807a4e0 con 0x7fd9f4154df0 2024-03-08T06:01:19.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.054+0000 7fd9fa363640 1 -- 172.21.15.76:0/662702670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd9f4153cd0 con 0x7fd9f4154df0 2024-03-08T06:01:19.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.054+0000 7fd9f0ff9640 1 -- 172.21.15.76:0/662702670 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd9e807e580 con 0x7fd9f4154df0 2024-03-08T06:01:19.055 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:01:19.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.056+0000 7fd9fa363640 1 -- 172.21.15.76:0/662702670 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd9e006b4d0 msgr2=0x7fd9e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.056+0000 7fd9fa363640 1 --2- 172.21.15.76:0/662702670 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd9e006b4d0 0x7fd9e006d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4000f60 tx=0x7fd9e4007d50 comp rx=0 tx=0).stop 2024-03-08T06:01:19.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.056+0000 7fd9fa363640 1 -- 172.21.15.76:0/662702670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9f4154df0 msgr2=0x7fd9f4151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.056+0000 7fd9fa363640 1 --2- 172.21.15.76:0/662702670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9f4154df0 0x7fd9f4151cc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8039040 tx=0x7fd9e80046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.056+0000 7fd9fa363640 1 -- 172.21.15.76:0/662702670 shutdown_connections 2024-03-08T06:01:19.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.056+0000 7fd9fa363640 1 --2- 172.21.15.76:0/662702670 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd9e006b4d0 0x7fd9e006d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.056+0000 7fd9fa363640 1 --2- 172.21.15.76:0/662702670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9f415af70 0x7fd9f4152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.056+0000 7fd9fa363640 1 --2- 172.21.15.76:0/662702670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9f4154df0 0x7fd9f4151cc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.056+0000 7fd9fa363640 1 --2- 172.21.15.76:0/662702670 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd9f40cc0d0 0x7fd9f4149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.056+0000 7fd9fa363640 1 -- 172.21.15.76:0/662702670 >> 172.21.15.76:0/662702670 conn(0x7fd9f406ac80 msgr2=0x7fd9f40ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:19.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.056+0000 7fd9fa363640 1 -- 172.21.15.76:0/662702670 shutdown_connections 2024-03-08T06:01:19.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.056+0000 7fd9fa363640 1 -- 172.21.15.76:0/662702670 wait complete. 2024-03-08T06:01:19.067 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T06:01:19.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.170+0000 7ffa5862c640 1 Processor -- start 2024-03-08T06:01:19.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.170+0000 7ffa5862c640 1 -- start start 2024-03-08T06:01:19.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.171+0000 7ffa5862c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffa5015b260 0x7ffa500b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.171+0000 7ffa5862c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ffa500b2c50 0x7ffa5015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.171+0000 7ffa5862c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5015b820 con 0x7ffa500aee10 2024-03-08T06:01:19.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.171+0000 7ffa5862c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa500b3060 con 0x7ffa5015b260 2024-03-08T06:01:19.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.171+0000 7ffa5862c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa5015e490 con 0x7ffa500b2c50 2024-03-08T06:01:19.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.171+0000 7ffa55bc7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffa5015b260 0x7ffa500b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.171+0000 7ffa563c8640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ffa500aee10 0x7ffa5015dc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39424/0 (socket says 172.21.15.76:39424) 2024-03-08T06:01:19.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.171+0000 7ffa55bc7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffa5015b260 0x7ffa500b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41580/0 (socket says 172.21.15.76:41580) 2024-03-08T06:01:19.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.171+0000 7ffa563c8640 1 -- 172.21.15.76:0/133960565 learned_addr learned my addr 172.21.15.76:0/133960565 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:19.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.171+0000 7ffa56bc9640 1 --2- 172.21.15.76:0/133960565 >> v2:172.21.15.81:3300/0 conn(0x7ffa500b2c50 0x7ffa5015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.171+0000 7ffa55bc7640 1 -- 172.21.15.76:0/133960565 >> v2:172.21.15.81:3300/0 conn(0x7ffa500b2c50 msgr2=0x7ffa5015bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:19.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.171+0000 7ffa55bc7640 1 --2- 172.21.15.76:0/133960565 >> v2:172.21.15.81:3300/0 conn(0x7ffa500b2c50 0x7ffa5015bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.171+0000 7ffa55bc7640 1 -- 172.21.15.76:0/133960565 >> v1:172.21.15.76:6789/0 conn(0x7ffa500aee10 legacy=0x7ffa5015dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.171+0000 7ffa553c6640 1 -- 172.21.15.76:0/133960565 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704926278 0 0) 0x7ffa5015b820 con 0x7ffa500aee10 2024-03-08T06:01:19.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.171+0000 7ffa55bc7640 1 -- 172.21.15.76:0/133960565 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5015e770 con 0x7ffa5015b260 2024-03-08T06:01:19.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.172+0000 7ffa55bc7640 1 --2- 172.21.15.76:0/133960565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa5015b260 0x7ffa500b2710 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffa50154c30 tx=0x7ffa4002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534227d46b829ced server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:19.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.172+0000 7ffa553c6640 1 -- 172.21.15.76:0/133960565 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa400315a0 con 0x7ffa5015b260 2024-03-08T06:01:19.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.172+0000 7ffa553c6640 1 -- 172.21.15.76:0/133960565 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa40037070 con 0x7ffa5015b260 2024-03-08T06:01:19.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.172+0000 7ffa553c6640 1 -- 172.21.15.76:0/133960565 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa40031960 con 0x7ffa5015b260 2024-03-08T06:01:19.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.172+0000 7ffa5862c640 1 -- 172.21.15.76:0/133960565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa5015b260 msgr2=0x7ffa500b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.173+0000 7ffa5862c640 1 --2- 172.21.15.76:0/133960565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa5015b260 0x7ffa500b2710 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffa50154c30 tx=0x7ffa4002b840 comp rx=0 tx=0).stop 2024-03-08T06:01:19.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.173+0000 7ffa5862c640 1 -- 172.21.15.76:0/133960565 shutdown_connections 2024-03-08T06:01:19.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.173+0000 7ffa5862c640 1 --2- 172.21.15.76:0/133960565 >> v2:172.21.15.81:3300/0 conn(0x7ffa500b2c50 0x7ffa5015bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.173+0000 7ffa5862c640 1 --2- 172.21.15.76:0/133960565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa5015b260 0x7ffa500b2710 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.173+0000 7ffa5862c640 1 -- 172.21.15.76:0/133960565 >> 172.21.15.76:0/133960565 conn(0x7ffa5006ac80 msgr2=0x7ffa5006b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:19.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.173+0000 7ffa5862c640 1 -- 172.21.15.76:0/133960565 shutdown_connections 2024-03-08T06:01:19.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.173+0000 7ffa5862c640 1 -- 172.21.15.76:0/133960565 wait complete. 2024-03-08T06:01:19.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.174+0000 7ffa5862c640 1 Processor -- start 2024-03-08T06:01:19.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.174+0000 7ffa5862c640 1 -- start start 2024-03-08T06:01:19.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.174+0000 7ffa5862c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa500aee10 0x7ffa50164fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.174+0000 7ffa5862c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffa500b2c50 0x7ffa5013ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.174+0000 7ffa5862c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa5015b260 0x7ffa5013afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.174+0000 7ffa5862c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa500cc160 con 0x7ffa500aee10 2024-03-08T06:01:19.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.174+0000 7ffa5862c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5015ea50 con 0x7ffa5015b260 2024-03-08T06:01:19.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.174+0000 7ffa5862c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa500cc2e0 con 0x7ffa500b2c50 2024-03-08T06:01:19.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.175+0000 7ffa563c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa500aee10 0x7ffa50164fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.175+0000 7ffa563c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa500aee10 0x7ffa50164fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41588/0 (socket says 172.21.15.76:41588) 2024-03-08T06:01:19.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.175+0000 7ffa563c8640 1 -- 172.21.15.76:0/2314570319 learned_addr learned my addr 172.21.15.76:0/2314570319 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:19.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.175+0000 7ffa56bc9640 1 --2- 172.21.15.76:0/2314570319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa5015b260 0x7ffa5013afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.175+0000 7ffa55bc7640 1 --2- 172.21.15.76:0/2314570319 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffa500b2c50 0x7ffa5013ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.175+0000 7ffa563c8640 1 -- 172.21.15.76:0/2314570319 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffa500b2c50 msgr2=0x7ffa5013ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.175+0000 7ffa563c8640 1 --2- 172.21.15.76:0/2314570319 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffa500b2c50 0x7ffa5013ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.175+0000 7ffa563c8640 1 -- 172.21.15.76:0/2314570319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa5015b260 msgr2=0x7ffa5013afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.175+0000 7ffa563c8640 1 --2- 172.21.15.76:0/2314570319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa5015b260 0x7ffa5013afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.175+0000 7ffa563c8640 1 -- 172.21.15.76:0/2314570319 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa50165510 con 0x7ffa500aee10 2024-03-08T06:01:19.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.175+0000 7ffa563c8640 1 --2- 172.21.15.76:0/2314570319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa500aee10 0x7ffa50164fd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00c500 tx=0x7ffa4c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:19.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.175+0000 7ffa46ffd640 1 -- 172.21.15.76:0/2314570319 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c018070 con 0x7ffa500aee10 2024-03-08T06:01:19.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.175+0000 7ffa46ffd640 1 -- 172.21.15.76:0/2314570319 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4c007580 con 0x7ffa500aee10 2024-03-08T06:01:19.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.175+0000 7ffa5862c640 1 -- 172.21.15.76:0/2314570319 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa50165b30 con 0x7ffa500aee10 2024-03-08T06:01:19.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.175+0000 7ffa46ffd640 1 -- 172.21.15.76:0/2314570319 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c004a20 con 0x7ffa500aee10 2024-03-08T06:01:19.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.175+0000 7ffa5862c640 1 -- 172.21.15.76:0/2314570319 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5006e600 con 0x7ffa500aee10 2024-03-08T06:01:19.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.176+0000 7ffa5862c640 1 -- 172.21.15.76:0/2314570319 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5004f030 con 0x7ffa500aee10 2024-03-08T06:01:19.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.177+0000 7ffa46ffd640 1 -- 172.21.15.76:0/2314570319 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa4c01b070 con 0x7ffa500aee10 2024-03-08T06:01:19.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.177+0000 7ffa46ffd640 1 --2- 172.21.15.76:0/2314570319 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffa3406b4d0 0x7ffa3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.177+0000 7ffa46ffd640 1 -- 172.21.15.76:0/2314570319 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ffa4c0897d0 con 0x7ffa500aee10 2024-03-08T06:01:19.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.177+0000 7ffa55bc7640 1 --2- 172.21.15.76:0/2314570319 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffa3406b4d0 0x7ffa3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.177+0000 7ffa55bc7640 1 --2- 172.21.15.76:0/2314570319 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffa3406b4d0 0x7ffa3406d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa40036000 tx=0x7ffa40008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:19.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.181+0000 7ffa46ffd640 1 -- 172.21.15.76:0/2314570319 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa4c057c90 con 0x7ffa500aee10 2024-03-08T06:01:19.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.351+0000 7ffa5862c640 1 -- 172.21.15.76:0/2314570319 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ffa500cce40 con 0x7ffa500aee10 2024-03-08T06:01:19.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.352+0000 7ffa46ffd640 1 -- 172.21.15.76:0/2314570319 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7ffa4c05bd30 con 0x7ffa500aee10 2024-03-08T06:01:19.352 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:19.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.354+0000 7ffa5862c640 1 -- 172.21.15.76:0/2314570319 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffa3406b4d0 msgr2=0x7ffa3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.354+0000 7ffa5862c640 1 --2- 172.21.15.76:0/2314570319 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffa3406b4d0 0x7ffa3406d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa40036000 tx=0x7ffa40008030 comp rx=0 tx=0).stop 2024-03-08T06:01:19.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.354+0000 7ffa5862c640 1 -- 172.21.15.76:0/2314570319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa500aee10 msgr2=0x7ffa50164fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.354+0000 7ffa5862c640 1 --2- 172.21.15.76:0/2314570319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa500aee10 0x7ffa50164fd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00c500 tx=0x7ffa4c00c9c0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.354+0000 7ffa5862c640 1 -- 172.21.15.76:0/2314570319 shutdown_connections 2024-03-08T06:01:19.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.354+0000 7ffa5862c640 1 --2- 172.21.15.76:0/2314570319 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffa3406b4d0 0x7ffa3406d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.354+0000 7ffa5862c640 1 --2- 172.21.15.76:0/2314570319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa5015b260 0x7ffa5013afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.354+0000 7ffa5862c640 1 --2- 172.21.15.76:0/2314570319 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffa500b2c50 0x7ffa5013ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.354+0000 7ffa5862c640 1 --2- 172.21.15.76:0/2314570319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa500aee10 0x7ffa50164fd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.354+0000 7ffa5862c640 1 -- 172.21.15.76:0/2314570319 >> 172.21.15.76:0/2314570319 conn(0x7ffa5006ac80 msgr2=0x7ffa5006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:19.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.354+0000 7ffa5862c640 1 -- 172.21.15.76:0/2314570319 shutdown_connections 2024-03-08T06:01:19.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.354+0000 7ffa5862c640 1 -- 172.21.15.76:0/2314570319 wait complete. 2024-03-08T06:01:19.365 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:19.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.465+0000 7fb53df81640 1 Processor -- start 2024-03-08T06:01:19.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.465+0000 7fb53df81640 1 -- start start 2024-03-08T06:01:19.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.465+0000 7fb53df81640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb5380b2cf0 0x7fb53815c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.465+0000 7fb53df81640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb53815cd70 0x7fb5380cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.465+0000 7fb53df81640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5380cc4e0 con 0x7fb53806ac80 2024-03-08T06:01:19.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.465+0000 7fb53df81640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5380cd3b0 con 0x7fb5380b2cf0 2024-03-08T06:01:19.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.465+0000 7fb53df81640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5380cc7a0 con 0x7fb53815cd70 2024-03-08T06:01:19.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.466+0000 7fb52effd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb5380b2cf0 0x7fb53815c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.466+0000 7fb537fff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb53815cd70 0x7fb5380cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.466+0000 7fb5377fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb53806ac80 0x7fb53806b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:47794/0 (socket says 172.21.15.76:47794) 2024-03-08T06:01:19.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.466+0000 7fb5377fe640 1 -- 172.21.15.76:0/1908813885 learned_addr learned my addr 172.21.15.76:0/1908813885 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:19.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.466+0000 7fb536ffd640 1 -- 172.21.15.76:0/1908813885 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800772398 0 0) 0x7fb5380cc4e0 con 0x7fb53806ac80 2024-03-08T06:01:19.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.466+0000 7fb537fff640 1 -- 172.21.15.76:0/1908813885 >> v2:172.21.15.81:3300/0 conn(0x7fb5380b2cf0 msgr2=0x7fb53815c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:19.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.466+0000 7fb537fff640 1 --2- 172.21.15.76:0/1908813885 >> v2:172.21.15.81:3300/0 conn(0x7fb5380b2cf0 0x7fb53815c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.466+0000 7fb537fff640 1 -- 172.21.15.76:0/1908813885 >> v1:172.21.15.81:6789/0 conn(0x7fb53806ac80 legacy=0x7fb53806b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.466+0000 7fb537fff640 1 -- 172.21.15.76:0/1908813885 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5380cca80 con 0x7fb53815cd70 2024-03-08T06:01:19.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.466+0000 7fb52effd640 1 --2- 172.21.15.76:0/1908813885 >> v2:172.21.15.81:3300/0 conn(0x7fb5380b2cf0 0x7fb53815c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:19.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.467+0000 7fb537fff640 1 --2- 172.21.15.76:0/1908813885 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb53815cd70 0x7fb5380cd840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb530009d90 tx=0x7fb53002bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=722a8501c25c35ee server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:19.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.467+0000 7fb536ffd640 1 -- 172.21.15.76:0/1908813885 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530039070 con 0x7fb53815cd70 2024-03-08T06:01:19.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.467+0000 7fb536ffd640 1 -- 172.21.15.76:0/1908813885 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53002cce0 con 0x7fb53815cd70 2024-03-08T06:01:19.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.467+0000 7fb536ffd640 1 -- 172.21.15.76:0/1908813885 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530034660 con 0x7fb53815cd70 2024-03-08T06:01:19.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.467+0000 7fb53df81640 1 -- 172.21.15.76:0/1908813885 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb53815cd70 msgr2=0x7fb5380cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.467+0000 7fb53df81640 1 --2- 172.21.15.76:0/1908813885 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb53815cd70 0x7fb5380cd840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb530009d90 tx=0x7fb53002bf30 comp rx=0 tx=0).stop 2024-03-08T06:01:19.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.468+0000 7fb53df81640 1 -- 172.21.15.76:0/1908813885 shutdown_connections 2024-03-08T06:01:19.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.468+0000 7fb53df81640 1 --2- 172.21.15.76:0/1908813885 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb53815cd70 0x7fb5380cd840 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.468+0000 7fb53df81640 1 --2- 172.21.15.76:0/1908813885 >> v2:172.21.15.81:3300/0 conn(0x7fb5380b2cf0 0x7fb53815c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.468+0000 7fb53df81640 1 -- 172.21.15.76:0/1908813885 >> 172.21.15.76:0/1908813885 conn(0x7fb538099a60 msgr2=0x7fb5380b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:19.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.468+0000 7fb53df81640 1 -- 172.21.15.76:0/1908813885 shutdown_connections 2024-03-08T06:01:19.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.468+0000 7fb53df81640 1 -- 172.21.15.76:0/1908813885 wait complete. 2024-03-08T06:01:19.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.469+0000 7fb53df81640 1 Processor -- start 2024-03-08T06:01:19.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.469+0000 7fb53df81640 1 -- start start 2024-03-08T06:01:19.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb53df81640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb53806ac80 0x7fb53814b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb53df81640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5380b2cf0 0x7fb5380b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb53df81640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb53815cd70 0x7fb5380b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb53df81640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5380cdd80 con 0x7fb53815cd70 2024-03-08T06:01:19.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb53df81640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5380cd040 con 0x7fb5380b2cf0 2024-03-08T06:01:19.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb53df81640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5380cdf00 con 0x7fb53806ac80 2024-03-08T06:01:19.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb537fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb53815cd70 0x7fb5380b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb537fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb53815cd70 0x7fb5380b1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41594/0 (socket says 172.21.15.76:41594) 2024-03-08T06:01:19.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb537fff640 1 -- 172.21.15.76:0/2163167993 learned_addr learned my addr 172.21.15.76:0/2163167993 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:19.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb52effd640 1 --2- 172.21.15.76:0/2163167993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5380b2cf0 0x7fb5380b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb5377fe640 1 --2- 172.21.15.76:0/2163167993 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb53806ac80 0x7fb53814b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb537fff640 1 -- 172.21.15.76:0/2163167993 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb53806ac80 msgr2=0x7fb53814b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb537fff640 1 --2- 172.21.15.76:0/2163167993 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb53806ac80 0x7fb53814b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb537fff640 1 -- 172.21.15.76:0/2163167993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5380b2cf0 msgr2=0x7fb5380b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb537fff640 1 --2- 172.21.15.76:0/2163167993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5380b2cf0 0x7fb5380b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb537fff640 1 -- 172.21.15.76:0/2163167993 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5380c7ad0 con 0x7fb53815cd70 2024-03-08T06:01:19.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb537fff640 1 --2- 172.21.15.76:0/2163167993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb53815cd70 0x7fb5380b1660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb530037040 tx=0x7fb530004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:19.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb534ff9640 1 -- 172.21.15.76:0/2163167993 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530039050 con 0x7fb53815cd70 2024-03-08T06:01:19.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb534ff9640 1 -- 172.21.15.76:0/2163167993 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53003a040 con 0x7fb53815cd70 2024-03-08T06:01:19.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb534ff9640 1 -- 172.21.15.76:0/2163167993 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530009450 con 0x7fb53815cd70 2024-03-08T06:01:19.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb53df81640 1 -- 172.21.15.76:0/2163167993 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5380c7db0 con 0x7fb53815cd70 2024-03-08T06:01:19.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.470+0000 7fb53df81640 1 -- 172.21.15.76:0/2163167993 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5380c8370 con 0x7fb53815cd70 2024-03-08T06:01:19.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.471+0000 7fb53df81640 1 -- 172.21.15.76:0/2163167993 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53804f030 con 0x7fb53815cd70 2024-03-08T06:01:19.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.472+0000 7fb534ff9640 1 -- 172.21.15.76:0/2163167993 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb530047030 con 0x7fb53815cd70 2024-03-08T06:01:19.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.472+0000 7fb534ff9640 1 --2- 172.21.15.76:0/2163167993 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb51806b4d0 0x7fb51806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.472+0000 7fb534ff9640 1 -- 172.21.15.76:0/2163167993 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb5300ac690 con 0x7fb53815cd70 2024-03-08T06:01:19.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.472+0000 7fb5377fe640 1 --2- 172.21.15.76:0/2163167993 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb51806b4d0 0x7fb51806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.473+0000 7fb5377fe640 1 --2- 172.21.15.76:0/2163167993 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb51806b4d0 0x7fb51806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb528000f60 tx=0x7fb528007b70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:19.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.476+0000 7fb534ff9640 1 -- 172.21.15.76:0/2163167993 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb53004e060 con 0x7fb53815cd70 2024-03-08T06:01:19.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.647+0000 7fb53df81640 1 -- 172.21.15.76:0/2163167993 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb538073070 con 0x7fb53815cd70 2024-03-08T06:01:19.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.648+0000 7fb534ff9640 1 -- 172.21.15.76:0/2163167993 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18757 (secure 0 0 0) 0x7fb5300a9960 con 0x7fb53815cd70 2024-03-08T06:01:19.649 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:19.649 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":28,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:01:17.313272+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:19.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.651+0000 7fb53df81640 1 -- 172.21.15.76:0/2163167993 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb51806b4d0 msgr2=0x7fb51806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.651+0000 7fb53df81640 1 --2- 172.21.15.76:0/2163167993 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb51806b4d0 0x7fb51806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb528000f60 tx=0x7fb528007b70 comp rx=0 tx=0).stop 2024-03-08T06:01:19.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.651+0000 7fb53df81640 1 -- 172.21.15.76:0/2163167993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb53815cd70 msgr2=0x7fb5380b1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.651+0000 7fb53df81640 1 --2- 172.21.15.76:0/2163167993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb53815cd70 0x7fb5380b1660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb530037040 tx=0x7fb530004a40 comp rx=0 tx=0).stop 2024-03-08T06:01:19.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.651+0000 7fb53df81640 1 -- 172.21.15.76:0/2163167993 shutdown_connections 2024-03-08T06:01:19.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.651+0000 7fb53df81640 1 --2- 172.21.15.76:0/2163167993 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb51806b4d0 0x7fb51806d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.651+0000 7fb53df81640 1 --2- 172.21.15.76:0/2163167993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb53815cd70 0x7fb5380b1660 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.651+0000 7fb53df81640 1 --2- 172.21.15.76:0/2163167993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5380b2cf0 0x7fb5380b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.651+0000 7fb53df81640 1 --2- 172.21.15.76:0/2163167993 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb53806ac80 0x7fb53814b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.651+0000 7fb53df81640 1 -- 172.21.15.76:0/2163167993 >> 172.21.15.76:0/2163167993 conn(0x7fb538099a60 msgr2=0x7fb53815b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:19.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.651+0000 7fb53df81640 1 -- 172.21.15.76:0/2163167993 shutdown_connections 2024-03-08T06:01:19.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.651+0000 7fb53df81640 1 -- 172.21.15.76:0/2163167993 wait complete. 2024-03-08T06:01:19.662 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:00:33.926651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:00:34.474030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T06:00:38.207853+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T06:01:19.663 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:01:19.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.764+0000 7f4897023640 1 Processor -- start 2024-03-08T06:01:19.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.764+0000 7f4897023640 1 -- start start 2024-03-08T06:01:19.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.765+0000 7f4897023640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f489015dbc0 0x7f48900d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.765+0000 7f4897023640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48900b5cf0 0x7f48900b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.765+0000 7f4897023640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48900d27d0 con 0x7f48900b9840 2024-03-08T06:01:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.765+0000 7f4897023640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48900d2ab0 con 0x7f489015dbc0 2024-03-08T06:01:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.765+0000 7f4897023640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48900d2d90 con 0x7f48900b5cf0 2024-03-08T06:01:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.765+0000 7f488ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f48900b9840 0x7f48900b55e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39430/0 (socket says 172.21.15.76:39430) 2024-03-08T06:01:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.765+0000 7f488ffff640 1 -- 172.21.15.76:0/2376945278 learned_addr learned my addr 172.21.15.76:0/2376945278 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488f7fe640 1 -- 172.21.15.76:0/2376945278 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969914638 0 0) 0x7f48900d27d0 con 0x7f48900b9840 2024-03-08T06:01:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488f7fe640 1 -- 172.21.15.76:0/2376945278 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4878003520 con 0x7f48900b9840 2024-03-08T06:01:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f4894dbf640 1 --2- 172.21.15.76:0/2376945278 >> v2:172.21.15.81:3300/0 conn(0x7f489015dbc0 0x7f48900d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f48955c0640 1 --2- 172.21.15.76:0/2376945278 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48900b5cf0 0x7f48900b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488f7fe640 1 -- 172.21.15.76:0/2376945278 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 335224653 0 0) 0x7f4878003520 con 0x7f48900b9840 2024-03-08T06:01:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488f7fe640 1 -- 172.21.15.76:0/2376945278 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48900b5cf0 msgr2=0x7f48900b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488f7fe640 1 --2- 172.21.15.76:0/2376945278 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48900b5cf0 0x7f48900b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488f7fe640 1 -- 172.21.15.76:0/2376945278 >> v2:172.21.15.81:3300/0 conn(0x7f489015dbc0 msgr2=0x7f48900d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488f7fe640 1 --2- 172.21.15.76:0/2376945278 >> v2:172.21.15.81:3300/0 conn(0x7f489015dbc0 0x7f48900d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488f7fe640 1 -- 172.21.15.76:0/2376945278 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48900d3070 con 0x7f48900b9840 2024-03-08T06:01:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488f7fe640 1 -- 172.21.15.76:0/2376945278 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f4884001670 con 0x7f48900b9840 2024-03-08T06:01:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488f7fe640 1 -- 172.21.15.76:0/2376945278 >> v1:172.21.15.76:6789/0 conn(0x7f48900b9840 legacy=0x7f48900b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488f7fe640 1 --2- 172.21.15.76:0/2376945278 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48900b5cf0 0x7f4878003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488f7fe640 1 --2- 172.21.15.76:0/2376945278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4878003fa0 0x7f4878004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488f7fe640 1 --2- 172.21.15.76:0/2376945278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4878004d40 0x7f48780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488f7fe640 1 -- 172.21.15.76:0/2376945278 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48900d2ab0 con 0x7f4878004d40 2024-03-08T06:01:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488f7fe640 1 -- 172.21.15.76:0/2376945278 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48900d2d90 con 0x7f4878003fa0 2024-03-08T06:01:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488f7fe640 1 -- 172.21.15.76:0/2376945278 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487800b9f0 con 0x7f48900b5cf0 2024-03-08T06:01:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f48955c0640 1 --2- 172.21.15.76:0/2376945278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4878004d40 0x7f48780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f4894dbf640 1 --2- 172.21.15.76:0/2376945278 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48900b5cf0 0x7f4878003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f488ffff640 1 --2- 172.21.15.76:0/2376945278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4878003fa0 0x7f4878004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f48955c0640 1 -- 172.21.15.76:0/2376945278 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48900b5cf0 msgr2=0x7f4878003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f48955c0640 1 --2- 172.21.15.76:0/2376945278 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48900b5cf0 0x7f4878003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f48955c0640 1 -- 172.21.15.76:0/2376945278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4878003fa0 msgr2=0x7f4878004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f48955c0640 1 --2- 172.21.15.76:0/2376945278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4878003fa0 0x7f4878004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.766+0000 7f48955c0640 1 -- 172.21.15.76:0/2376945278 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4878003520 con 0x7f4878004d40 2024-03-08T06:01:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.767+0000 7f48955c0640 1 --2- 172.21.15.76:0/2376945278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4878004d40 0x7f48780051c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4888009f90 tx=0x7f4878005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5534711e7733654 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.767+0000 7f488f7fe640 1 -- 172.21.15.76:0/2376945278 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888039070 con 0x7f4878004d40 2024-03-08T06:01:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.767+0000 7f488f7fe640 1 -- 172.21.15.76:0/2376945278 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488802cbd0 con 0x7f4878004d40 2024-03-08T06:01:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.767+0000 7f488f7fe640 1 -- 172.21.15.76:0/2376945278 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888009c10 con 0x7f4878004d40 2024-03-08T06:01:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.768+0000 7f4897023640 1 -- 172.21.15.76:0/2376945278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4878004d40 msgr2=0x7f48780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.768+0000 7f4897023640 1 --2- 172.21.15.76:0/2376945278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4878004d40 0x7f48780051c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4888009f90 tx=0x7f4878005830 comp rx=0 tx=0).stop 2024-03-08T06:01:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.768+0000 7f48955c0640 1 -- 172.21.15.76:0/2376945278 reap_dead start 2024-03-08T06:01:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.768+0000 7f4897023640 1 -- 172.21.15.76:0/2376945278 shutdown_connections 2024-03-08T06:01:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.768+0000 7f4897023640 1 -- 172.21.15.76:0/2376945278 >> 172.21.15.76:0/2376945278 conn(0x7f48900670d0 msgr2=0x7f48900d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.768+0000 7f4897023640 1 -- 172.21.15.76:0/2376945278 shutdown_connections 2024-03-08T06:01:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.768+0000 7f4897023640 1 -- 172.21.15.76:0/2376945278 wait complete. 2024-03-08T06:01:19.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.769+0000 7f4897023640 1 Processor -- start 2024-03-08T06:01:19.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.769+0000 7f4897023640 1 -- start start 2024-03-08T06:01:19.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f4897023640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48901e7730 0x7f48900b7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f4897023640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48901e7b10 0x7f4890146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f4897023640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4890146c80 0x7f48900cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f4897023640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48900d25b0 con 0x7f48901e7b10 2024-03-08T06:01:19.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f4897023640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48900d2430 con 0x7f4890146c80 2024-03-08T06:01:19.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f4897023640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489015dfa0 con 0x7f48901e7730 2024-03-08T06:01:19.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f488ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48901e7b10 0x7f4890146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f488ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48901e7b10 0x7f4890146740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41600/0 (socket says 172.21.15.76:41600) 2024-03-08T06:01:19.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f488ffff640 1 -- 172.21.15.76:0/2816412535 learned_addr learned my addr 172.21.15.76:0/2816412535 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:19.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f4894dbf640 1 --2- 172.21.15.76:0/2816412535 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48901e7730 0x7f48900b7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f48955c0640 1 --2- 172.21.15.76:0/2816412535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4890146c80 0x7f48900cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f488ffff640 1 -- 172.21.15.76:0/2816412535 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48901e7730 msgr2=0x7f48900b7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f488ffff640 1 --2- 172.21.15.76:0/2816412535 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48901e7730 0x7f48900b7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f488ffff640 1 -- 172.21.15.76:0/2816412535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4890146c80 msgr2=0x7f48900cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f488ffff640 1 --2- 172.21.15.76:0/2816412535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4890146c80 0x7f48900cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f488ffff640 1 -- 172.21.15.76:0/2816412535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48900cc140 con 0x7f48901e7b10 2024-03-08T06:01:19.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f4894dbf640 1 --2- 172.21.15.76:0/2816412535 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48901e7730 0x7f48900b7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:19.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f48955c0640 1 --2- 172.21.15.76:0/2816412535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4890146c80 0x7f48900cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:19.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.770+0000 7f488ffff640 1 --2- 172.21.15.76:0/2816412535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48901e7b10 0x7f4890146740 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f48840072c0 tx=0x7f4884007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:19.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.771+0000 7f488d7fa640 1 -- 172.21.15.76:0/2816412535 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400f9c0 con 0x7f48901e7b10 2024-03-08T06:01:19.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.771+0000 7f488d7fa640 1 -- 172.21.15.76:0/2816412535 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4884018070 con 0x7f48901e7b10 2024-03-08T06:01:19.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.771+0000 7f4897023640 1 -- 172.21.15.76:0/2816412535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48900cc420 con 0x7f48901e7b10 2024-03-08T06:01:19.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.771+0000 7f488d7fa640 1 -- 172.21.15.76:0/2816412535 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400fde0 con 0x7f48901e7b10 2024-03-08T06:01:19.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.771+0000 7f4897023640 1 -- 172.21.15.76:0/2816412535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48900cccc0 con 0x7f48901e7b10 2024-03-08T06:01:19.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.772+0000 7f4897023640 1 -- 172.21.15.76:0/2816412535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48900cc5f0 con 0x7f48901e7b10 2024-03-08T06:01:19.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.772+0000 7f488d7fa640 1 -- 172.21.15.76:0/2816412535 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f488401d460 con 0x7f48901e7b10 2024-03-08T06:01:19.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.773+0000 7f488d7fa640 1 --2- 172.21.15.76:0/2816412535 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f487c06b4d0 0x7f487c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:19.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.773+0000 7f488d7fa640 1 -- 172.21.15.76:0/2816412535 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f488408a770 con 0x7f48901e7b10 2024-03-08T06:01:19.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.773+0000 7f4894dbf640 1 --2- 172.21.15.76:0/2816412535 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f487c06b4d0 0x7f487c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:19.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.773+0000 7f4894dbf640 1 --2- 172.21.15.76:0/2816412535 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f487c06b4d0 0x7f487c06d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4880009b30 tx=0x7f4880008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:19.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.777+0000 7f488d7fa640 1 -- 172.21.15.76:0/2816412535 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48840591f0 con 0x7f48901e7b10 2024-03-08T06:01:19.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.948+0000 7f4897023640 1 -- 172.21.15.76:0/2816412535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f489004f030 con 0x7f48901e7b10 2024-03-08T06:01:19.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.948+0000 7f488d7fa640 1 -- 172.21.15.76:0/2816412535 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4884058af0 con 0x7f48901e7b10 2024-03-08T06:01:19.949 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:01:19.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.950+0000 7f4897023640 1 -- 172.21.15.76:0/2816412535 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f487c06b4d0 msgr2=0x7f487c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.950+0000 7f4897023640 1 --2- 172.21.15.76:0/2816412535 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f487c06b4d0 0x7f487c06d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4880009b30 tx=0x7f4880008040 comp rx=0 tx=0).stop 2024-03-08T06:01:19.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.950+0000 7f4897023640 1 -- 172.21.15.76:0/2816412535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48901e7b10 msgr2=0x7f4890146740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:19.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.950+0000 7f4897023640 1 --2- 172.21.15.76:0/2816412535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48901e7b10 0x7f4890146740 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f48840072c0 tx=0x7f4884007780 comp rx=0 tx=0).stop 2024-03-08T06:01:19.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.950+0000 7f4897023640 1 -- 172.21.15.76:0/2816412535 shutdown_connections 2024-03-08T06:01:19.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.950+0000 7f4897023640 1 --2- 172.21.15.76:0/2816412535 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f487c06b4d0 0x7f487c06d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.950+0000 7f4897023640 1 --2- 172.21.15.76:0/2816412535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4890146c80 0x7f48900cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.950+0000 7f4897023640 1 --2- 172.21.15.76:0/2816412535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48901e7b10 0x7f4890146740 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.950+0000 7f4897023640 1 --2- 172.21.15.76:0/2816412535 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48901e7730 0x7f48900b7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:19.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.950+0000 7f4897023640 1 -- 172.21.15.76:0/2816412535 >> 172.21.15.76:0/2816412535 conn(0x7f48900670d0 msgr2=0x7f48900c7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:19.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.950+0000 7f4897023640 1 -- 172.21.15.76:0/2816412535 shutdown_connections 2024-03-08T06:01:19.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:19.950+0000 7f4897023640 1 -- 172.21.15.76:0/2816412535 wait complete. 2024-03-08T06:01:19.961 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:01:20.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.065+0000 7f0268d79640 1 Processor -- start 2024-03-08T06:01:20.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.065+0000 7f0268d79640 1 -- start start 2024-03-08T06:01:20.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.066+0000 7f0268d79640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f02640b9840 0x7f02640b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.066+0000 7f0268d79640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f02640b5b20 0x7f02640b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.066+0000 7f0268d79640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02640b5f40 con 0x7f026415dbc0 2024-03-08T06:01:20.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.066+0000 7f0268d79640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02640d27d0 con 0x7f02640b5b20 2024-03-08T06:01:20.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.066+0000 7f0268d79640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02640d2ab0 con 0x7f02640b9840 2024-03-08T06:01:20.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.066+0000 7f0262d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f02640b5b20 0x7f02640b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.066+0000 7f026259c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f026415dbc0 0x7f02640d1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39440/0 (socket says 172.21.15.76:39440) 2024-03-08T06:01:20.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.066+0000 7f026259c640 1 -- 172.21.15.76:0/3419995811 learned_addr learned my addr 172.21.15.76:0/3419995811 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:20.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.066+0000 7f0262d9d640 1 -- 172.21.15.76:0/3419995811 >> v2:172.21.15.81:3300/0 conn(0x7f02640b9840 msgr2=0x7f02640b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:20.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.066+0000 7f0259d9b640 1 --2- 172.21.15.76:0/3419995811 >> v2:172.21.15.81:3300/0 conn(0x7f02640b9840 0x7f02640b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.066+0000 7f0262d9d640 1 --2- 172.21.15.76:0/3419995811 >> v2:172.21.15.81:3300/0 conn(0x7f02640b9840 0x7f02640b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.066+0000 7f0261d9b640 1 -- 172.21.15.76:0/3419995811 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621201384 0 0) 0x7f02640b5f40 con 0x7f026415dbc0 2024-03-08T06:01:20.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.066+0000 7f0262d9d640 1 -- 172.21.15.76:0/3419995811 >> v1:172.21.15.76:6789/0 conn(0x7f026415dbc0 legacy=0x7f02640d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.066+0000 7f0262d9d640 1 -- 172.21.15.76:0/3419995811 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02640d2d90 con 0x7f02640b5b20 2024-03-08T06:01:20.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.067+0000 7f0262d9d640 1 --2- 172.21.15.76:0/3419995811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02640b5b20 0x7f02640b6c90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f02640d19d0 tx=0x7f025c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdc20b952642631 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:20.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.067+0000 7f0261d9b640 1 -- 172.21.15.76:0/3419995811 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c039070 con 0x7f02640b5b20 2024-03-08T06:01:20.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.067+0000 7f0261d9b640 1 -- 172.21.15.76:0/3419995811 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025c02cc80 con 0x7f02640b5b20 2024-03-08T06:01:20.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.067+0000 7f0261d9b640 1 -- 172.21.15.76:0/3419995811 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c034660 con 0x7f02640b5b20 2024-03-08T06:01:20.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.068+0000 7f0268d79640 1 -- 172.21.15.76:0/3419995811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02640b5b20 msgr2=0x7f02640b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.068+0000 7f0268d79640 1 --2- 172.21.15.76:0/3419995811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02640b5b20 0x7f02640b6c90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f02640d19d0 tx=0x7f025c02bed0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.068+0000 7f0268d79640 1 -- 172.21.15.76:0/3419995811 shutdown_connections 2024-03-08T06:01:20.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.068+0000 7f0268d79640 1 --2- 172.21.15.76:0/3419995811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02640b5b20 0x7f02640b6c90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.068+0000 7f0268d79640 1 --2- 172.21.15.76:0/3419995811 >> v2:172.21.15.81:3300/0 conn(0x7f02640b9840 0x7f02640b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.068+0000 7f0268d79640 1 -- 172.21.15.76:0/3419995811 >> 172.21.15.76:0/3419995811 conn(0x7f02640670d0 msgr2=0x7f02640d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:20.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.068+0000 7f0268d79640 1 -- 172.21.15.76:0/3419995811 shutdown_connections 2024-03-08T06:01:20.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.068+0000 7f0268d79640 1 -- 172.21.15.76:0/3419995811 wait complete. 2024-03-08T06:01:20.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.069+0000 7f0268d79640 1 Processor -- start 2024-03-08T06:01:20.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.069+0000 7f0268d79640 1 -- start start 2024-03-08T06:01:20.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f0268d79640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f02640b5b20 0x7f026414d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f0268d79640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02640b9840 0x7f02640b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f0268d79640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f026415dbc0 0x7f02640b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f0268d79640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026415dfa0 con 0x7f02640b9840 2024-03-08T06:01:20.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f0268d79640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02640d3870 con 0x7f026415dbc0 2024-03-08T06:01:20.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f0268d79640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026415e120 con 0x7f02640b5b20 2024-03-08T06:01:20.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f0259d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02640b9840 0x7f02640b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f0259d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02640b9840 0x7f02640b6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41618/0 (socket says 172.21.15.76:41618) 2024-03-08T06:01:20.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f0259d9b640 1 -- 172.21.15.76:0/3603425999 learned_addr learned my addr 172.21.15.76:0/3603425999 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:20.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f026259c640 1 --2- 172.21.15.76:0/3603425999 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f02640b5b20 0x7f026414d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f0262d9d640 1 --2- 172.21.15.76:0/3603425999 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f026415dbc0 0x7f02640b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f0259d9b640 1 -- 172.21.15.76:0/3603425999 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f02640b5b20 msgr2=0x7f026414d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f0259d9b640 1 --2- 172.21.15.76:0/3603425999 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f02640b5b20 0x7f026414d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f0259d9b640 1 -- 172.21.15.76:0/3603425999 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f026415dbc0 msgr2=0x7f02640b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f0259d9b640 1 --2- 172.21.15.76:0/3603425999 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f026415dbc0 0x7f02640b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f0259d9b640 1 -- 172.21.15.76:0/3603425999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026415a0e0 con 0x7f02640b9840 2024-03-08T06:01:20.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f026259c640 1 --2- 172.21.15.76:0/3603425999 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f02640b5b20 0x7f026414d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:20.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.070+0000 7f0259d9b640 1 --2- 172.21.15.76:0/3603425999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02640b9840 0x7f02640b6e80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c7f0 tx=0x7f024c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:20.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.071+0000 7f025b7fe640 1 -- 172.21.15.76:0/3603425999 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c015070 con 0x7f02640b9840 2024-03-08T06:01:20.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.071+0000 7f025b7fe640 1 -- 172.21.15.76:0/3603425999 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c004240 con 0x7f02640b9840 2024-03-08T06:01:20.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.071+0000 7f0268d79640 1 -- 172.21.15.76:0/3603425999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026415a3c0 con 0x7f02640b9840 2024-03-08T06:01:20.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.071+0000 7f025b7fe640 1 -- 172.21.15.76:0/3603425999 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c003610 con 0x7f02640b9840 2024-03-08T06:01:20.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.071+0000 7f0268d79640 1 -- 172.21.15.76:0/3603425999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026415a980 con 0x7f02640b9840 2024-03-08T06:01:20.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.072+0000 7f0268d79640 1 -- 172.21.15.76:0/3603425999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026404f030 con 0x7f02640b9840 2024-03-08T06:01:20.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.072+0000 7f025b7fe640 1 -- 172.21.15.76:0/3603425999 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f024c0039c0 con 0x7f02640b9840 2024-03-08T06:01:20.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.072+0000 7f025b7fe640 1 --2- 172.21.15.76:0/3603425999 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f024406b4d0 0x7f024406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.072+0000 7f025b7fe640 1 -- 172.21.15.76:0/3603425999 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f024c08aa30 con 0x7f02640b9840 2024-03-08T06:01:20.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.073+0000 7f026259c640 1 --2- 172.21.15.76:0/3603425999 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f024406b4d0 0x7f024406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.073+0000 7f026259c640 1 --2- 172.21.15.76:0/3603425999 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f024406b4d0 0x7f024406d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0254000f60 tx=0x7f0254007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:20.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.077+0000 7f025b7fe640 1 -- 172.21.15.76:0/3603425999 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f024c0594b0 con 0x7f02640b9840 2024-03-08T06:01:20.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.249+0000 7f0268d79640 1 -- 172.21.15.76:0/3603425999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0264069e30 con 0x7f02640b9840 2024-03-08T06:01:20.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.250+0000 7f025b7fe640 1 -- 172.21.15.76:0/3603425999 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f024c05de20 con 0x7f02640b9840 2024-03-08T06:01:20.250 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:20.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.251+0000 7f0268d79640 1 -- 172.21.15.76:0/3603425999 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f024406b4d0 msgr2=0x7f024406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.251+0000 7f0268d79640 1 --2- 172.21.15.76:0/3603425999 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f024406b4d0 0x7f024406d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0254000f60 tx=0x7f0254007ca0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.252+0000 7f0268d79640 1 -- 172.21.15.76:0/3603425999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02640b9840 msgr2=0x7f02640b6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.252+0000 7f0268d79640 1 --2- 172.21.15.76:0/3603425999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02640b9840 0x7f02640b6e80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c7f0 tx=0x7f024c00ccb0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.252+0000 7f0268d79640 1 -- 172.21.15.76:0/3603425999 shutdown_connections 2024-03-08T06:01:20.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.252+0000 7f0268d79640 1 --2- 172.21.15.76:0/3603425999 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f024406b4d0 0x7f024406d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.252+0000 7f0268d79640 1 --2- 172.21.15.76:0/3603425999 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f026415dbc0 0x7f02640b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.252+0000 7f0268d79640 1 --2- 172.21.15.76:0/3603425999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02640b9840 0x7f02640b6e80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.252+0000 7f0268d79640 1 --2- 172.21.15.76:0/3603425999 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f02640b5b20 0x7f026414d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.252+0000 7f0268d79640 1 -- 172.21.15.76:0/3603425999 >> 172.21.15.76:0/3603425999 conn(0x7f02640670d0 msgr2=0x7f02640ccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:20.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.252+0000 7f0268d79640 1 -- 172.21.15.76:0/3603425999 shutdown_connections 2024-03-08T06:01:20.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.252+0000 7f0268d79640 1 -- 172.21.15.76:0/3603425999 wait complete. 2024-03-08T06:01:20.263 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:01:20.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.367+0000 7f57e0fd7640 1 Processor -- start 2024-03-08T06:01:20.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.367+0000 7f57e0fd7640 1 -- start start 2024-03-08T06:01:20.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.368+0000 7f57e0fd7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f57dc0b3c50 0x7f57dc0b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.368+0000 7f57e0fd7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f57dc0adea0 0x7f57dc06b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.368+0000 7f57e0fd7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57dc0b4570 con 0x7f57dc06ac80 2024-03-08T06:01:20.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.368+0000 7f57e0fd7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57dc06b720 con 0x7f57dc0adea0 2024-03-08T06:01:20.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.368+0000 7f57e0fd7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc06ba00 con 0x7f57dc0b3c50 2024-03-08T06:01:20.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.368+0000 7f57dad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f57dc0adea0 0x7f57dc06b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.368+0000 7f57dad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f57dc0adea0 0x7f57dc06b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41620/0 (socket says 172.21.15.76:41620) 2024-03-08T06:01:20.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.368+0000 7f57dad9d640 1 -- 172.21.15.76:0/4142876812 learned_addr learned my addr 172.21.15.76:0/4142876812 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:20.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.368+0000 7f57da59c640 1 --2- 172.21.15.76:0/4142876812 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f57dc0b3c50 0x7f57dc0b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.369+0000 7f57dad9d640 1 -- 172.21.15.76:0/4142876812 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f57dc0b3c50 msgr2=0x7f57dc0b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:20.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.369+0000 7f57dad9d640 1 --2- 172.21.15.76:0/4142876812 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f57dc0b3c50 0x7f57dc0b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.369+0000 7f57dad9d640 1 -- 172.21.15.76:0/4142876812 >> v1:172.21.15.81:6789/0 conn(0x7f57dc06ac80 legacy=0x7f57dc06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.369+0000 7f57dad9d640 1 -- 172.21.15.76:0/4142876812 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc06bce0 con 0x7f57dc0adea0 2024-03-08T06:01:20.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.369+0000 7f57da59c640 1 --2- 172.21.15.76:0/4142876812 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f57dc0b3c50 0x7f57dc0b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:20.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.369+0000 7f57dad9d640 1 --2- 172.21.15.76:0/4142876812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57dc0adea0 0x7f57dc06b0c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009870 tx=0x7f57d402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ecea298a52a660 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:20.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.369+0000 7f57d959a640 1 -- 172.21.15.76:0/4142876812 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4034070 con 0x7f57dc0adea0 2024-03-08T06:01:20.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.370+0000 7f57d959a640 1 -- 172.21.15.76:0/4142876812 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d4030840 con 0x7f57dc0adea0 2024-03-08T06:01:20.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.370+0000 7f57d959a640 1 -- 172.21.15.76:0/4142876812 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4030c10 con 0x7f57dc0adea0 2024-03-08T06:01:20.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.370+0000 7f57e0fd7640 1 -- 172.21.15.76:0/4142876812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57dc0adea0 msgr2=0x7f57dc06b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.370+0000 7f57e0fd7640 1 --2- 172.21.15.76:0/4142876812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57dc0adea0 0x7f57dc06b0c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009870 tx=0x7f57d402b980 comp rx=0 tx=0).stop 2024-03-08T06:01:20.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.370+0000 7f57e0fd7640 1 -- 172.21.15.76:0/4142876812 shutdown_connections 2024-03-08T06:01:20.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.370+0000 7f57e0fd7640 1 --2- 172.21.15.76:0/4142876812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57dc0adea0 0x7f57dc06b0c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.370+0000 7f57e0fd7640 1 --2- 172.21.15.76:0/4142876812 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f57dc0b3c50 0x7f57dc0b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.370+0000 7f57e0fd7640 1 -- 172.21.15.76:0/4142876812 >> 172.21.15.76:0/4142876812 conn(0x7f57dc06dd40 msgr2=0x7f57dc06e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:20.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.370+0000 7f57e0fd7640 1 -- 172.21.15.76:0/4142876812 shutdown_connections 2024-03-08T06:01:20.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.370+0000 7f57e0fd7640 1 -- 172.21.15.76:0/4142876812 wait complete. 2024-03-08T06:01:20.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.371+0000 7f57e0fd7640 1 Processor -- start 2024-03-08T06:01:20.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.371+0000 7f57e0fd7640 1 -- start start 2024-03-08T06:01:20.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.372+0000 7f57e0fd7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57dc06ac80 0x7f57dc0cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.372+0000 7f57e0fd7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57dc0adea0 0x7f57dc0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.372+0000 7f57e0fd7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f57dc0b3c50 0x7f57dc0c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.372+0000 7f57e0fd7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc070500 con 0x7f57dc06ac80 2024-03-08T06:01:20.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.372+0000 7f57e0fd7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc06c2a0 con 0x7f57dc0adea0 2024-03-08T06:01:20.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.372+0000 7f57e0fd7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc1612c0 con 0x7f57dc0b3c50 2024-03-08T06:01:20.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.372+0000 7f57da59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57dc06ac80 0x7f57dc0cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.372+0000 7f57da59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57dc06ac80 0x7f57dc0cbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41636/0 (socket says 172.21.15.76:41636) 2024-03-08T06:01:20.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.373+0000 7f57da59c640 1 -- 172.21.15.76:0/676718222 learned_addr learned my addr 172.21.15.76:0/676718222 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:20.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.373+0000 7f57d9d9b640 1 --2- 172.21.15.76:0/676718222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57dc0adea0 0x7f57dc0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.373+0000 7f57dad9d640 1 --2- 172.21.15.76:0/676718222 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f57dc0b3c50 0x7f57dc0c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.373+0000 7f57da59c640 1 -- 172.21.15.76:0/676718222 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f57dc0b3c50 msgr2=0x7f57dc0c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.373+0000 7f57da59c640 1 --2- 172.21.15.76:0/676718222 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f57dc0b3c50 0x7f57dc0c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.373+0000 7f57da59c640 1 -- 172.21.15.76:0/676718222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57dc0adea0 msgr2=0x7f57dc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.373+0000 7f57da59c640 1 --2- 172.21.15.76:0/676718222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57dc0adea0 0x7f57dc0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.373+0000 7f57da59c640 1 -- 172.21.15.76:0/676718222 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc0c6010 con 0x7f57dc06ac80 2024-03-08T06:01:20.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.373+0000 7f57da59c640 1 --2- 172.21.15.76:0/676718222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57dc06ac80 0x7f57dc0cbd90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f57cc007ef0 tx=0x7f57cc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:20.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.373+0000 7f57caffd640 1 -- 172.21.15.76:0/676718222 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc01a070 con 0x7f57dc06ac80 2024-03-08T06:01:20.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.373+0000 7f57caffd640 1 -- 172.21.15.76:0/676718222 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc003b90 con 0x7f57dc06ac80 2024-03-08T06:01:20.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.373+0000 7f57caffd640 1 -- 172.21.15.76:0/676718222 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc004030 con 0x7f57dc06ac80 2024-03-08T06:01:20.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.373+0000 7f57e0fd7640 1 -- 172.21.15.76:0/676718222 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57dc0cafd0 con 0x7f57dc06ac80 2024-03-08T06:01:20.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.373+0000 7f57e0fd7640 1 -- 172.21.15.76:0/676718222 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57dc0cb590 con 0x7f57dc06ac80 2024-03-08T06:01:20.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.374+0000 7f57e0fd7640 1 -- 172.21.15.76:0/676718222 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57dc072560 con 0x7f57dc06ac80 2024-03-08T06:01:20.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.374+0000 7f57caffd640 1 -- 172.21.15.76:0/676718222 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57cc01d070 con 0x7f57dc06ac80 2024-03-08T06:01:20.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.375+0000 7f57caffd640 1 --2- 172.21.15.76:0/676718222 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f57bc06b4d0 0x7f57bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.375+0000 7f57caffd640 1 -- 172.21.15.76:0/676718222 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f57cc08a750 con 0x7f57dc06ac80 2024-03-08T06:01:20.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.375+0000 7f57d9d9b640 1 --2- 172.21.15.76:0/676718222 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f57bc06b4d0 0x7f57bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.375+0000 7f57d9d9b640 1 --2- 172.21.15.76:0/676718222 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f57bc06b4d0 0x7f57bc06d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f57dc1f6c10 tx=0x7f57d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:20.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.379+0000 7f57caffd640 1 -- 172.21.15.76:0/676718222 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57dc072560 con 0x7f57dc06ac80 2024-03-08T06:01:20.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.550+0000 7f57e0fd7640 1 -- 172.21.15.76:0/676718222 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57dc06cdf0 con 0x7f57dc06ac80 2024-03-08T06:01:20.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.550+0000 7f57caffd640 1 -- 172.21.15.76:0/676718222 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57cc010070 con 0x7f57dc06ac80 2024-03-08T06:01:20.551 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:01:20.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.552+0000 7f57e0fd7640 1 -- 172.21.15.76:0/676718222 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f57bc06b4d0 msgr2=0x7f57bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.552+0000 7f57e0fd7640 1 --2- 172.21.15.76:0/676718222 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f57bc06b4d0 0x7f57bc06d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f57dc1f6c10 tx=0x7f57d0009040 comp rx=0 tx=0).stop 2024-03-08T06:01:20.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.552+0000 7f57e0fd7640 1 -- 172.21.15.76:0/676718222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57dc06ac80 msgr2=0x7f57dc0cbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.552+0000 7f57e0fd7640 1 --2- 172.21.15.76:0/676718222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57dc06ac80 0x7f57dc0cbd90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f57cc007ef0 tx=0x7f57cc00d4a0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.552+0000 7f57e0fd7640 1 -- 172.21.15.76:0/676718222 shutdown_connections 2024-03-08T06:01:20.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.552+0000 7f57e0fd7640 1 --2- 172.21.15.76:0/676718222 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f57bc06b4d0 0x7f57bc06d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.552+0000 7f57e0fd7640 1 --2- 172.21.15.76:0/676718222 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f57dc0b3c50 0x7f57dc0c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.552+0000 7f57e0fd7640 1 --2- 172.21.15.76:0/676718222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57dc0adea0 0x7f57dc0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.552+0000 7f57e0fd7640 1 --2- 172.21.15.76:0/676718222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57dc06ac80 0x7f57dc0cbd90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.552+0000 7f57e0fd7640 1 -- 172.21.15.76:0/676718222 >> 172.21.15.76:0/676718222 conn(0x7f57dc06dd40 msgr2=0x7f57dc0ae760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:20.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.552+0000 7f57e0fd7640 1 -- 172.21.15.76:0/676718222 shutdown_connections 2024-03-08T06:01:20.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.553+0000 7f57e0fd7640 1 -- 172.21.15.76:0/676718222 wait complete. 2024-03-08T06:01:20.564 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T06:01:20.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.666+0000 7fcdcddd0640 1 Processor -- start 2024-03-08T06:01:20.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.666+0000 7fcdcddd0640 1 -- start start 2024-03-08T06:01:20.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.667+0000 7fcdcddd0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcdc815bd70 0x7fcdc80ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.667+0000 7fcdcddd0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc80cc5b0 con 0x7fcdc806ac80 2024-03-08T06:01:20.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.667+0000 7fcdcddd0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc80cc890 con 0x7fcdc80b2f20 2024-03-08T06:01:20.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.667+0000 7fcdcddd0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc80ccb70 con 0x7fcdc815bd70 2024-03-08T06:01:20.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.667+0000 7fcdc77fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fcdc806ac80 0x7fcdc80d00a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39442/0 (socket says 172.21.15.76:39442) 2024-03-08T06:01:20.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.667+0000 7fcdc77fe640 1 -- 172.21.15.76:0/3733781234 learned_addr learned my addr 172.21.15.76:0/3733781234 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:20.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.667+0000 7fcdc7fff640 1 --2- 172.21.15.76:0/3733781234 >> v2:172.21.15.76:3300/0 conn(0x7fcdc815bd70 0x7fcdc80ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.668+0000 7fcdc7fff640 1 -- 172.21.15.76:0/3733781234 >> v1:172.21.15.81:6789/0 conn(0x7fcdc80b2f20 legacy=0x7fcdc806b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:20.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.668+0000 7fcdc7fff640 1 -- 172.21.15.76:0/3733781234 >> v1:172.21.15.76:6789/0 conn(0x7fcdc806ac80 legacy=0x7fcdc80d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.668+0000 7fcdc7fff640 1 -- 172.21.15.76:0/3733781234 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc80cce50 con 0x7fcdc815bd70 2024-03-08T06:01:20.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.668+0000 7fcdc7fff640 1 --2- 172.21.15.76:0/3733781234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdc815bd70 0x7fcdc80ad600 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009d90 tx=0x7fcdc002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f3f55fc716cae2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:20.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.668+0000 7fcdc67fc640 1 -- 172.21.15.76:0/3733781234 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0039070 con 0x7fcdc815bd70 2024-03-08T06:01:20.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.668+0000 7fcdc67fc640 1 -- 172.21.15.76:0/3733781234 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc002cc60 con 0x7fcdc815bd70 2024-03-08T06:01:20.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.668+0000 7fcdc67fc640 1 -- 172.21.15.76:0/3733781234 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0034660 con 0x7fcdc815bd70 2024-03-08T06:01:20.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.669+0000 7fcdcddd0640 1 -- 172.21.15.76:0/3733781234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdc815bd70 msgr2=0x7fcdc80ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.669+0000 7fcdcddd0640 1 --2- 172.21.15.76:0/3733781234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdc815bd70 0x7fcdc80ad600 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009d90 tx=0x7fcdc002beb0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.669+0000 7fcdcddd0640 1 -- 172.21.15.76:0/3733781234 shutdown_connections 2024-03-08T06:01:20.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.669+0000 7fcdcddd0640 1 --2- 172.21.15.76:0/3733781234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdc815bd70 0x7fcdc80ad600 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.669+0000 7fcdcddd0640 1 -- 172.21.15.76:0/3733781234 >> 172.21.15.76:0/3733781234 conn(0x7fcdc80b2320 msgr2=0x7fcdc80b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:20.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.669+0000 7fcdcddd0640 1 -- 172.21.15.76:0/3733781234 shutdown_connections 2024-03-08T06:01:20.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.669+0000 7fcdcddd0640 1 -- 172.21.15.76:0/3733781234 wait complete. 2024-03-08T06:01:20.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.670+0000 7fcdcddd0640 1 Processor -- start 2024-03-08T06:01:20.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.670+0000 7fcdcddd0640 1 -- start start 2024-03-08T06:01:20.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdcddd0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdc806ac80 0x7fcdc80af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdcddd0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcdc80b2f20 0x7fcdc81559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdcddd0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdc815bd70 0x7fcdc81f17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdcddd0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc80cc350 con 0x7fcdc806ac80 2024-03-08T06:01:20.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdcddd0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc80cc1d0 con 0x7fcdc815bd70 2024-03-08T06:01:20.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdcddd0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc80adb40 con 0x7fcdc80b2f20 2024-03-08T06:01:20.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdc77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdc806ac80 0x7fcdc80af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdc77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdc806ac80 0x7fcdc80af9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41656/0 (socket says 172.21.15.76:41656) 2024-03-08T06:01:20.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdc77fe640 1 -- 172.21.15.76:0/4269753525 learned_addr learned my addr 172.21.15.76:0/4269753525 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:20.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdc7fff640 1 --2- 172.21.15.76:0/4269753525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdc815bd70 0x7fcdc81f17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdc6ffd640 1 --2- 172.21.15.76:0/4269753525 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcdc80b2f20 0x7fcdc81559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdc77fe640 1 -- 172.21.15.76:0/4269753525 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcdc80b2f20 msgr2=0x7fcdc81559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdc77fe640 1 --2- 172.21.15.76:0/4269753525 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcdc80b2f20 0x7fcdc81559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdc77fe640 1 -- 172.21.15.76:0/4269753525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdc815bd70 msgr2=0x7fcdc81f17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdc77fe640 1 --2- 172.21.15.76:0/4269753525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdc815bd70 0x7fcdc81f17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdc77fe640 1 -- 172.21.15.76:0/4269753525 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc81f4d40 con 0x7fcdc806ac80 2024-03-08T06:01:20.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdc6ffd640 1 --2- 172.21.15.76:0/4269753525 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcdc80b2f20 0x7fcdc81559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:20.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdc7fff640 1 --2- 172.21.15.76:0/4269753525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdc815bd70 0x7fcdc81f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:20.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.671+0000 7fcdc77fe640 1 --2- 172.21.15.76:0/4269753525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdc806ac80 0x7fcdc80af9e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800c400 tx=0x7fcdb800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:20.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.672+0000 7fcdaffff640 1 -- 172.21.15.76:0/4269753525 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8017070 con 0x7fcdc806ac80 2024-03-08T06:01:20.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.672+0000 7fcdaffff640 1 -- 172.21.15.76:0/4269753525 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb8005ce0 con 0x7fcdc806ac80 2024-03-08T06:01:20.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.672+0000 7fcdcddd0640 1 -- 172.21.15.76:0/4269753525 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc80ae2e0 con 0x7fcdc806ac80 2024-03-08T06:01:20.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.672+0000 7fcdaffff640 1 -- 172.21.15.76:0/4269753525 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb80072f0 con 0x7fcdc806ac80 2024-03-08T06:01:20.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.672+0000 7fcdcddd0640 1 -- 172.21.15.76:0/4269753525 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc80ae8a0 con 0x7fcdc806ac80 2024-03-08T06:01:20.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.673+0000 7fcdcddd0640 1 -- 172.21.15.76:0/4269753525 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc804f030 con 0x7fcdc806ac80 2024-03-08T06:01:20.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.673+0000 7fcdaffff640 1 -- 172.21.15.76:0/4269753525 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcdb801a070 con 0x7fcdc806ac80 2024-03-08T06:01:20.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.673+0000 7fcdaffff640 1 --2- 172.21.15.76:0/4269753525 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcdb406b4d0 0x7fcdb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.674+0000 7fcdaffff640 1 -- 172.21.15.76:0/4269753525 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fcdb8089a40 con 0x7fcdc806ac80 2024-03-08T06:01:20.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.674+0000 7fcdc6ffd640 1 --2- 172.21.15.76:0/4269753525 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcdb406b4d0 0x7fcdb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.674+0000 7fcdc6ffd640 1 --2- 172.21.15.76:0/4269753525 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcdb406b4d0 0x7fcdb406d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcdc81f5e20 tx=0x7fcdbc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:20.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.678+0000 7fcdaffff640 1 -- 172.21.15.76:0/4269753525 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcdb80584c0 con 0x7fcdc806ac80 2024-03-08T06:01:20.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.848+0000 7fcdcddd0640 1 -- 172.21.15.76:0/4269753525 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcdc8069ca0 con 0x7fcdc806ac80 2024-03-08T06:01:20.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.848+0000 7fcdaffff640 1 -- 172.21.15.76:0/4269753525 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcdb8089e20 con 0x7fcdc806ac80 2024-03-08T06:01:20.849 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:20.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.850+0000 7fcdcddd0640 1 -- 172.21.15.76:0/4269753525 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcdb406b4d0 msgr2=0x7fcdb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.850+0000 7fcdcddd0640 1 --2- 172.21.15.76:0/4269753525 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcdb406b4d0 0x7fcdb406d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcdc81f5e20 tx=0x7fcdbc009040 comp rx=0 tx=0).stop 2024-03-08T06:01:20.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.850+0000 7fcdcddd0640 1 -- 172.21.15.76:0/4269753525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdc806ac80 msgr2=0x7fcdc80af9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.850+0000 7fcdcddd0640 1 --2- 172.21.15.76:0/4269753525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdc806ac80 0x7fcdc80af9e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800c400 tx=0x7fcdb800c8c0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.850+0000 7fcdcddd0640 1 -- 172.21.15.76:0/4269753525 shutdown_connections 2024-03-08T06:01:20.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.850+0000 7fcdcddd0640 1 --2- 172.21.15.76:0/4269753525 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcdb406b4d0 0x7fcdb406d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.850+0000 7fcdcddd0640 1 --2- 172.21.15.76:0/4269753525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdc815bd70 0x7fcdc81f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.850+0000 7fcdcddd0640 1 --2- 172.21.15.76:0/4269753525 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcdc80b2f20 0x7fcdc81559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.850+0000 7fcdcddd0640 1 --2- 172.21.15.76:0/4269753525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdc806ac80 0x7fcdc80af9e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.850+0000 7fcdcddd0640 1 -- 172.21.15.76:0/4269753525 >> 172.21.15.76:0/4269753525 conn(0x7fcdc80b2320 msgr2=0x7fcdc80b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:20.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.850+0000 7fcdcddd0640 1 -- 172.21.15.76:0/4269753525 shutdown_connections 2024-03-08T06:01:20.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.850+0000 7fcdcddd0640 1 -- 172.21.15.76:0/4269753525 wait complete. 2024-03-08T06:01:20.861 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:20.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.961+0000 7f948c457640 1 Processor -- start 2024-03-08T06:01:20.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.961+0000 7f948c457640 1 -- start start 2024-03-08T06:01:20.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.962+0000 7f948c457640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f948406d3f0 0x7f94840cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.962+0000 7f948c457640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948406d880 con 0x7f94840cd310 2024-03-08T06:01:20.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.962+0000 7f948c457640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948406e010 con 0x7f9484071390 2024-03-08T06:01:20.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.962+0000 7f948c457640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948406e2f0 con 0x7f948406d3f0 2024-03-08T06:01:20.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.962+0000 7f948a9f4640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f94840cd310 0x7f94840cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39456/0 (socket says 172.21.15.76:39456) 2024-03-08T06:01:20.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.962+0000 7f948a9f4640 1 -- 172.21.15.76:0/677388019 learned_addr learned my addr 172.21.15.76:0/677388019 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:20.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.962+0000 7f94899f2640 1 --2- 172.21.15.76:0/677388019 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f948406d3f0 0x7f94840cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94891f1640 1 -- 172.21.15.76:0/677388019 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170560441 0 0) 0x7f948406d880 con 0x7f94840cd310 2024-03-08T06:01:20.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94891f1640 1 -- 172.21.15.76:0/677388019 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9470003520 con 0x7f94840cd310 2024-03-08T06:01:20.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94891f1640 1 -- 172.21.15.76:0/677388019 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2202810107 0 0) 0x7f9470003520 con 0x7f94840cd310 2024-03-08T06:01:20.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94891f1640 1 -- 172.21.15.76:0/677388019 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f948406d3f0 msgr2=0x7f94840cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:20.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94891f1640 1 --2- 172.21.15.76:0/677388019 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f948406d3f0 0x7f94840cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94891f1640 1 -- 172.21.15.76:0/677388019 >> v1:172.21.15.81:6789/0 conn(0x7f9484071390 legacy=0x7f9484071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94891f1640 1 -- 172.21.15.76:0/677388019 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94840ce7b0 con 0x7f94840cd310 2024-03-08T06:01:20.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94891f1640 1 -- 172.21.15.76:0/677388019 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f947c001670 con 0x7f94840cd310 2024-03-08T06:01:20.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94891f1640 1 -- 172.21.15.76:0/677388019 >> v1:172.21.15.76:6789/0 conn(0x7f94840cd310 legacy=0x7f94840cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94891f1640 1 --2- 172.21.15.76:0/677388019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9470003a50 0x7f9470003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94891f1640 1 --2- 172.21.15.76:0/677388019 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9470004800 0x7f9470004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94891f1640 1 --2- 172.21.15.76:0/677388019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94700051c0 0x7f947000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94891f1640 1 -- 172.21.15.76:0/677388019 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948406e2f0 con 0x7f94700051c0 2024-03-08T06:01:20.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94891f1640 1 -- 172.21.15.76:0/677388019 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947000bfb0 con 0x7f9470003a50 2024-03-08T06:01:20.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94891f1640 1 -- 172.21.15.76:0/677388019 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947000c1b0 con 0x7f9470004800 2024-03-08T06:01:20.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94899f2640 1 --2- 172.21.15.76:0/677388019 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f948406d3f0 0x7f94840cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:20.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f948a9f4640 1 --2- 172.21.15.76:0/677388019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94700051c0 0x7f947000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f948a1f3640 1 --2- 172.21.15.76:0/677388019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9470003a50 0x7f9470003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f94899f2640 1 --2- 172.21.15.76:0/677388019 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9470004800 0x7f9470004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f948a9f4640 1 -- 172.21.15.76:0/677388019 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9470004800 msgr2=0x7f9470004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:20.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f948a9f4640 1 --2- 172.21.15.76:0/677388019 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9470004800 0x7f9470004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f948a9f4640 1 -- 172.21.15.76:0/677388019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9470003a50 msgr2=0x7f9470003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:20.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f948a9f4640 1 --2- 172.21.15.76:0/677388019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9470003a50 0x7f9470003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.963+0000 7f948a9f4640 1 -- 172.21.15.76:0/677388019 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9470003520 con 0x7f94700051c0 2024-03-08T06:01:20.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.964+0000 7f948a9f4640 1 --2- 172.21.15.76:0/677388019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94700051c0 0x7f947000b9c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f947c00c190 tx=0x7f9470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3054b317504d5a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:20.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.964+0000 7f94891f1640 1 -- 172.21.15.76:0/677388019 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c007cc0 con 0x7f94700051c0 2024-03-08T06:01:20.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.964+0000 7f94891f1640 1 -- 172.21.15.76:0/677388019 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c030070 con 0x7f94700051c0 2024-03-08T06:01:20.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.964+0000 7f94891f1640 1 -- 172.21.15.76:0/677388019 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c005d10 con 0x7f94700051c0 2024-03-08T06:01:20.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.965+0000 7f948c457640 1 -- 172.21.15.76:0/677388019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94700051c0 msgr2=0x7f947000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.965+0000 7f948c457640 1 --2- 172.21.15.76:0/677388019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94700051c0 0x7f947000b9c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f947c00c190 tx=0x7f9470005830 comp rx=0 tx=0).stop 2024-03-08T06:01:20.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.965+0000 7f948a9f4640 1 -- 172.21.15.76:0/677388019 reap_dead start 2024-03-08T06:01:20.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.965+0000 7f948c457640 1 -- 172.21.15.76:0/677388019 shutdown_connections 2024-03-08T06:01:20.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.965+0000 7f948c457640 1 -- 172.21.15.76:0/677388019 >> 172.21.15.76:0/677388019 conn(0x7f94840cf740 msgr2=0x7f94840cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:20.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.965+0000 7f948c457640 1 -- 172.21.15.76:0/677388019 shutdown_connections 2024-03-08T06:01:20.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.965+0000 7f948c457640 1 -- 172.21.15.76:0/677388019 wait complete. 2024-03-08T06:01:20.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.966+0000 7f948c457640 1 Processor -- start 2024-03-08T06:01:20.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.966+0000 7f948c457640 1 -- start start 2024-03-08T06:01:20.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948c457640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94700051c0 0x7f948414f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948c457640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94840c8d90 0x7f94840c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948c457640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94840b2fa0 0x7f94840b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948c457640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948415c060 con 0x7f94700051c0 2024-03-08T06:01:20.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948c457640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948415bee0 con 0x7f94840c8d90 2024-03-08T06:01:20.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948c457640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94840cf0e0 con 0x7f94840b2fa0 2024-03-08T06:01:20.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948a1f3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94700051c0 0x7f948414f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948a1f3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94700051c0 0x7f948414f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41664/0 (socket says 172.21.15.76:41664) 2024-03-08T06:01:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948a1f3640 1 -- 172.21.15.76:0/3943509362 learned_addr learned my addr 172.21.15.76:0/3943509362 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f94899f2640 1 --2- 172.21.15.76:0/3943509362 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94840c8d90 0x7f94840c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948a1f3640 1 -- 172.21.15.76:0/3943509362 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94840b2fa0 msgr2=0x7f94840b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948a9f4640 1 --2- 172.21.15.76:0/3943509362 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94840b2fa0 0x7f94840b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948a1f3640 1 --2- 172.21.15.76:0/3943509362 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94840b2fa0 0x7f94840b3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948a1f3640 1 -- 172.21.15.76:0/3943509362 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94840c8d90 msgr2=0x7f94840c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948a1f3640 1 --2- 172.21.15.76:0/3943509362 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94840c8d90 0x7f94840c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948a1f3640 1 -- 172.21.15.76:0/3943509362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948414fbd0 con 0x7f94700051c0 2024-03-08T06:01:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948a1f3640 1 --2- 172.21.15.76:0/3943509362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94700051c0 0x7f948414f2d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f948000fb90 tx=0x7f948000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f947affd640 1 -- 172.21.15.76:0/3943509362 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948000dc30 con 0x7f94700051c0 2024-03-08T06:01:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f947affd640 1 -- 172.21.15.76:0/3943509362 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9480018070 con 0x7f94700051c0 2024-03-08T06:01:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948c457640 1 -- 172.21.15.76:0/3943509362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948414fe70 con 0x7f94700051c0 2024-03-08T06:01:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f947affd640 1 -- 172.21.15.76:0/3943509362 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480003970 con 0x7f94700051c0 2024-03-08T06:01:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.967+0000 7f948c457640 1 -- 172.21.15.76:0/3943509362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9484150430 con 0x7f94700051c0 2024-03-08T06:01:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.968+0000 7f948c457640 1 -- 172.21.15.76:0/3943509362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948404f030 con 0x7f94700051c0 2024-03-08T06:01:20.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.969+0000 7f947affd640 1 -- 172.21.15.76:0/3943509362 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9480003b30 con 0x7f94700051c0 2024-03-08T06:01:20.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.969+0000 7f947affd640 1 --2- 172.21.15.76:0/3943509362 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f946806b4d0 0x7f946806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:20.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.969+0000 7f947affd640 1 -- 172.21.15.76:0/3943509362 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f948008a880 con 0x7f94700051c0 2024-03-08T06:01:20.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.969+0000 7f94899f2640 1 --2- 172.21.15.76:0/3943509362 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f946806b4d0 0x7f946806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:20.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.970+0000 7f94899f2640 1 --2- 172.21.15.76:0/3943509362 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f946806b4d0 0x7f946806d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f94840b1130 tx=0x7f9474008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:20.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:20.974+0000 7f947affd640 1 -- 172.21.15.76:0/3943509362 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9480003e20 con 0x7f94700051c0 2024-03-08T06:01:21.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.144+0000 7f948c457640 1 -- 172.21.15.76:0/3943509362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94840cbfa0 con 0x7f94700051c0 2024-03-08T06:01:21.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.145+0000 7f947affd640 1 -- 172.21.15.76:0/3943509362 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18757 (secure 0 0 0) 0x7f9480059390 con 0x7f94700051c0 2024-03-08T06:01:21.146 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:21.146 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":28,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:01:17.313272+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:21.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.147+0000 7f948c457640 1 -- 172.21.15.76:0/3943509362 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f946806b4d0 msgr2=0x7f946806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.147+0000 7f948c457640 1 --2- 172.21.15.76:0/3943509362 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f946806b4d0 0x7f946806d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f94840b1130 tx=0x7f9474008040 comp rx=0 tx=0).stop 2024-03-08T06:01:21.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.147+0000 7f948c457640 1 -- 172.21.15.76:0/3943509362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94700051c0 msgr2=0x7f948414f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.147+0000 7f948c457640 1 --2- 172.21.15.76:0/3943509362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94700051c0 0x7f948414f2d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f948000fb90 tx=0x7f948000d520 comp rx=0 tx=0).stop 2024-03-08T06:01:21.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.147+0000 7f948c457640 1 -- 172.21.15.76:0/3943509362 shutdown_connections 2024-03-08T06:01:21.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.147+0000 7f948c457640 1 --2- 172.21.15.76:0/3943509362 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f946806b4d0 0x7f946806d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.147+0000 7f948c457640 1 --2- 172.21.15.76:0/3943509362 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94840b2fa0 0x7f94840b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.147+0000 7f948c457640 1 --2- 172.21.15.76:0/3943509362 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94840c8d90 0x7f94840c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.147+0000 7f948c457640 1 --2- 172.21.15.76:0/3943509362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94700051c0 0x7f948414f2d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.147+0000 7f948c457640 1 -- 172.21.15.76:0/3943509362 >> 172.21.15.76:0/3943509362 conn(0x7f94840cf740 msgr2=0x7f94840b5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:21.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.148+0000 7f948c457640 1 -- 172.21.15.76:0/3943509362 shutdown_connections 2024-03-08T06:01:21.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.148+0000 7f948c457640 1 -- 172.21.15.76:0/3943509362 wait complete. 2024-03-08T06:01:21.159 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:00:33.926651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:00:34.474030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T06:00:38.207853+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T06:01:21.159 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:01:21.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.261+0000 7f41e1888640 1 Processor -- start 2024-03-08T06:01:21.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.261+0000 7f41e1888640 1 -- start start 2024-03-08T06:01:21.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.262+0000 7f41e1888640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f41dc15aa80 0x7f41dc06e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.262+0000 7f41e1888640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f41dc06eec0 0x7f41dc15ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.262+0000 7f41e1888640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41dc15b930 con 0x7f41dc0ce440 2024-03-08T06:01:21.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.262+0000 7f41e1888640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41dc15bc10 con 0x7f41dc15aa80 2024-03-08T06:01:21.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.262+0000 7f41e1888640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41dc15bef0 con 0x7f41dc06eec0 2024-03-08T06:01:21.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.262+0000 7f41daffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f41dc15aa80 0x7f41dc06e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.262+0000 7f41db7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f41dc0ce440 0x7f41dc0cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39472/0 (socket says 172.21.15.76:39472) 2024-03-08T06:01:21.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.262+0000 7f41db7fe640 1 -- 172.21.15.76:0/3924326079 learned_addr learned my addr 172.21.15.76:0/3924326079 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:21.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.263+0000 7f41da7fc640 1 --2- 172.21.15.76:0/3924326079 >> v2:172.21.15.81:3300/0 conn(0x7f41dc06eec0 0x7f41dc15ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.263+0000 7f41daffd640 1 -- 172.21.15.76:0/3924326079 >> v2:172.21.15.81:3300/0 conn(0x7f41dc06eec0 msgr2=0x7f41dc15ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.263+0000 7f41daffd640 1 --2- 172.21.15.76:0/3924326079 >> v2:172.21.15.81:3300/0 conn(0x7f41dc06eec0 0x7f41dc15ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.263+0000 7f41daffd640 1 -- 172.21.15.76:0/3924326079 >> v1:172.21.15.76:6789/0 conn(0x7f41dc0ce440 legacy=0x7f41dc0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.263+0000 7f41daffd640 1 -- 172.21.15.76:0/3924326079 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc0cd320 con 0x7f41dc15aa80 2024-03-08T06:01:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.263+0000 7f41d9ffb640 1 -- 172.21.15.76:0/3924326079 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258852129 0 0) 0x7f41dc15b930 con 0x7f41dc0ce440 2024-03-08T06:01:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.263+0000 7f41daffd640 1 --2- 172.21.15.76:0/3924326079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41dc15aa80 0x7f41dc06e980 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f41dc15f780 tx=0x7f41cc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30def9ef3a5a4183 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.264+0000 7f41d9ffb640 1 -- 172.21.15.76:0/3924326079 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc039070 con 0x7f41dc15aa80 2024-03-08T06:01:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.264+0000 7f41d9ffb640 1 -- 172.21.15.76:0/3924326079 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc02cc80 con 0x7f41dc15aa80 2024-03-08T06:01:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.264+0000 7f41d9ffb640 1 -- 172.21.15.76:0/3924326079 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc034660 con 0x7f41dc15aa80 2024-03-08T06:01:21.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.264+0000 7f41e1888640 1 -- 172.21.15.76:0/3924326079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41dc15aa80 msgr2=0x7f41dc06e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.264+0000 7f41e1888640 1 --2- 172.21.15.76:0/3924326079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41dc15aa80 0x7f41dc06e980 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f41dc15f780 tx=0x7f41cc02bed0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.264+0000 7f41e1888640 1 -- 172.21.15.76:0/3924326079 shutdown_connections 2024-03-08T06:01:21.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.264+0000 7f41e1888640 1 --2- 172.21.15.76:0/3924326079 >> v2:172.21.15.81:3300/0 conn(0x7f41dc06eec0 0x7f41dc15ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.264+0000 7f41e1888640 1 --2- 172.21.15.76:0/3924326079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41dc15aa80 0x7f41dc06e980 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.264+0000 7f41e1888640 1 -- 172.21.15.76:0/3924326079 >> 172.21.15.76:0/3924326079 conn(0x7f41dc099a60 msgr2=0x7f41dc099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:21.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.264+0000 7f41e1888640 1 -- 172.21.15.76:0/3924326079 shutdown_connections 2024-03-08T06:01:21.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.264+0000 7f41e1888640 1 -- 172.21.15.76:0/3924326079 wait complete. 2024-03-08T06:01:21.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.265+0000 7f41e1888640 1 Processor -- start 2024-03-08T06:01:21.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.265+0000 7f41e1888640 1 -- start start 2024-03-08T06:01:21.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.266+0000 7f41e1888640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41dc06eec0 0x7f41dc1518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.266+0000 7f41e1888640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41dc0ce440 0x7f41dc15cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.266+0000 7f41e1888640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41dc15aa80 0x7f41dc159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.266+0000 7f41e1888640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc15b520 con 0x7f41dc15aa80 2024-03-08T06:01:21.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.266+0000 7f41e1888640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc15b3a0 con 0x7f41dc0ce440 2024-03-08T06:01:21.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.266+0000 7f41e1888640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc15b6a0 con 0x7f41dc06eec0 2024-03-08T06:01:21.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.266+0000 7f41db7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41dc15aa80 0x7f41dc159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.266+0000 7f41db7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41dc15aa80 0x7f41dc159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41684/0 (socket says 172.21.15.76:41684) 2024-03-08T06:01:21.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.266+0000 7f41db7fe640 1 -- 172.21.15.76:0/3912396489 learned_addr learned my addr 172.21.15.76:0/3912396489 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:21.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.267+0000 7f41da7fc640 1 --2- 172.21.15.76:0/3912396489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41dc0ce440 0x7f41dc15cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.267+0000 7f41daffd640 1 --2- 172.21.15.76:0/3912396489 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41dc06eec0 0x7f41dc1518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.267+0000 7f41db7fe640 1 -- 172.21.15.76:0/3912396489 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41dc06eec0 msgr2=0x7f41dc1518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.267+0000 7f41db7fe640 1 --2- 172.21.15.76:0/3912396489 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41dc06eec0 0x7f41dc1518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.267+0000 7f41db7fe640 1 -- 172.21.15.76:0/3912396489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41dc0ce440 msgr2=0x7f41dc15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.267+0000 7f41db7fe640 1 --2- 172.21.15.76:0/3912396489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41dc0ce440 0x7f41dc15cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.267+0000 7f41db7fe640 1 -- 172.21.15.76:0/3912396489 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc0c8d60 con 0x7f41dc15aa80 2024-03-08T06:01:21.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.267+0000 7f41da7fc640 1 --2- 172.21.15.76:0/3912396489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41dc0ce440 0x7f41dc15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:21.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.267+0000 7f41daffd640 1 --2- 172.21.15.76:0/3912396489 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41dc06eec0 0x7f41dc1518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:21.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.267+0000 7f41db7fe640 1 --2- 172.21.15.76:0/3912396489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41dc15aa80 0x7f41dc159330 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f41d400c480 tx=0x7f41d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:21.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.267+0000 7f41c3fff640 1 -- 172.21.15.76:0/3912396489 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4018070 con 0x7f41dc15aa80 2024-03-08T06:01:21.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.267+0000 7f41c3fff640 1 -- 172.21.15.76:0/3912396489 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d4007500 con 0x7f41dc15aa80 2024-03-08T06:01:21.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.267+0000 7f41c3fff640 1 -- 172.21.15.76:0/3912396489 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4004a20 con 0x7f41dc15aa80 2024-03-08T06:01:21.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.267+0000 7f41e1888640 1 -- 172.21.15.76:0/3912396489 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41dc15d890 con 0x7f41dc15aa80 2024-03-08T06:01:21.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.267+0000 7f41e1888640 1 -- 172.21.15.76:0/3912396489 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41dc15d5b0 con 0x7f41dc15aa80 2024-03-08T06:01:21.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.268+0000 7f41e1888640 1 -- 172.21.15.76:0/3912396489 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41dc04f030 con 0x7f41dc15aa80 2024-03-08T06:01:21.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.268+0000 7f41c3fff640 1 -- 172.21.15.76:0/3912396489 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41d401b070 con 0x7f41dc15aa80 2024-03-08T06:01:21.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.269+0000 7f41c3fff640 1 --2- 172.21.15.76:0/3912396489 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f41c806b4d0 0x7f41c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.269+0000 7f41c3fff640 1 -- 172.21.15.76:0/3912396489 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f41d40898a0 con 0x7f41dc15aa80 2024-03-08T06:01:21.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.269+0000 7f41daffd640 1 --2- 172.21.15.76:0/3912396489 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f41c806b4d0 0x7f41c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.269+0000 7f41daffd640 1 --2- 172.21.15.76:0/3912396489 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f41c806b4d0 0x7f41c806d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f41cc036000 tx=0x7f41cc030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:21.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.273+0000 7f41c3fff640 1 -- 172.21.15.76:0/3912396489 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f41d40583b0 con 0x7f41dc15aa80 2024-03-08T06:01:21.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.444+0000 7f41e1888640 1 -- 172.21.15.76:0/3912396489 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41dc068640 con 0x7f41dc15aa80 2024-03-08T06:01:21.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.444+0000 7f41c3fff640 1 -- 172.21.15.76:0/3912396489 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41d4057cb0 con 0x7f41dc15aa80 2024-03-08T06:01:21.445 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:01:21.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.446+0000 7f41e1888640 1 -- 172.21.15.76:0/3912396489 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f41c806b4d0 msgr2=0x7f41c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.446+0000 7f41e1888640 1 --2- 172.21.15.76:0/3912396489 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f41c806b4d0 0x7f41c806d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f41cc036000 tx=0x7f41cc030000 comp rx=0 tx=0).stop 2024-03-08T06:01:21.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.446+0000 7f41e1888640 1 -- 172.21.15.76:0/3912396489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41dc15aa80 msgr2=0x7f41dc159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.446+0000 7f41e1888640 1 --2- 172.21.15.76:0/3912396489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41dc15aa80 0x7f41dc159330 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f41d400c480 tx=0x7f41d400c940 comp rx=0 tx=0).stop 2024-03-08T06:01:21.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.446+0000 7f41e1888640 1 -- 172.21.15.76:0/3912396489 shutdown_connections 2024-03-08T06:01:21.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.446+0000 7f41e1888640 1 --2- 172.21.15.76:0/3912396489 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f41c806b4d0 0x7f41c806d990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.446+0000 7f41e1888640 1 --2- 172.21.15.76:0/3912396489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41dc15aa80 0x7f41dc159330 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.446+0000 7f41e1888640 1 --2- 172.21.15.76:0/3912396489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41dc0ce440 0x7f41dc15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.447+0000 7f41e1888640 1 --2- 172.21.15.76:0/3912396489 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41dc06eec0 0x7f41dc1518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.447+0000 7f41e1888640 1 -- 172.21.15.76:0/3912396489 >> 172.21.15.76:0/3912396489 conn(0x7f41dc099a60 msgr2=0x7f41dc0adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:21.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.447+0000 7f41e1888640 1 -- 172.21.15.76:0/3912396489 shutdown_connections 2024-03-08T06:01:21.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.447+0000 7f41e1888640 1 -- 172.21.15.76:0/3912396489 wait complete. 2024-03-08T06:01:21.458 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:01:21.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.561+0000 7f1f37925640 1 Processor -- start 2024-03-08T06:01:21.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.561+0000 7f1f37925640 1 -- start start 2024-03-08T06:01:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f37925640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1f300ce5e0 0x7f1f3006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f37925640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f30161840 con 0x7f1f300ce9c0 2024-03-08T06:01:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f37925640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f3015eb30 con 0x7f1f3015dfb0 2024-03-08T06:01:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f37925640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f30161b90 con 0x7f1f300ce5e0 2024-03-08T06:01:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f34ec0640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1f300ce9c0 0x7f1f30162020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39478/0 (socket says 172.21.15.76:39478) 2024-03-08T06:01:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f34ec0640 1 -- 172.21.15.76:0/3881341752 learned_addr learned my addr 172.21.15.76:0/3881341752 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f356c1640 1 --2- 172.21.15.76:0/3881341752 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1f300ce5e0 0x7f1f3006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f27fff640 1 -- 172.21.15.76:0/3881341752 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2533435469 0 0) 0x7f1f30161840 con 0x7f1f300ce9c0 2024-03-08T06:01:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f27fff640 1 -- 172.21.15.76:0/3881341752 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f1c003520 con 0x7f1f300ce9c0 2024-03-08T06:01:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f27fff640 1 -- 172.21.15.76:0/3881341752 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1052856297 0 0) 0x7f1f1c003520 con 0x7f1f300ce9c0 2024-03-08T06:01:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f27fff640 1 -- 172.21.15.76:0/3881341752 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1f300ce5e0 msgr2=0x7f1f3006d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f27fff640 1 --2- 172.21.15.76:0/3881341752 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1f300ce5e0 0x7f1f3006d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f27fff640 1 -- 172.21.15.76:0/3881341752 >> v1:172.21.15.81:6789/0 conn(0x7f1f3015dfb0 legacy=0x7f1f3006bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f27fff640 1 -- 172.21.15.76:0/3881341752 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3015ee10 con 0x7f1f300ce9c0 2024-03-08T06:01:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f27fff640 1 -- 172.21.15.76:0/3881341752 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f1f20001670 con 0x7f1f300ce9c0 2024-03-08T06:01:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f27fff640 1 -- 172.21.15.76:0/3881341752 >> v1:172.21.15.76:6789/0 conn(0x7f1f300ce9c0 legacy=0x7f1f30162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f27fff640 1 --2- 172.21.15.76:0/3881341752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f1c003a50 0x7f1f1c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f27fff640 1 --2- 172.21.15.76:0/3881341752 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1f300ce5e0 0x7f1f1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f27fff640 1 --2- 172.21.15.76:0/3881341752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f1c004d40 0x7f1f1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f27fff640 1 -- 172.21.15.76:0/3881341752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f30161b90 con 0x7f1f1c004d40 2024-03-08T06:01:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f27fff640 1 -- 172.21.15.76:0/3881341752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c00bb60 con 0x7f1f1c003a50 2024-03-08T06:01:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.562+0000 7f1f27fff640 1 -- 172.21.15.76:0/3881341752 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c00bd60 con 0x7f1f300ce5e0 2024-03-08T06:01:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.563+0000 7f1f35ec2640 1 --2- 172.21.15.76:0/3881341752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f1c004d40 0x7f1f1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.563+0000 7f1f356c1640 1 --2- 172.21.15.76:0/3881341752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f1c003a50 0x7f1f1c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.563+0000 7f1f34ec0640 1 --2- 172.21.15.76:0/3881341752 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1f300ce5e0 0x7f1f1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.563+0000 7f1f35ec2640 1 -- 172.21.15.76:0/3881341752 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1f300ce5e0 msgr2=0x7f1f1c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.563+0000 7f1f35ec2640 1 --2- 172.21.15.76:0/3881341752 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1f300ce5e0 0x7f1f1c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.563+0000 7f1f35ec2640 1 -- 172.21.15.76:0/3881341752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f1c003a50 msgr2=0x7f1f1c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.563+0000 7f1f35ec2640 1 --2- 172.21.15.76:0/3881341752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f1c003a50 0x7f1f1c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.563+0000 7f1f35ec2640 1 -- 172.21.15.76:0/3881341752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c003520 con 0x7f1f1c004d40 2024-03-08T06:01:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.564+0000 7f1f35ec2640 1 --2- 172.21.15.76:0/3881341752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f1c004d40 0x7f1f1c0051c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1f28001e20 tx=0x7f1f1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6846fd59f82bfd39 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.564+0000 7f1f27fff640 1 -- 172.21.15.76:0/3881341752 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28001a70 con 0x7f1f1c004d40 2024-03-08T06:01:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.564+0000 7f1f27fff640 1 -- 172.21.15.76:0/3881341752 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f28003b70 con 0x7f1f1c004d40 2024-03-08T06:01:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.564+0000 7f1f27fff640 1 -- 172.21.15.76:0/3881341752 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f280050d0 con 0x7f1f1c004d40 2024-03-08T06:01:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.564+0000 7f1f37925640 1 -- 172.21.15.76:0/3881341752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f1c004d40 msgr2=0x7f1f1c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.564+0000 7f1f37925640 1 --2- 172.21.15.76:0/3881341752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f1c004d40 0x7f1f1c0051c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1f28001e20 tx=0x7f1f1c005830 comp rx=0 tx=0).stop 2024-03-08T06:01:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.564+0000 7f1f35ec2640 1 -- 172.21.15.76:0/3881341752 reap_dead start 2024-03-08T06:01:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.564+0000 7f1f37925640 1 -- 172.21.15.76:0/3881341752 shutdown_connections 2024-03-08T06:01:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.564+0000 7f1f37925640 1 -- 172.21.15.76:0/3881341752 >> 172.21.15.76:0/3881341752 conn(0x7f1f3006ae60 msgr2=0x7f1f3006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:21.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.564+0000 7f1f37925640 1 -- 172.21.15.76:0/3881341752 shutdown_connections 2024-03-08T06:01:21.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.564+0000 7f1f37925640 1 -- 172.21.15.76:0/3881341752 wait complete. 2024-03-08T06:01:21.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.565+0000 7f1f37925640 1 Processor -- start 2024-03-08T06:01:21.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.565+0000 7f1f37925640 1 -- start start 2024-03-08T06:01:21.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.566+0000 7f1f37925640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1f301f7360 0x7f1f300c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.566+0000 7f1f37925640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f300c6670 0x7f1f30151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.566+0000 7f1f37925640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f30151a40 0x7f1f30151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.566+0000 7f1f37925640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f300cee20 con 0x7f1f300c6670 2024-03-08T06:01:21.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.566+0000 7f1f37925640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f30162730 con 0x7f1f30151a40 2024-03-08T06:01:21.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.566+0000 7f1f37925640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3006c6b0 con 0x7f1f301f7360 2024-03-08T06:01:21.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.566+0000 7f1f34ec0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f300c6670 0x7f1f30151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.566+0000 7f1f34ec0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f300c6670 0x7f1f30151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41702/0 (socket says 172.21.15.76:41702) 2024-03-08T06:01:21.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.566+0000 7f1f34ec0640 1 -- 172.21.15.76:0/2103051351 learned_addr learned my addr 172.21.15.76:0/2103051351 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:21.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.567+0000 7f1f34ec0640 1 -- 172.21.15.76:0/2103051351 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1f301f7360 msgr2=0x7f1f300c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.567+0000 7f1f34ec0640 1 --2- 172.21.15.76:0/2103051351 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1f301f7360 0x7f1f300c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.567+0000 7f1f34ec0640 1 -- 172.21.15.76:0/2103051351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f30151a40 msgr2=0x7f1f30151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.567+0000 7f1f34ec0640 1 --2- 172.21.15.76:0/2103051351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f30151a40 0x7f1f30151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.567+0000 7f1f34ec0640 1 -- 172.21.15.76:0/2103051351 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f300c6ea0 con 0x7f1f300c6670 2024-03-08T06:01:21.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.567+0000 7f1f34ec0640 1 --2- 172.21.15.76:0/2103051351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f300c6670 0x7f1f30151500 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1f200072c0 tx=0x7f1f20007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:21.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.567+0000 7f1f25ffb640 1 -- 172.21.15.76:0/2103051351 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2000f860 con 0x7f1f300c6670 2024-03-08T06:01:21.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.567+0000 7f1f25ffb640 1 -- 172.21.15.76:0/2103051351 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f20018070 con 0x7f1f300c6670 2024-03-08T06:01:21.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.567+0000 7f1f37925640 1 -- 172.21.15.76:0/2103051351 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f300c7180 con 0x7f1f300c6670 2024-03-08T06:01:21.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.567+0000 7f1f25ffb640 1 -- 172.21.15.76:0/2103051351 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2000fc60 con 0x7f1f300c6670 2024-03-08T06:01:21.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.567+0000 7f1f37925640 1 -- 172.21.15.76:0/2103051351 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f300c7740 con 0x7f1f300c6670 2024-03-08T06:01:21.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.568+0000 7f1f37925640 1 -- 172.21.15.76:0/2103051351 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3004f030 con 0x7f1f300c6670 2024-03-08T06:01:21.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.568+0000 7f1f25ffb640 1 -- 172.21.15.76:0/2103051351 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f2001d460 con 0x7f1f300c6670 2024-03-08T06:01:21.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.569+0000 7f1f25ffb640 1 --2- 172.21.15.76:0/2103051351 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1f1406b4d0 0x7f1f1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.569+0000 7f1f25ffb640 1 -- 172.21.15.76:0/2103051351 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1f2008a6b0 con 0x7f1f300c6670 2024-03-08T06:01:21.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.569+0000 7f1f356c1640 1 --2- 172.21.15.76:0/2103051351 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1f1406b4d0 0x7f1f1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.569+0000 7f1f356c1640 1 --2- 172.21.15.76:0/2103051351 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1f1406b4d0 0x7f1f1406d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009ab0 tx=0x7f1f2c0095d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:21.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.573+0000 7f1f25ffb640 1 -- 172.21.15.76:0/2103051351 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f20059160 con 0x7f1f300c6670 2024-03-08T06:01:21.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.742+0000 7f1f37925640 1 -- 172.21.15.76:0/2103051351 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1f30161840 con 0x7f1f300c6670 2024-03-08T06:01:21.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.743+0000 7f1f25ffb640 1 -- 172.21.15.76:0/2103051351 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1f20058a60 con 0x7f1f300c6670 2024-03-08T06:01:21.758 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:21.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.744+0000 7f1f37925640 1 -- 172.21.15.76:0/2103051351 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1f1406b4d0 msgr2=0x7f1f1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.744+0000 7f1f37925640 1 --2- 172.21.15.76:0/2103051351 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1f1406b4d0 0x7f1f1406d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009ab0 tx=0x7f1f2c0095d0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.744+0000 7f1f37925640 1 -- 172.21.15.76:0/2103051351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f300c6670 msgr2=0x7f1f30151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.744+0000 7f1f37925640 1 --2- 172.21.15.76:0/2103051351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f300c6670 0x7f1f30151500 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1f200072c0 tx=0x7f1f20007780 comp rx=0 tx=0).stop 2024-03-08T06:01:21.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.745+0000 7f1f37925640 1 -- 172.21.15.76:0/2103051351 shutdown_connections 2024-03-08T06:01:21.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.745+0000 7f1f37925640 1 --2- 172.21.15.76:0/2103051351 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1f1406b4d0 0x7f1f1406d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.745+0000 7f1f37925640 1 --2- 172.21.15.76:0/2103051351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f30151a40 0x7f1f30151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.745+0000 7f1f37925640 1 --2- 172.21.15.76:0/2103051351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f300c6670 0x7f1f30151500 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.745+0000 7f1f37925640 1 --2- 172.21.15.76:0/2103051351 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1f301f7360 0x7f1f300c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.745+0000 7f1f37925640 1 -- 172.21.15.76:0/2103051351 >> 172.21.15.76:0/2103051351 conn(0x7f1f3006ae60 msgr2=0x7f1f3015fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:21.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.745+0000 7f1f37925640 1 -- 172.21.15.76:0/2103051351 shutdown_connections 2024-03-08T06:01:21.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.745+0000 7f1f37925640 1 -- 172.21.15.76:0/2103051351 wait complete. 2024-03-08T06:01:21.760 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:01:21.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.863+0000 7f5371781640 1 Processor -- start 2024-03-08T06:01:21.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.863+0000 7f5371781640 1 -- start start 2024-03-08T06:01:21.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.864+0000 7f5371781640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f536c15dbc0 0x7f536c0d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.864+0000 7f5371781640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f536c0b9840 0x7f536c0b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.864+0000 7f5371781640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f536c0b5b20 0x7f536c06c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.864+0000 7f5371781640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536c0b5f90 con 0x7f536c0b5b20 2024-03-08T06:01:21.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.864+0000 7f5371781640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536c0d27d0 con 0x7f536c0b9840 2024-03-08T06:01:21.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.864+0000 7f5371781640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c0d2ab0 con 0x7f536c15dbc0 2024-03-08T06:01:21.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.864+0000 7f536b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f536c0b5b20 0x7f536c06c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.864+0000 7f536b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f536c0b5b20 0x7f536c06c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41706/0 (socket says 172.21.15.76:41706) 2024-03-08T06:01:21.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.864+0000 7f536b7fe640 1 -- 172.21.15.76:0/1321798235 learned_addr learned my addr 172.21.15.76:0/1321798235 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:21.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.864+0000 7f536b7fe640 1 -- 172.21.15.76:0/1321798235 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f536c15dbc0 msgr2=0x7f536c0d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:21.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.864+0000 7f536b7fe640 1 --2- 172.21.15.76:0/1321798235 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f536c15dbc0 0x7f536c0d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.864+0000 7f536b7fe640 1 -- 172.21.15.76:0/1321798235 >> v2:172.21.15.81:3300/0 conn(0x7f536c0b9840 msgr2=0x7f536c0b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:21.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.864+0000 7f536b7fe640 1 --2- 172.21.15.76:0/1321798235 >> v2:172.21.15.81:3300/0 conn(0x7f536c0b9840 0x7f536c0b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.864+0000 7f536b7fe640 1 -- 172.21.15.76:0/1321798235 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c0d2d90 con 0x7f536c0b5b20 2024-03-08T06:01:21.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.865+0000 7f536b7fe640 1 --2- 172.21.15.76:0/1321798235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c0b5b20 0x7f536c06c190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f536c0d29e0 tx=0x7f536402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818dfb3e9a9dec00 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.865+0000 7f5369ffb640 1 -- 172.21.15.76:0/1321798235 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364035020 con 0x7f536c0b5b20 2024-03-08T06:01:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.865+0000 7f5369ffb640 1 -- 172.21.15.76:0/1321798235 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5364031640 con 0x7f536c0b5b20 2024-03-08T06:01:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.865+0000 7f5369ffb640 1 -- 172.21.15.76:0/1321798235 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536402d040 con 0x7f536c0b5b20 2024-03-08T06:01:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.866+0000 7f5371781640 1 -- 172.21.15.76:0/1321798235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c0b5b20 msgr2=0x7f536c06c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.866+0000 7f5371781640 1 --2- 172.21.15.76:0/1321798235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c0b5b20 0x7f536c06c190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f536c0d29e0 tx=0x7f536402b8f0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.866+0000 7f5371781640 1 -- 172.21.15.76:0/1321798235 shutdown_connections 2024-03-08T06:01:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.866+0000 7f5371781640 1 --2- 172.21.15.76:0/1321798235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c0b5b20 0x7f536c06c190 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.866+0000 7f5371781640 1 --2- 172.21.15.76:0/1321798235 >> v2:172.21.15.81:3300/0 conn(0x7f536c0b9840 0x7f536c0b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.866+0000 7f5371781640 1 --2- 172.21.15.76:0/1321798235 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f536c15dbc0 0x7f536c0d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.866+0000 7f5371781640 1 -- 172.21.15.76:0/1321798235 >> 172.21.15.76:0/1321798235 conn(0x7f536c0670d0 msgr2=0x7f536c0d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.866+0000 7f5371781640 1 -- 172.21.15.76:0/1321798235 shutdown_connections 2024-03-08T06:01:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.866+0000 7f5371781640 1 -- 172.21.15.76:0/1321798235 wait complete. 2024-03-08T06:01:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.867+0000 7f5371781640 1 Processor -- start 2024-03-08T06:01:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.867+0000 7f5371781640 1 -- start start 2024-03-08T06:01:21.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f5371781640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c0b5b20 0x7f536c158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f5371781640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f536c0b9840 0x7f536c0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f5371781640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f536c15dbc0 0x7f536c1f6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f5371781640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c0d2430 con 0x7f536c0b5b20 2024-03-08T06:01:21.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f5371781640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c0d3870 con 0x7f536c15dbc0 2024-03-08T06:01:21.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f5371781640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c0d25b0 con 0x7f536c0b9840 2024-03-08T06:01:21.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f536affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c0b5b20 0x7f536c158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f536affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c0b5b20 0x7f536c158980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41708/0 (socket says 172.21.15.76:41708) 2024-03-08T06:01:21.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f536affd640 1 -- 172.21.15.76:0/4224498229 learned_addr learned my addr 172.21.15.76:0/4224498229 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f536b7fe640 1 --2- 172.21.15.76:0/4224498229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f536c15dbc0 0x7f536c1f6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f536a7fc640 1 --2- 172.21.15.76:0/4224498229 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f536c0b9840 0x7f536c0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f536affd640 1 -- 172.21.15.76:0/4224498229 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f536c0b9840 msgr2=0x7f536c0cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f536affd640 1 --2- 172.21.15.76:0/4224498229 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f536c0b9840 0x7f536c0cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f536affd640 1 -- 172.21.15.76:0/4224498229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f536c15dbc0 msgr2=0x7f536c1f6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f536affd640 1 --2- 172.21.15.76:0/4224498229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f536c15dbc0 0x7f536c1f6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f536affd640 1 -- 172.21.15.76:0/4224498229 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c158f80 con 0x7f536c0b5b20 2024-03-08T06:01:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.868+0000 7f536affd640 1 --2- 172.21.15.76:0/4224498229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c0b5b20 0x7f536c158980 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f535c00ba50 tx=0x7f535c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.869+0000 7f53537fe640 1 -- 172.21.15.76:0/4224498229 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c019070 con 0x7f536c0b5b20 2024-03-08T06:01:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.869+0000 7f53537fe640 1 -- 172.21.15.76:0/4224498229 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535c00cc00 con 0x7f536c0b5b20 2024-03-08T06:01:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.869+0000 7f5371781640 1 -- 172.21.15.76:0/4224498229 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536c159260 con 0x7f536c0b5b20 2024-03-08T06:01:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.869+0000 7f53537fe640 1 -- 172.21.15.76:0/4224498229 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c014440 con 0x7f536c0b5b20 2024-03-08T06:01:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.869+0000 7f5371781640 1 -- 172.21.15.76:0/4224498229 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536c159820 con 0x7f536c0b5b20 2024-03-08T06:01:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.870+0000 7f5371781640 1 -- 172.21.15.76:0/4224498229 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536c04f030 con 0x7f536c0b5b20 2024-03-08T06:01:21.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.870+0000 7f53537fe640 1 -- 172.21.15.76:0/4224498229 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f535c01c070 con 0x7f536c0b5b20 2024-03-08T06:01:21.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.870+0000 7f53537fe640 1 --2- 172.21.15.76:0/4224498229 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f535806b4d0 0x7f535806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:21.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.871+0000 7f53537fe640 1 -- 172.21.15.76:0/4224498229 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f535c08a3e0 con 0x7f536c0b5b20 2024-03-08T06:01:21.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.871+0000 7f536a7fc640 1 --2- 172.21.15.76:0/4224498229 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f535806b4d0 0x7f535806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:21.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.871+0000 7f536a7fc640 1 --2- 172.21.15.76:0/4224498229 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f535806b4d0 0x7f535806d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f536c147850 tx=0x7f5360008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:21.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:21.875+0000 7f53537fe640 1 -- 172.21.15.76:0/4224498229 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f535c058e60 con 0x7f536c0b5b20 2024-03-08T06:01:22.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.045+0000 7f5371781640 1 -- 172.21.15.76:0/4224498229 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f536c0d2210 con 0x7f536c0b5b20 2024-03-08T06:01:22.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.045+0000 7f53537fe640 1 -- 172.21.15.76:0/4224498229 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f535c058760 con 0x7f536c0b5b20 2024-03-08T06:01:22.046 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:01:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.047+0000 7f5371781640 1 -- 172.21.15.76:0/4224498229 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f535806b4d0 msgr2=0x7f535806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.047+0000 7f5371781640 1 --2- 172.21.15.76:0/4224498229 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f535806b4d0 0x7f535806d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f536c147850 tx=0x7f5360008040 comp rx=0 tx=0).stop 2024-03-08T06:01:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.047+0000 7f5371781640 1 -- 172.21.15.76:0/4224498229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c0b5b20 msgr2=0x7f536c158980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.047+0000 7f5371781640 1 --2- 172.21.15.76:0/4224498229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c0b5b20 0x7f536c158980 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f535c00ba50 tx=0x7f535c00bf10 comp rx=0 tx=0).stop 2024-03-08T06:01:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.047+0000 7f5371781640 1 -- 172.21.15.76:0/4224498229 shutdown_connections 2024-03-08T06:01:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.047+0000 7f5371781640 1 --2- 172.21.15.76:0/4224498229 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f535806b4d0 0x7f535806d990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.047+0000 7f5371781640 1 --2- 172.21.15.76:0/4224498229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f536c15dbc0 0x7f536c1f6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.047+0000 7f5371781640 1 --2- 172.21.15.76:0/4224498229 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f536c0b9840 0x7f536c0cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.047+0000 7f5371781640 1 --2- 172.21.15.76:0/4224498229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c0b5b20 0x7f536c158980 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.047+0000 7f5371781640 1 -- 172.21.15.76:0/4224498229 >> 172.21.15.76:0/4224498229 conn(0x7f536c0670d0 msgr2=0x7f536c0d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.047+0000 7f5371781640 1 -- 172.21.15.76:0/4224498229 shutdown_connections 2024-03-08T06:01:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.047+0000 7f5371781640 1 -- 172.21.15.76:0/4224498229 wait complete. 2024-03-08T06:01:22.058 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T06:01:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.161+0000 7f3a32783640 1 Processor -- start 2024-03-08T06:01:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.161+0000 7f3a32783640 1 -- start start 2024-03-08T06:01:22.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.162+0000 7f3a32783640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3a2c0af950 0x7f3a2c06fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.162+0000 7f3a32783640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3a2c0afde0 0x7f3a2c070040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.162+0000 7f3a32783640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2c0cd580 con 0x7f3a2c15dcb0 2024-03-08T06:01:22.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.162+0000 7f3a32783640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2c0cd860 con 0x7f3a2c0af950 2024-03-08T06:01:22.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.162+0000 7f3a32783640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2c0cdb40 con 0x7f3a2c0afde0 2024-03-08T06:01:22.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.162+0000 7f3a2b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3a2c0af950 0x7f3a2c06fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.162+0000 7f3a2b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3a2c0af950 0x7f3a2c06fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41718/0 (socket says 172.21.15.76:41718) 2024-03-08T06:01:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.162+0000 7f3a2bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3a2c15dcb0 0x7f3a2c0ad600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39494/0 (socket says 172.21.15.76:39494) 2024-03-08T06:01:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.162+0000 7f3a2b7fe640 1 -- 172.21.15.76:0/1099309687 learned_addr learned my addr 172.21.15.76:0/1099309687 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.162+0000 7f3a30d20640 1 --2- 172.21.15.76:0/1099309687 >> v2:172.21.15.81:3300/0 conn(0x7f3a2c0afde0 0x7f3a2c070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.162+0000 7f3a2b7fe640 1 -- 172.21.15.76:0/1099309687 >> v2:172.21.15.81:3300/0 conn(0x7f3a2c0afde0 msgr2=0x7f3a2c070040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.162+0000 7f3a2b7fe640 1 --2- 172.21.15.76:0/1099309687 >> v2:172.21.15.81:3300/0 conn(0x7f3a2c0afde0 0x7f3a2c070040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.162+0000 7f3a2b7fe640 1 -- 172.21.15.76:0/1099309687 >> v1:172.21.15.76:6789/0 conn(0x7f3a2c15dcb0 legacy=0x7f3a2c0ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.162+0000 7f3a2b7fe640 1 -- 172.21.15.76:0/1099309687 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c0cde20 con 0x7f3a2c0af950 2024-03-08T06:01:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.162+0000 7f3a30d20640 1 --2- 172.21.15.76:0/1099309687 >> v2:172.21.15.81:3300/0 conn(0x7f3a2c0afde0 0x7f3a2c070040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.163+0000 7f3a2b7fe640 1 --2- 172.21.15.76:0/1099309687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2c0af950 0x7f3a2c06fb00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c067490 tx=0x7f3a2002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e728fc28cedde3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.163+0000 7f3a2affd640 1 -- 172.21.15.76:0/1099309687 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20034070 con 0x7f3a2c0af950 2024-03-08T06:01:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.163+0000 7f3a2affd640 1 -- 172.21.15.76:0/1099309687 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a20030860 con 0x7f3a2c0af950 2024-03-08T06:01:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.163+0000 7f3a2affd640 1 -- 172.21.15.76:0/1099309687 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20030c30 con 0x7f3a2c0af950 2024-03-08T06:01:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.163+0000 7f3a32783640 1 -- 172.21.15.76:0/1099309687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2c0af950 msgr2=0x7f3a2c06fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.163+0000 7f3a32783640 1 --2- 172.21.15.76:0/1099309687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2c0af950 0x7f3a2c06fb00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c067490 tx=0x7f3a2002b9a0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.163+0000 7f3a32783640 1 -- 172.21.15.76:0/1099309687 shutdown_connections 2024-03-08T06:01:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.163+0000 7f3a32783640 1 --2- 172.21.15.76:0/1099309687 >> v2:172.21.15.81:3300/0 conn(0x7f3a2c0afde0 0x7f3a2c070040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.163+0000 7f3a32783640 1 --2- 172.21.15.76:0/1099309687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2c0af950 0x7f3a2c06fb00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.163+0000 7f3a32783640 1 -- 172.21.15.76:0/1099309687 >> 172.21.15.76:0/1099309687 conn(0x7f3a2c0b2320 msgr2=0x7f3a2c0b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.163+0000 7f3a32783640 1 -- 172.21.15.76:0/1099309687 shutdown_connections 2024-03-08T06:01:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.163+0000 7f3a32783640 1 -- 172.21.15.76:0/1099309687 wait complete. 2024-03-08T06:01:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.165+0000 7f3a32783640 1 Processor -- start 2024-03-08T06:01:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.165+0000 7f3a32783640 1 -- start start 2024-03-08T06:01:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.165+0000 7f3a32783640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2c0af950 0x7f3a2c0c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.165+0000 7f3a32783640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a2c0afde0 0x7f3a2c1f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.165+0000 7f3a32783640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3a2c15dcb0 0x7f3a2c0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.165+0000 7f3a32783640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c15e210 con 0x7f3a2c0af950 2024-03-08T06:01:22.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.165+0000 7f3a32783640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c15e090 con 0x7f3a2c0afde0 2024-03-08T06:01:22.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.165+0000 7f3a32783640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c06b040 con 0x7f3a2c15dcb0 2024-03-08T06:01:22.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.166+0000 7f3a2bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2c0af950 0x7f3a2c0c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.166+0000 7f3a2bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2c0af950 0x7f3a2c0c8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41720/0 (socket says 172.21.15.76:41720) 2024-03-08T06:01:22.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.166+0000 7f3a2bfff640 1 -- 172.21.15.76:0/3598654455 learned_addr learned my addr 172.21.15.76:0/3598654455 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:22.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.166+0000 7f3a2bfff640 1 -- 172.21.15.76:0/3598654455 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3a2c15dcb0 msgr2=0x7f3a2c0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.166+0000 7f3a2bfff640 1 --2- 172.21.15.76:0/3598654455 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3a2c15dcb0 0x7f3a2c0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.166+0000 7f3a2bfff640 1 -- 172.21.15.76:0/3598654455 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a2c0afde0 msgr2=0x7f3a2c1f7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.166+0000 7f3a2b7fe640 1 --2- 172.21.15.76:0/3598654455 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a2c0afde0 0x7f3a2c1f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.166+0000 7f3a2bfff640 1 --2- 172.21.15.76:0/3598654455 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a2c0afde0 0x7f3a2c1f7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.166+0000 7f3a2bfff640 1 -- 172.21.15.76:0/3598654455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c0c9580 con 0x7f3a2c0af950 2024-03-08T06:01:22.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.166+0000 7f3a2bfff640 1 --2- 172.21.15.76:0/3598654455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2c0af950 0x7f3a2c0c8f10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00c3c0 tx=0x7f3a1c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:22.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.166+0000 7f3a28ff9640 1 -- 172.21.15.76:0/3598654455 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c0074e0 con 0x7f3a2c0af950 2024-03-08T06:01:22.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.166+0000 7f3a28ff9640 1 -- 172.21.15.76:0/3598654455 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1c007680 con 0x7f3a2c0af950 2024-03-08T06:01:22.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.166+0000 7f3a28ff9640 1 -- 172.21.15.76:0/3598654455 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c004a00 con 0x7f3a2c0af950 2024-03-08T06:01:22.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.166+0000 7f3a32783640 1 -- 172.21.15.76:0/3598654455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2c0c9860 con 0x7f3a2c0af950 2024-03-08T06:01:22.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.166+0000 7f3a32783640 1 -- 172.21.15.76:0/3598654455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c0c9e20 con 0x7f3a2c0af950 2024-03-08T06:01:22.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.167+0000 7f3a32783640 1 -- 172.21.15.76:0/3598654455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2c0cf560 con 0x7f3a2c0af950 2024-03-08T06:01:22.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.167+0000 7f3a28ff9640 1 -- 172.21.15.76:0/3598654455 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a1c01a020 con 0x7f3a2c0af950 2024-03-08T06:01:22.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.168+0000 7f3a28ff9640 1 --2- 172.21.15.76:0/3598654455 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3a1806b4d0 0x7f3a1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.168+0000 7f3a28ff9640 1 -- 172.21.15.76:0/3598654455 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3a1c089ce0 con 0x7f3a2c0af950 2024-03-08T06:01:22.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.168+0000 7f3a2b7fe640 1 --2- 172.21.15.76:0/3598654455 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3a1806b4d0 0x7f3a1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.168+0000 7f3a2b7fe640 1 --2- 172.21.15.76:0/3598654455 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3a1806b4d0 0x7f3a1806d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a20036000 tx=0x7f3a2003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:22.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.172+0000 7f3a28ff9640 1 -- 172.21.15.76:0/3598654455 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a2c0cf560 con 0x7f3a2c0af950 2024-03-08T06:01:22.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.344+0000 7f3a32783640 1 -- 172.21.15.76:0/3598654455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3a2c04f030 con 0x7f3a2c0af950 2024-03-08T06:01:22.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.344+0000 7f3a28ff9640 1 -- 172.21.15.76:0/3598654455 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3a1c0587f0 con 0x7f3a2c0af950 2024-03-08T06:01:22.344 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:22.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.346+0000 7f3a32783640 1 -- 172.21.15.76:0/3598654455 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3a1806b4d0 msgr2=0x7f3a1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.346+0000 7f3a32783640 1 --2- 172.21.15.76:0/3598654455 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3a1806b4d0 0x7f3a1806d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a20036000 tx=0x7f3a2003b040 comp rx=0 tx=0).stop 2024-03-08T06:01:22.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.346+0000 7f3a32783640 1 -- 172.21.15.76:0/3598654455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2c0af950 msgr2=0x7f3a2c0c8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.346+0000 7f3a32783640 1 --2- 172.21.15.76:0/3598654455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2c0af950 0x7f3a2c0c8f10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00c3c0 tx=0x7f3a1c00c880 comp rx=0 tx=0).stop 2024-03-08T06:01:22.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.346+0000 7f3a32783640 1 -- 172.21.15.76:0/3598654455 shutdown_connections 2024-03-08T06:01:22.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.346+0000 7f3a32783640 1 --2- 172.21.15.76:0/3598654455 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3a1806b4d0 0x7f3a1806d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.346+0000 7f3a32783640 1 --2- 172.21.15.76:0/3598654455 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3a2c15dcb0 0x7f3a2c0b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.346+0000 7f3a32783640 1 --2- 172.21.15.76:0/3598654455 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a2c0afde0 0x7f3a2c1f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.346+0000 7f3a32783640 1 --2- 172.21.15.76:0/3598654455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2c0af950 0x7f3a2c0c8f10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.346+0000 7f3a32783640 1 -- 172.21.15.76:0/3598654455 >> 172.21.15.76:0/3598654455 conn(0x7f3a2c0b2320 msgr2=0x7f3a2c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:22.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.346+0000 7f3a32783640 1 -- 172.21.15.76:0/3598654455 shutdown_connections 2024-03-08T06:01:22.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.346+0000 7f3a32783640 1 -- 172.21.15.76:0/3598654455 wait complete. 2024-03-08T06:01:22.379 INFO:tasks.cephfs_test_runner:Starting test: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) That the per-client labelled perf counters depict the clients 2024-03-08T06:01:22.379 INFO:tasks.cephfs_test_runner:test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-08T06:01:22.379 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:22.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.478+0000 7f0ae45d0640 1 Processor -- start 2024-03-08T06:01:22.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.478+0000 7f0ae45d0640 1 -- start start 2024-03-08T06:01:22.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.479+0000 7f0ae45d0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0adc06d3f0 0x7f0adc0cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.479+0000 7f0ae45d0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0adc06d880 con 0x7f0adc0cd310 2024-03-08T06:01:22.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.479+0000 7f0ae45d0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0adc06e010 con 0x7f0adc071390 2024-03-08T06:01:22.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.479+0000 7f0ae45d0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc06e2f0 con 0x7f0adc06d3f0 2024-03-08T06:01:22.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.479+0000 7f0ae2b6d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0adc0cd310 0x7f0adc0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39510/0 (socket says 172.21.15.76:39510) 2024-03-08T06:01:22.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.479+0000 7f0ae2b6d640 1 -- 172.21.15.76:0/1995172721 learned_addr learned my addr 172.21.15.76:0/1995172721 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:22.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.479+0000 7f0ae1b6b640 1 --2- 172.21.15.76:0/1995172721 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0adc06d3f0 0x7f0adc0cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.479+0000 7f0ae136a640 1 -- 172.21.15.76:0/1995172721 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075338439 0 0) 0x7f0adc06d880 con 0x7f0adc0cd310 2024-03-08T06:01:22.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.479+0000 7f0ae136a640 1 -- 172.21.15.76:0/1995172721 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ac8003520 con 0x7f0adc0cd310 2024-03-08T06:01:22.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.479+0000 7f0ae136a640 1 -- 172.21.15.76:0/1995172721 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4209156009 0 0) 0x7f0adc06e010 con 0x7f0adc071390 2024-03-08T06:01:22.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae136a640 1 -- 172.21.15.76:0/1995172721 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0adc06d880 con 0x7f0adc071390 2024-03-08T06:01:22.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae136a640 1 -- 172.21.15.76:0/1995172721 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1771083798 0 0) 0x7f0ac8003520 con 0x7f0adc0cd310 2024-03-08T06:01:22.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae136a640 1 -- 172.21.15.76:0/1995172721 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0adc06d3f0 msgr2=0x7f0adc0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:22.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae136a640 1 --2- 172.21.15.76:0/1995172721 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0adc06d3f0 0x7f0adc0cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae136a640 1 -- 172.21.15.76:0/1995172721 >> v1:172.21.15.81:6789/0 conn(0x7f0adc071390 legacy=0x7f0adc071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae136a640 1 -- 172.21.15.76:0/1995172721 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0adc0ce7b0 con 0x7f0adc0cd310 2024-03-08T06:01:22.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae136a640 1 -- 172.21.15.76:0/1995172721 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f0ad4001670 con 0x7f0adc0cd310 2024-03-08T06:01:22.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae136a640 1 -- 172.21.15.76:0/1995172721 >> v1:172.21.15.76:6789/0 conn(0x7f0adc0cd310 legacy=0x7f0adc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae136a640 1 --2- 172.21.15.76:0/1995172721 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0ac8003be0 0x7f0ac8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae136a640 1 --2- 172.21.15.76:0/1995172721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ac8004800 0x7f0ac8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae136a640 1 --2- 172.21.15.76:0/1995172721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ac80051c0 0x7f0ac800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae136a640 1 -- 172.21.15.76:0/1995172721 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc06e2f0 con 0x7f0ac80051c0 2024-03-08T06:01:22.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae136a640 1 -- 172.21.15.76:0/1995172721 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac800bf00 con 0x7f0ac8004800 2024-03-08T06:01:22.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae136a640 1 -- 172.21.15.76:0/1995172721 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac800c100 con 0x7f0ac8003be0 2024-03-08T06:01:22.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae2b6d640 1 --2- 172.21.15.76:0/1995172721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ac80051c0 0x7f0ac800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae236c640 1 --2- 172.21.15.76:0/1995172721 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0ac8003be0 0x7f0ac8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae1b6b640 1 --2- 172.21.15.76:0/1995172721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ac8004800 0x7f0ac8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae2b6d640 1 -- 172.21.15.76:0/1995172721 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0ac8003be0 msgr2=0x7f0ac8003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:22.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae2b6d640 1 --2- 172.21.15.76:0/1995172721 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0ac8003be0 0x7f0ac8003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae2b6d640 1 -- 172.21.15.76:0/1995172721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ac8004800 msgr2=0x7f0ac8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:22.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae2b6d640 1 --2- 172.21.15.76:0/1995172721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ac8004800 0x7f0ac8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.480+0000 7f0ae2b6d640 1 -- 172.21.15.76:0/1995172721 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac8003520 con 0x7f0ac80051c0 2024-03-08T06:01:22.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.481+0000 7f0ae2b6d640 1 --2- 172.21.15.76:0/1995172721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ac80051c0 0x7f0ac800b9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400c1c0 tx=0x7f0ac8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269cf9d47d389fd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:22.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.481+0000 7f0ae136a640 1 -- 172.21.15.76:0/1995172721 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4007cc0 con 0x7f0ac80051c0 2024-03-08T06:01:22.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.481+0000 7f0ae136a640 1 -- 172.21.15.76:0/1995172721 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad4030070 con 0x7f0ac80051c0 2024-03-08T06:01:22.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.481+0000 7f0ae136a640 1 -- 172.21.15.76:0/1995172721 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4005d10 con 0x7f0ac80051c0 2024-03-08T06:01:22.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.481+0000 7f0ae45d0640 1 -- 172.21.15.76:0/1995172721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ac80051c0 msgr2=0x7f0ac800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.481+0000 7f0ae45d0640 1 --2- 172.21.15.76:0/1995172721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ac80051c0 0x7f0ac800b9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400c1c0 tx=0x7f0ac8005830 comp rx=0 tx=0).stop 2024-03-08T06:01:22.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.481+0000 7f0ae2b6d640 1 -- 172.21.15.76:0/1995172721 reap_dead start 2024-03-08T06:01:22.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.481+0000 7f0ae45d0640 1 -- 172.21.15.76:0/1995172721 shutdown_connections 2024-03-08T06:01:22.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.481+0000 7f0ae45d0640 1 -- 172.21.15.76:0/1995172721 >> 172.21.15.76:0/1995172721 conn(0x7f0adc0cf740 msgr2=0x7f0adc0cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:22.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.481+0000 7f0ae45d0640 1 -- 172.21.15.76:0/1995172721 shutdown_connections 2024-03-08T06:01:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.481+0000 7f0ae45d0640 1 -- 172.21.15.76:0/1995172721 wait complete. 2024-03-08T06:01:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.483+0000 7f0ae45d0640 1 Processor -- start 2024-03-08T06:01:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.483+0000 7f0ae45d0640 1 -- start start 2024-03-08T06:01:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.483+0000 7f0ae45d0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ac80051c0 0x7f0adc0c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.483+0000 7f0ae45d0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0adc149be0 0x7f0adc14a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.483+0000 7f0ae45d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0adc0c9f90 0x7f0adc0ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.483+0000 7f0ae45d0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc15c060 con 0x7f0adc0c9f90 2024-03-08T06:01:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.483+0000 7f0ae45d0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc15bee0 con 0x7f0ac80051c0 2024-03-08T06:01:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.483+0000 7f0ae45d0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc0cf0e0 con 0x7f0adc149be0 2024-03-08T06:01:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.483+0000 7f0ae2b6d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0adc0c9f90 0x7f0adc0ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.483+0000 7f0ae2b6d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0adc0c9f90 0x7f0adc0ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41744/0 (socket says 172.21.15.76:41744) 2024-03-08T06:01:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.483+0000 7f0ae2b6d640 1 -- 172.21.15.76:0/3287125822 learned_addr learned my addr 172.21.15.76:0/3287125822 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.484+0000 7f0ae236c640 1 --2- 172.21.15.76:0/3287125822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ac80051c0 0x7f0adc0c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.484+0000 7f0ae2b6d640 1 -- 172.21.15.76:0/3287125822 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0adc149be0 msgr2=0x7f0adc14a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.484+0000 7f0ae1b6b640 1 --2- 172.21.15.76:0/3287125822 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0adc149be0 0x7f0adc14a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.484+0000 7f0ae2b6d640 1 --2- 172.21.15.76:0/3287125822 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0adc149be0 0x7f0adc14a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.484+0000 7f0ae2b6d640 1 -- 172.21.15.76:0/3287125822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ac80051c0 msgr2=0x7f0adc0c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.484+0000 7f0ae2b6d640 1 --2- 172.21.15.76:0/3287125822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ac80051c0 0x7f0adc0c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.484+0000 7f0ae2b6d640 1 -- 172.21.15.76:0/3287125822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0adc0cbd60 con 0x7f0adc0c9f90 2024-03-08T06:01:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.484+0000 7f0ae2b6d640 1 --2- 172.21.15.76:0/3287125822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0adc0c9f90 0x7f0adc0ca3f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400c2e0 tx=0x7f0ad4001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.484+0000 7f0ad2ffd640 1 -- 172.21.15.76:0/3287125822 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4046070 con 0x7f0adc0c9f90 2024-03-08T06:01:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.484+0000 7f0ad2ffd640 1 -- 172.21.15.76:0/3287125822 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad4005b20 con 0x7f0adc0c9f90 2024-03-08T06:01:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.484+0000 7f0ae45d0640 1 -- 172.21.15.76:0/3287125822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0adc0cacc0 con 0x7f0adc0c9f90 2024-03-08T06:01:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.484+0000 7f0ad2ffd640 1 -- 172.21.15.76:0/3287125822 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4001c10 con 0x7f0adc0c9f90 2024-03-08T06:01:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.484+0000 7f0ae45d0640 1 -- 172.21.15.76:0/3287125822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0adc0cb280 con 0x7f0adc0c9f90 2024-03-08T06:01:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.485+0000 7f0ae45d0640 1 -- 172.21.15.76:0/3287125822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0adc04f030 con 0x7f0adc0c9f90 2024-03-08T06:01:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.485+0000 7f0ad2ffd640 1 -- 172.21.15.76:0/3287125822 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ad4049070 con 0x7f0adc0c9f90 2024-03-08T06:01:22.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.486+0000 7f0ad2ffd640 1 --2- 172.21.15.76:0/3287125822 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0ac006b4d0 0x7f0ac006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.486+0000 7f0ae236c640 1 --2- 172.21.15.76:0/3287125822 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0ac006b4d0 0x7f0ac006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.486+0000 7f0ad2ffd640 1 -- 172.21.15.76:0/3287125822 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0ad40ab820 con 0x7f0adc0c9f90 2024-03-08T06:01:22.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.486+0000 7f0ae236c640 1 --2- 172.21.15.76:0/3287125822 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0ac006b4d0 0x7f0ac006d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8003e90 tx=0x7f0ad8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:22.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.490+0000 7f0ad2ffd640 1 -- 172.21.15.76:0/3287125822 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ad407a210 con 0x7f0adc0c9f90 2024-03-08T06:01:22.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.661+0000 7f0ae45d0640 1 -- 172.21.15.76:0/3287125822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0adc04f030 con 0x7f0adc0c9f90 2024-03-08T06:01:22.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.662+0000 7f0ad2ffd640 1 -- 172.21.15.76:0/3287125822 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18757 (secure 0 0 0) 0x7f0ad4079b10 con 0x7f0adc0c9f90 2024-03-08T06:01:22.663 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:22.663 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":28,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:01:17.313272+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:22.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.664+0000 7f0ae45d0640 1 -- 172.21.15.76:0/3287125822 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0ac006b4d0 msgr2=0x7f0ac006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.664+0000 7f0ae45d0640 1 --2- 172.21.15.76:0/3287125822 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0ac006b4d0 0x7f0ac006d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8003e90 tx=0x7f0ad8009040 comp rx=0 tx=0).stop 2024-03-08T06:01:22.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.664+0000 7f0ae45d0640 1 -- 172.21.15.76:0/3287125822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0adc0c9f90 msgr2=0x7f0adc0ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.664+0000 7f0ae45d0640 1 --2- 172.21.15.76:0/3287125822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0adc0c9f90 0x7f0adc0ca3f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400c2e0 tx=0x7f0ad4001a30 comp rx=0 tx=0).stop 2024-03-08T06:01:22.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.665+0000 7f0ae45d0640 1 -- 172.21.15.76:0/3287125822 shutdown_connections 2024-03-08T06:01:22.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.665+0000 7f0ae45d0640 1 --2- 172.21.15.76:0/3287125822 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0ac006b4d0 0x7f0ac006d990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.665+0000 7f0ae45d0640 1 --2- 172.21.15.76:0/3287125822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0adc0c9f90 0x7f0adc0ca3f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.665+0000 7f0ae45d0640 1 --2- 172.21.15.76:0/3287125822 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0adc149be0 0x7f0adc14a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.665+0000 7f0ae45d0640 1 --2- 172.21.15.76:0/3287125822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ac80051c0 0x7f0adc0c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.665+0000 7f0ae45d0640 1 -- 172.21.15.76:0/3287125822 >> 172.21.15.76:0/3287125822 conn(0x7f0adc0cf740 msgr2=0x7f0adc0b5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:22.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.665+0000 7f0ae45d0640 1 -- 172.21.15.76:0/3287125822 shutdown_connections 2024-03-08T06:01:22.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.665+0000 7f0ae45d0640 1 -- 172.21.15.76:0/3287125822 wait complete. 2024-03-08T06:01:22.676 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:00:33.926651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:00:34.474030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T06:00:38.207853+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T06:01:22.676 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:01:22.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.779+0000 7f9136dd0640 1 Processor -- start 2024-03-08T06:01:22.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.779+0000 7f9136dd0640 1 -- start start 2024-03-08T06:01:22.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.780+0000 7f9136dd0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f913006fbd0 0x7f91300cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.780+0000 7f9136dd0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9130071750 0x7f9130071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.780+0000 7f9136dd0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91300cad50 con 0x7f91300cdea0 2024-03-08T06:01:22.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.780+0000 7f9136dd0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91300c5da0 con 0x7f9130071750 2024-03-08T06:01:22.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.780+0000 7f9136dd0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91300c5b40 con 0x7f913006fbd0 2024-03-08T06:01:22.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.780+0000 7f912ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f91300cdea0 0x7f91300c5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39518/0 (socket says 172.21.15.76:39518) 2024-03-08T06:01:22.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.780+0000 7f912ffff640 1 -- 172.21.15.76:0/2840041248 learned_addr learned my addr 172.21.15.76:0/2840041248 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:22.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.780+0000 7f912f7fe640 1 -- 172.21.15.76:0/2840041248 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2950096001 0 0) 0x7f91300cad50 con 0x7f91300cdea0 2024-03-08T06:01:22.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f912f7fe640 1 -- 172.21.15.76:0/2840041248 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9118003520 con 0x7f91300cdea0 2024-03-08T06:01:22.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f913536d640 1 --2- 172.21.15.76:0/2840041248 >> v2:172.21.15.81:3300/0 conn(0x7f9130071750 0x7f9130071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f9134b6c640 1 --2- 172.21.15.76:0/2840041248 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f913006fbd0 0x7f91300cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f912f7fe640 1 -- 172.21.15.76:0/2840041248 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 963436673 0 0) 0x7f9118003520 con 0x7f91300cdea0 2024-03-08T06:01:22.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f912f7fe640 1 -- 172.21.15.76:0/2840041248 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f913006fbd0 msgr2=0x7f91300cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:22.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f912f7fe640 1 --2- 172.21.15.76:0/2840041248 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f913006fbd0 0x7f91300cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f912f7fe640 1 -- 172.21.15.76:0/2840041248 >> v2:172.21.15.81:3300/0 conn(0x7f9130071750 msgr2=0x7f9130071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:22.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f912f7fe640 1 --2- 172.21.15.76:0/2840041248 >> v2:172.21.15.81:3300/0 conn(0x7f9130071750 0x7f9130071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f912f7fe640 1 -- 172.21.15.76:0/2840041248 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91300caff0 con 0x7f91300cdea0 2024-03-08T06:01:22.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f913536d640 1 --2- 172.21.15.76:0/2840041248 >> v2:172.21.15.81:3300/0 conn(0x7f9130071750 0x7f9130071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:22.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f912f7fe640 1 -- 172.21.15.76:0/2840041248 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f9124001670 con 0x7f91300cdea0 2024-03-08T06:01:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f912f7fe640 1 -- 172.21.15.76:0/2840041248 >> v1:172.21.15.76:6789/0 conn(0x7f91300cdea0 legacy=0x7f91300c5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f912f7fe640 1 --2- 172.21.15.76:0/2840041248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9118003a20 0x7f9118003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f912f7fe640 1 --2- 172.21.15.76:0/2840041248 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f913006fbd0 0x7f9118004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f912f7fe640 1 --2- 172.21.15.76:0/2840041248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9118004d40 0x7f91180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f912f7fe640 1 -- 172.21.15.76:0/2840041248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91300c5da0 con 0x7f9118004d40 2024-03-08T06:01:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f912f7fe640 1 -- 172.21.15.76:0/2840041248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91300c5b40 con 0x7f9118003a20 2024-03-08T06:01:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f912f7fe640 1 -- 172.21.15.76:0/2840041248 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911800bb80 con 0x7f913006fbd0 2024-03-08T06:01:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f913536d640 1 --2- 172.21.15.76:0/2840041248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9118004d40 0x7f91180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f9134b6c640 1 --2- 172.21.15.76:0/2840041248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9118003a20 0x7f9118003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f912ffff640 1 --2- 172.21.15.76:0/2840041248 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f913006fbd0 0x7f9118004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f913536d640 1 -- 172.21.15.76:0/2840041248 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f913006fbd0 msgr2=0x7f9118004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f913536d640 1 --2- 172.21.15.76:0/2840041248 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f913006fbd0 0x7f9118004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f913536d640 1 -- 172.21.15.76:0/2840041248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9118003a20 msgr2=0x7f9118003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f913536d640 1 --2- 172.21.15.76:0/2840041248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9118003a20 0x7f9118003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.781+0000 7f913536d640 1 -- 172.21.15.76:0/2840041248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9118003520 con 0x7f9118004d40 2024-03-08T06:01:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.782+0000 7f913536d640 1 --2- 172.21.15.76:0/2840041248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9118004d40 0x7f91180051c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f91280073b0 tx=0x7f9118005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383ffe3101934a29 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.782+0000 7f912f7fe640 1 -- 172.21.15.76:0/2840041248 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128039070 con 0x7f9118004d40 2024-03-08T06:01:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.782+0000 7f912f7fe640 1 -- 172.21.15.76:0/2840041248 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912802cd90 con 0x7f9118004d40 2024-03-08T06:01:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.782+0000 7f912f7fe640 1 -- 172.21.15.76:0/2840041248 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128009c80 con 0x7f9118004d40 2024-03-08T06:01:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.783+0000 7f9136dd0640 1 -- 172.21.15.76:0/2840041248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9118004d40 msgr2=0x7f91180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.783+0000 7f9136dd0640 1 --2- 172.21.15.76:0/2840041248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9118004d40 0x7f91180051c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f91280073b0 tx=0x7f9118005830 comp rx=0 tx=0).stop 2024-03-08T06:01:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.783+0000 7f913536d640 1 -- 172.21.15.76:0/2840041248 reap_dead start 2024-03-08T06:01:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.783+0000 7f9136dd0640 1 -- 172.21.15.76:0/2840041248 shutdown_connections 2024-03-08T06:01:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.783+0000 7f9136dd0640 1 -- 172.21.15.76:0/2840041248 >> 172.21.15.76:0/2840041248 conn(0x7f91300cc600 msgr2=0x7f91300cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.783+0000 7f9136dd0640 1 -- 172.21.15.76:0/2840041248 shutdown_connections 2024-03-08T06:01:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.783+0000 7f9136dd0640 1 -- 172.21.15.76:0/2840041248 wait complete. 2024-03-08T06:01:22.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.784+0000 7f9136dd0640 1 Processor -- start 2024-03-08T06:01:22.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.784+0000 7f9136dd0640 1 -- start start 2024-03-08T06:01:22.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.785+0000 7f9136dd0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9118004d40 0x7f91301547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.785+0000 7f9136dd0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f913015ea30 0x7f91300b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.785+0000 7f9136dd0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91300b4790 0x7f913015cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.785+0000 7f9136dd0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91300cb890 con 0x7f913015ea30 2024-03-08T06:01:22.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.785+0000 7f9136dd0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91300cb5b0 con 0x7f91300b4790 2024-03-08T06:01:22.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.785+0000 7f9136dd0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913006ffb0 con 0x7f9118004d40 2024-03-08T06:01:22.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.785+0000 7f912ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f913015ea30 0x7f91300b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.785+0000 7f912ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f913015ea30 0x7f91300b4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41760/0 (socket says 172.21.15.76:41760) 2024-03-08T06:01:22.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.785+0000 7f912ffff640 1 -- 172.21.15.76:0/1569839808 learned_addr learned my addr 172.21.15.76:0/1569839808 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:22.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.785+0000 7f912ffff640 1 -- 172.21.15.76:0/1569839808 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9118004d40 msgr2=0x7f91301547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.785+0000 7f912ffff640 1 --2- 172.21.15.76:0/1569839808 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9118004d40 0x7f91301547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.785+0000 7f912ffff640 1 -- 172.21.15.76:0/1569839808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91300b4790 msgr2=0x7f913015cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.785+0000 7f912ffff640 1 --2- 172.21.15.76:0/1569839808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91300b4790 0x7f913015cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.785+0000 7f912ffff640 1 -- 172.21.15.76:0/1569839808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91301f7120 con 0x7f913015ea30 2024-03-08T06:01:22.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.785+0000 7f912ffff640 1 --2- 172.21.15.76:0/1569839808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f913015ea30 0x7f91300b4250 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f91240073a0 tx=0x7f9124007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:22.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.785+0000 7f912d7fa640 1 -- 172.21.15.76:0/1569839808 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912401a070 con 0x7f913015ea30 2024-03-08T06:01:22.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.786+0000 7f912d7fa640 1 -- 172.21.15.76:0/1569839808 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912400f8e0 con 0x7f913015ea30 2024-03-08T06:01:22.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.786+0000 7f912d7fa640 1 -- 172.21.15.76:0/1569839808 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912400fcb0 con 0x7f913015ea30 2024-03-08T06:01:22.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.786+0000 7f9136dd0640 1 -- 172.21.15.76:0/1569839808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91301f7400 con 0x7f913015ea30 2024-03-08T06:01:22.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.786+0000 7f9136dd0640 1 -- 172.21.15.76:0/1569839808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91301f79c0 con 0x7f913015ea30 2024-03-08T06:01:22.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.786+0000 7f9136dd0640 1 -- 172.21.15.76:0/1569839808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913004f030 con 0x7f913015ea30 2024-03-08T06:01:22.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.787+0000 7f912d7fa640 1 -- 172.21.15.76:0/1569839808 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f912401d070 con 0x7f913015ea30 2024-03-08T06:01:22.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.787+0000 7f912d7fa640 1 --2- 172.21.15.76:0/1569839808 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f911c06b500 0x7f911c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:22.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.787+0000 7f912d7fa640 1 -- 172.21.15.76:0/1569839808 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f9124089e80 con 0x7f913015ea30 2024-03-08T06:01:22.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.787+0000 7f9134b6c640 1 --2- 172.21.15.76:0/1569839808 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f911c06b500 0x7f911c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:22.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.788+0000 7f9134b6c640 1 --2- 172.21.15.76:0/1569839808 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f911c06b500 0x7f911c06d9c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9120006fd0 tx=0x7f91200040a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:22.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.792+0000 7f912d7fa640 1 -- 172.21.15.76:0/1569839808 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9124058900 con 0x7f913015ea30 2024-03-08T06:01:22.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.962+0000 7f9136dd0640 1 -- 172.21.15.76:0/1569839808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f91300cd2a0 con 0x7f913015ea30 2024-03-08T06:01:22.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.962+0000 7f912d7fa640 1 -- 172.21.15.76:0/1569839808 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9124058200 con 0x7f913015ea30 2024-03-08T06:01:22.963 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:01:22.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.964+0000 7f9136dd0640 1 -- 172.21.15.76:0/1569839808 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f911c06b500 msgr2=0x7f911c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.964+0000 7f9136dd0640 1 --2- 172.21.15.76:0/1569839808 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f911c06b500 0x7f911c06d9c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9120006fd0 tx=0x7f91200040a0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.964+0000 7f9136dd0640 1 -- 172.21.15.76:0/1569839808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f913015ea30 msgr2=0x7f91300b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:22.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.964+0000 7f9136dd0640 1 --2- 172.21.15.76:0/1569839808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f913015ea30 0x7f91300b4250 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f91240073a0 tx=0x7f9124007860 comp rx=0 tx=0).stop 2024-03-08T06:01:22.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.964+0000 7f9136dd0640 1 -- 172.21.15.76:0/1569839808 shutdown_connections 2024-03-08T06:01:22.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.964+0000 7f9136dd0640 1 --2- 172.21.15.76:0/1569839808 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f911c06b500 0x7f911c06d9c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.964+0000 7f9136dd0640 1 --2- 172.21.15.76:0/1569839808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91300b4790 0x7f913015cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.964+0000 7f9136dd0640 1 --2- 172.21.15.76:0/1569839808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f913015ea30 0x7f91300b4250 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.964+0000 7f9136dd0640 1 --2- 172.21.15.76:0/1569839808 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9118004d40 0x7f91301547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:22.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.964+0000 7f9136dd0640 1 -- 172.21.15.76:0/1569839808 >> 172.21.15.76:0/1569839808 conn(0x7f91300cc600 msgr2=0x7f91300aedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:22.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.964+0000 7f9136dd0640 1 -- 172.21.15.76:0/1569839808 shutdown_connections 2024-03-08T06:01:22.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:22.964+0000 7f9136dd0640 1 -- 172.21.15.76:0/1569839808 wait complete. 2024-03-08T06:01:22.975 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:01:23.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.078+0000 7f6497a40640 1 Processor -- start 2024-03-08T06:01:23.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.079+0000 7f6497a40640 1 -- start start 2024-03-08T06:01:23.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.079+0000 7f6497a40640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f64900d0800 0x7f64900b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.079+0000 7f6497a40640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64900cfa80 con 0x7f64900d2150 2024-03-08T06:01:23.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.079+0000 7f6497a40640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64900b1000 con 0x7f64900ce690 2024-03-08T06:01:23.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.079+0000 7f6497a40640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64900cfd20 con 0x7f64900d0800 2024-03-08T06:01:23.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.080+0000 7f6495fdd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f64900d0800 0x7f64900b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.080+0000 7f6494fdb640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f64900d2150 0x7f64900cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39530/0 (socket says 172.21.15.76:39530) 2024-03-08T06:01:23.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.080+0000 7f6494fdb640 1 -- 172.21.15.76:0/2019535964 learned_addr learned my addr 172.21.15.76:0/2019535964 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:23.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.080+0000 7f6495fdd640 1 -- 172.21.15.76:0/2019535964 >> v1:172.21.15.81:6789/0 conn(0x7f64900ce690 legacy=0x7f64900b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:23.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.080+0000 7f6495fdd640 1 -- 172.21.15.76:0/2019535964 >> v1:172.21.15.76:6789/0 conn(0x7f64900d2150 legacy=0x7f64900cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.080+0000 7f6487fff640 1 -- 172.21.15.76:0/2019535964 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522941065 0 0) 0x7f64900cfa80 con 0x7f64900d2150 2024-03-08T06:01:23.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.080+0000 7f6495fdd640 1 -- 172.21.15.76:0/2019535964 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64900d0000 con 0x7f64900d0800 2024-03-08T06:01:23.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.081+0000 7f6495fdd640 1 --2- 172.21.15.76:0/2019535964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64900d0800 0x7f64900b48a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6488009d90 tx=0x7f648802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea80f18ec4e5815e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:23.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.081+0000 7f6487fff640 1 -- 172.21.15.76:0/2019535964 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6488039070 con 0x7f64900d0800 2024-03-08T06:01:23.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.081+0000 7f6487fff640 1 -- 172.21.15.76:0/2019535964 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648802cc60 con 0x7f64900d0800 2024-03-08T06:01:23.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.081+0000 7f6487fff640 1 -- 172.21.15.76:0/2019535964 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6488034660 con 0x7f64900d0800 2024-03-08T06:01:23.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.081+0000 7f6497a40640 1 -- 172.21.15.76:0/2019535964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64900d0800 msgr2=0x7f64900b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.081+0000 7f6497a40640 1 --2- 172.21.15.76:0/2019535964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64900d0800 0x7f64900b48a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6488009d90 tx=0x7f648802beb0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.081+0000 7f6497a40640 1 -- 172.21.15.76:0/2019535964 shutdown_connections 2024-03-08T06:01:23.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.081+0000 7f6497a40640 1 --2- 172.21.15.76:0/2019535964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64900d0800 0x7f64900b48a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.081+0000 7f6497a40640 1 -- 172.21.15.76:0/2019535964 >> 172.21.15.76:0/2019535964 conn(0x7f649006ae60 msgr2=0x7f649006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:23.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.081+0000 7f6497a40640 1 -- 172.21.15.76:0/2019535964 shutdown_connections 2024-03-08T06:01:23.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.081+0000 7f6497a40640 1 -- 172.21.15.76:0/2019535964 wait complete. 2024-03-08T06:01:23.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.083+0000 7f6497a40640 1 Processor -- start 2024-03-08T06:01:23.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.083+0000 7f6497a40640 1 -- start start 2024-03-08T06:01:23.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.083+0000 7f6497a40640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64900ce690 0x7f64900c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.083+0000 7f6497a40640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64900d0800 0x7f64900c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.083+0000 7f6497a40640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f64900d2150 0x7f649015c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.083+0000 7f6497a40640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649015bfa0 con 0x7f64900d0800 2024-03-08T06:01:23.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.083+0000 7f6497a40640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649015be20 con 0x7f64900ce690 2024-03-08T06:01:23.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.083+0000 7f6497a40640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64900d0c50 con 0x7f64900d2150 2024-03-08T06:01:23.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.083+0000 7f6494fdb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64900d0800 0x7f64900c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.083+0000 7f6494fdb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64900d0800 0x7f64900c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41780/0 (socket says 172.21.15.76:41780) 2024-03-08T06:01:23.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.083+0000 7f6494fdb640 1 -- 172.21.15.76:0/968554818 learned_addr learned my addr 172.21.15.76:0/968554818 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:23.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.084+0000 7f6495fdd640 1 --2- 172.21.15.76:0/968554818 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f64900d2150 0x7f649015c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.084+0000 7f64957dc640 1 --2- 172.21.15.76:0/968554818 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64900ce690 0x7f64900c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.084+0000 7f6494fdb640 1 -- 172.21.15.76:0/968554818 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f64900d2150 msgr2=0x7f649015c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.084+0000 7f6494fdb640 1 --2- 172.21.15.76:0/968554818 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f64900d2150 0x7f649015c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.084+0000 7f6494fdb640 1 -- 172.21.15.76:0/968554818 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64900ce690 msgr2=0x7f64900c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.084+0000 7f6494fdb640 1 --2- 172.21.15.76:0/968554818 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64900ce690 0x7f64900c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.084+0000 7f6494fdb640 1 -- 172.21.15.76:0/968554818 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64900cbaa0 con 0x7f64900d0800 2024-03-08T06:01:23.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.084+0000 7f6495fdd640 1 --2- 172.21.15.76:0/968554818 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f64900d2150 0x7f649015c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:23.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.084+0000 7f64957dc640 1 --2- 172.21.15.76:0/968554818 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64900ce690 0x7f64900c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:23.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.084+0000 7f6494fdb640 1 --2- 172.21.15.76:0/968554818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64900d0800 0x7f64900c8a20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f648000c430 tx=0x7f648000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:23.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.084+0000 7f6485ffb640 1 -- 172.21.15.76:0/968554818 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64800074e0 con 0x7f64900d0800 2024-03-08T06:01:23.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.084+0000 7f6485ffb640 1 -- 172.21.15.76:0/968554818 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64800047e0 con 0x7f64900d0800 2024-03-08T06:01:23.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.084+0000 7f6485ffb640 1 -- 172.21.15.76:0/968554818 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480004bd0 con 0x7f64900d0800 2024-03-08T06:01:23.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.084+0000 7f6497a40640 1 -- 172.21.15.76:0/968554818 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64900c7df0 con 0x7f64900d0800 2024-03-08T06:01:23.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.084+0000 7f6497a40640 1 -- 172.21.15.76:0/968554818 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64900cb4e0 con 0x7f64900d0800 2024-03-08T06:01:23.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.085+0000 7f6497a40640 1 -- 172.21.15.76:0/968554818 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649004f030 con 0x7f64900d0800 2024-03-08T06:01:23.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.085+0000 7f6485ffb640 1 -- 172.21.15.76:0/968554818 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6480011070 con 0x7f64900d0800 2024-03-08T06:01:23.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.086+0000 7f6485ffb640 1 --2- 172.21.15.76:0/968554818 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f647406b4d0 0x7f647406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.086+0000 7f6485ffb640 1 -- 172.21.15.76:0/968554818 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f648008a280 con 0x7f64900d0800 2024-03-08T06:01:23.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.086+0000 7f64957dc640 1 --2- 172.21.15.76:0/968554818 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f647406b4d0 0x7f647406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.086+0000 7f64957dc640 1 --2- 172.21.15.76:0/968554818 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f647406b4d0 0x7f647406d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f649014a780 tx=0x7f648c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:23.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.090+0000 7f6485ffb640 1 -- 172.21.15.76:0/968554818 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6480058d00 con 0x7f64900d0800 2024-03-08T06:01:23.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.264+0000 7f6497a40640 1 -- 172.21.15.76:0/968554818 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f649006b450 con 0x7f64900d0800 2024-03-08T06:01:23.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.265+0000 7f6485ffb640 1 -- 172.21.15.76:0/968554818 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6480058600 con 0x7f64900d0800 2024-03-08T06:01:23.265 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:23.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.267+0000 7f6497a40640 1 -- 172.21.15.76:0/968554818 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f647406b4d0 msgr2=0x7f647406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.267+0000 7f6497a40640 1 --2- 172.21.15.76:0/968554818 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f647406b4d0 0x7f647406d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f649014a780 tx=0x7f648c009040 comp rx=0 tx=0).stop 2024-03-08T06:01:23.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.267+0000 7f6497a40640 1 -- 172.21.15.76:0/968554818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64900d0800 msgr2=0x7f64900c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.267+0000 7f6497a40640 1 --2- 172.21.15.76:0/968554818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64900d0800 0x7f64900c8a20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f648000c430 tx=0x7f648000c8f0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.267+0000 7f6497a40640 1 -- 172.21.15.76:0/968554818 shutdown_connections 2024-03-08T06:01:23.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.267+0000 7f6497a40640 1 --2- 172.21.15.76:0/968554818 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f647406b4d0 0x7f647406d990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.267+0000 7f6497a40640 1 --2- 172.21.15.76:0/968554818 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f64900d2150 0x7f649015c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.267+0000 7f6497a40640 1 --2- 172.21.15.76:0/968554818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64900d0800 0x7f64900c8a20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.267+0000 7f6497a40640 1 --2- 172.21.15.76:0/968554818 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64900ce690 0x7f64900c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.267+0000 7f6497a40640 1 -- 172.21.15.76:0/968554818 >> 172.21.15.76:0/968554818 conn(0x7f649006ae60 msgr2=0x7f64900d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:23.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.267+0000 7f6497a40640 1 -- 172.21.15.76:0/968554818 shutdown_connections 2024-03-08T06:01:23.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.267+0000 7f6497a40640 1 -- 172.21.15.76:0/968554818 wait complete. 2024-03-08T06:01:23.278 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:01:23.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.382+0000 7fde57c83640 1 Processor -- start 2024-03-08T06:01:23.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.382+0000 7fde57c83640 1 -- start start 2024-03-08T06:01:23.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.383+0000 7fde57c83640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fde500d2150 0x7fde500cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.383+0000 7fde57c83640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde500cfa80 con 0x7fde500ce690 2024-03-08T06:01:23.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.383+0000 7fde57c83640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde500cefb0 con 0x7fde500d0800 2024-03-08T06:01:23.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.383+0000 7fde57c83640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde500b1000 con 0x7fde500d2150 2024-03-08T06:01:23.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.383+0000 7fde55a1f640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fde500ce690 0x7fde500b0840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39542/0 (socket says 172.21.15.76:39542) 2024-03-08T06:01:23.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.383+0000 7fde55a1f640 1 -- 172.21.15.76:0/3903844275 learned_addr learned my addr 172.21.15.76:0/3903844275 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:23.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.383+0000 7fde54a1d640 1 -- 172.21.15.76:0/3903844275 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879867100 0 0) 0x7fde500cfa80 con 0x7fde500ce690 2024-03-08T06:01:23.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde54a1d640 1 -- 172.21.15.76:0/3903844275 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde3c003520 con 0x7fde500ce690 2024-03-08T06:01:23.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde5521e640 1 --2- 172.21.15.76:0/3903844275 >> v2:172.21.15.81:3300/0 conn(0x7fde500d2150 0x7fde500cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde54a1d640 1 -- 172.21.15.76:0/3903844275 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2455025779 0 0) 0x7fde3c003520 con 0x7fde500ce690 2024-03-08T06:01:23.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde54a1d640 1 -- 172.21.15.76:0/3903844275 >> v2:172.21.15.81:3300/0 conn(0x7fde500d2150 msgr2=0x7fde500cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:23.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde54a1d640 1 --2- 172.21.15.76:0/3903844275 >> v2:172.21.15.81:3300/0 conn(0x7fde500d2150 0x7fde500cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde54a1d640 1 -- 172.21.15.76:0/3903844275 >> v1:172.21.15.81:6789/0 conn(0x7fde500d0800 legacy=0x7fde500b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde54a1d640 1 -- 172.21.15.76:0/3903844275 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde500cfd20 con 0x7fde500ce690 2024-03-08T06:01:23.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde54a1d640 1 -- 172.21.15.76:0/3903844275 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fde4c0019f0 con 0x7fde500ce690 2024-03-08T06:01:23.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde54a1d640 1 -- 172.21.15.76:0/3903844275 >> v1:172.21.15.76:6789/0 conn(0x7fde500ce690 legacy=0x7fde500b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde54a1d640 1 --2- 172.21.15.76:0/3903844275 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde3c003a20 0x7fde3c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde54a1d640 1 --2- 172.21.15.76:0/3903844275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde3c004800 0x7fde3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde54a1d640 1 --2- 172.21.15.76:0/3903844275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde3c0051c0 0x7fde3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde54a1d640 1 -- 172.21.15.76:0/3903844275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500b1000 con 0x7fde3c004800 2024-03-08T06:01:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde54a1d640 1 -- 172.21.15.76:0/3903844275 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c00c0f0 con 0x7fde3c0051c0 2024-03-08T06:01:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde54a1d640 1 -- 172.21.15.76:0/3903844275 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c00c2f0 con 0x7fde3c003a20 2024-03-08T06:01:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde5521e640 1 --2- 172.21.15.76:0/3903844275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde3c004800 0x7fde3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde55a1f640 1 --2- 172.21.15.76:0/3903844275 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde3c003a20 0x7fde3c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde56220640 1 --2- 172.21.15.76:0/3903844275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde3c0051c0 0x7fde3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde5521e640 1 -- 172.21.15.76:0/3903844275 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde3c003a20 msgr2=0x7fde3c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde5521e640 1 --2- 172.21.15.76:0/3903844275 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde3c003a20 0x7fde3c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde5521e640 1 -- 172.21.15.76:0/3903844275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde3c0051c0 msgr2=0x7fde3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde5521e640 1 --2- 172.21.15.76:0/3903844275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde3c0051c0 0x7fde3c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde56220640 1 -- 172.21.15.76:0/3903844275 reap_dead start 2024-03-08T06:01:23.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde5521e640 1 -- 172.21.15.76:0/3903844275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c003520 con 0x7fde3c004800 2024-03-08T06:01:23.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.384+0000 7fde55a1f640 1 --2- 172.21.15.76:0/3903844275 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde3c003a20 0x7fde3c003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:23.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.385+0000 7fde5521e640 1 --2- 172.21.15.76:0/3903844275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde3c004800 0x7fde3c004c80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fde40009ac0 tx=0x7fde3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5497eade31523fa server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:23.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.385+0000 7fde54a1d640 1 -- 172.21.15.76:0/3903844275 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40039070 con 0x7fde3c004800 2024-03-08T06:01:23.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.385+0000 7fde54a1d640 1 -- 172.21.15.76:0/3903844275 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4002c6b0 con 0x7fde3c004800 2024-03-08T06:01:23.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.385+0000 7fde54a1d640 1 -- 172.21.15.76:0/3903844275 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40009690 con 0x7fde3c004800 2024-03-08T06:01:23.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.386+0000 7fde57c83640 1 -- 172.21.15.76:0/3903844275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde3c004800 msgr2=0x7fde3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.386+0000 7fde57c83640 1 --2- 172.21.15.76:0/3903844275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde3c004800 0x7fde3c004c80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fde40009ac0 tx=0x7fde3c005830 comp rx=0 tx=0).stop 2024-03-08T06:01:23.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.386+0000 7fde57c83640 1 -- 172.21.15.76:0/3903844275 shutdown_connections 2024-03-08T06:01:23.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.386+0000 7fde57c83640 1 --2- 172.21.15.76:0/3903844275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde3c004800 0x7fde3c004c80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.386+0000 7fde57c83640 1 -- 172.21.15.76:0/3903844275 >> 172.21.15.76:0/3903844275 conn(0x7fde5006ae60 msgr2=0x7fde5006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:23.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.386+0000 7fde57c83640 1 -- 172.21.15.76:0/3903844275 shutdown_connections 2024-03-08T06:01:23.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.386+0000 7fde57c83640 1 -- 172.21.15.76:0/3903844275 wait complete. 2024-03-08T06:01:23.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.387+0000 7fde57c83640 1 Processor -- start 2024-03-08T06:01:23.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.387+0000 7fde57c83640 1 -- start start 2024-03-08T06:01:23.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde57c83640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde3c004800 0x7fde5015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde57c83640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde501485e0 0x7fde50146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde57c83640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde5015d830 0x7fde5015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde57c83640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5015be20 con 0x7fde3c004800 2024-03-08T06:01:23.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde57c83640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500d02e0 con 0x7fde5015d830 2024-03-08T06:01:23.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde57c83640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5015bfa0 con 0x7fde501485e0 2024-03-08T06:01:23.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde55a1f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde3c004800 0x7fde5015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde55a1f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde3c004800 0x7fde5015ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41792/0 (socket says 172.21.15.76:41792) 2024-03-08T06:01:23.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde55a1f640 1 -- 172.21.15.76:0/2026784194 learned_addr learned my addr 172.21.15.76:0/2026784194 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:23.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde5521e640 1 --2- 172.21.15.76:0/2026784194 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde501485e0 0x7fde50146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde56220640 1 --2- 172.21.15.76:0/2026784194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde5015d830 0x7fde5015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde55a1f640 1 -- 172.21.15.76:0/2026784194 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde501485e0 msgr2=0x7fde50146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde55a1f640 1 --2- 172.21.15.76:0/2026784194 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde501485e0 0x7fde50146550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde55a1f640 1 -- 172.21.15.76:0/2026784194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde5015d830 msgr2=0x7fde5015dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde55a1f640 1 --2- 172.21.15.76:0/2026784194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde5015d830 0x7fde5015dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde55a1f640 1 -- 172.21.15.76:0/2026784194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde50146dc0 con 0x7fde3c004800 2024-03-08T06:01:23.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde55a1f640 1 --2- 172.21.15.76:0/2026784194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde3c004800 0x7fde5015ebf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0079e0 tx=0x7fde4c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:23.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde467fc640 1 -- 172.21.15.76:0/2026784194 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c00ee50 con 0x7fde3c004800 2024-03-08T06:01:23.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde467fc640 1 -- 172.21.15.76:0/2026784194 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4c018070 con 0x7fde3c004800 2024-03-08T06:01:23.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde57c83640 1 -- 172.21.15.76:0/2026784194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde50148d30 con 0x7fde3c004800 2024-03-08T06:01:23.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde467fc640 1 -- 172.21.15.76:0/2026784194 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c013620 con 0x7fde3c004800 2024-03-08T06:01:23.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.388+0000 7fde57c83640 1 -- 172.21.15.76:0/2026784194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde501492f0 con 0x7fde3c004800 2024-03-08T06:01:23.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.389+0000 7fde57c83640 1 -- 172.21.15.76:0/2026784194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde50149530 con 0x7fde3c004800 2024-03-08T06:01:23.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.390+0000 7fde467fc640 1 -- 172.21.15.76:0/2026784194 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde4c0137c0 con 0x7fde3c004800 2024-03-08T06:01:23.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.390+0000 7fde467fc640 1 --2- 172.21.15.76:0/2026784194 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fde3406b670 0x7fde3406db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.390+0000 7fde467fc640 1 -- 172.21.15.76:0/2026784194 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fde4c08abd0 con 0x7fde3c004800 2024-03-08T06:01:23.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.390+0000 7fde5521e640 1 --2- 172.21.15.76:0/2026784194 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fde3406b670 0x7fde3406db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.391+0000 7fde5521e640 1 --2- 172.21.15.76:0/2026784194 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fde3406b670 0x7fde3406db30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fde40036000 tx=0x7fde40030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:23.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.395+0000 7fde467fc640 1 -- 172.21.15.76:0/2026784194 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde50149530 con 0x7fde3c004800 2024-03-08T06:01:23.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.563+0000 7fde57c83640 1 -- 172.21.15.76:0/2026784194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fde5004f030 con 0x7fde3c004800 2024-03-08T06:01:23.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.564+0000 7fde467fc640 1 -- 172.21.15.76:0/2026784194 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fde4c059650 con 0x7fde3c004800 2024-03-08T06:01:23.564 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:01:23.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.565+0000 7fde57c83640 1 -- 172.21.15.76:0/2026784194 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fde3406b670 msgr2=0x7fde3406db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.565+0000 7fde57c83640 1 --2- 172.21.15.76:0/2026784194 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fde3406b670 0x7fde3406db30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fde40036000 tx=0x7fde40030000 comp rx=0 tx=0).stop 2024-03-08T06:01:23.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.565+0000 7fde57c83640 1 -- 172.21.15.76:0/2026784194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde3c004800 msgr2=0x7fde5015ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.566+0000 7fde57c83640 1 --2- 172.21.15.76:0/2026784194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde3c004800 0x7fde5015ebf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0079e0 tx=0x7fde4c00e530 comp rx=0 tx=0).stop 2024-03-08T06:01:23.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.566+0000 7fde57c83640 1 -- 172.21.15.76:0/2026784194 shutdown_connections 2024-03-08T06:01:23.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.566+0000 7fde57c83640 1 --2- 172.21.15.76:0/2026784194 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fde3406b670 0x7fde3406db30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.566+0000 7fde57c83640 1 --2- 172.21.15.76:0/2026784194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde5015d830 0x7fde5015dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.566+0000 7fde57c83640 1 --2- 172.21.15.76:0/2026784194 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde501485e0 0x7fde50146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.566+0000 7fde57c83640 1 --2- 172.21.15.76:0/2026784194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde3c004800 0x7fde5015ebf0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.566+0000 7fde57c83640 1 -- 172.21.15.76:0/2026784194 >> 172.21.15.76:0/2026784194 conn(0x7fde5006ae60 msgr2=0x7fde500b67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:23.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.566+0000 7fde57c83640 1 -- 172.21.15.76:0/2026784194 shutdown_connections 2024-03-08T06:01:23.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.566+0000 7fde57c83640 1 -- 172.21.15.76:0/2026784194 wait complete. 2024-03-08T06:01:23.577 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T06:01:23.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.679+0000 7fb956eda640 1 Processor -- start 2024-03-08T06:01:23.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.679+0000 7fb956eda640 1 -- start start 2024-03-08T06:01:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.680+0000 7fb956eda640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb9500cf010 0x7fb9500cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.680+0000 7fb956eda640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9500b1550 con 0x7fb95015e120 2024-03-08T06:01:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.680+0000 7fb956eda640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9500ae1d0 con 0x7fb95015e500 2024-03-08T06:01:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.680+0000 7fb956eda640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9500ae4b0 con 0x7fb9500cf010 2024-03-08T06:01:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.680+0000 7fb954c76640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb95015e120 0x7fb9500ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39556/0 (socket says 172.21.15.76:39556) 2024-03-08T06:01:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.680+0000 7fb954c76640 1 -- 172.21.15.76:0/2166114087 learned_addr learned my addr 172.21.15.76:0/2166114087 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb94f7fe640 1 -- 172.21.15.76:0/2166114087 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3672904887 0 0) 0x7fb9500b1550 con 0x7fb95015e120 2024-03-08T06:01:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb94f7fe640 1 -- 172.21.15.76:0/2166114087 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb938003520 con 0x7fb95015e120 2024-03-08T06:01:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb955477640 1 --2- 172.21.15.76:0/2166114087 >> v2:172.21.15.81:3300/0 conn(0x7fb9500cf010 0x7fb9500cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb94f7fe640 1 -- 172.21.15.76:0/2166114087 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2947168845 0 0) 0x7fb938003520 con 0x7fb95015e120 2024-03-08T06:01:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb94f7fe640 1 -- 172.21.15.76:0/2166114087 >> v2:172.21.15.81:3300/0 conn(0x7fb9500cf010 msgr2=0x7fb9500cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb94f7fe640 1 --2- 172.21.15.76:0/2166114087 >> v2:172.21.15.81:3300/0 conn(0x7fb9500cf010 0x7fb9500cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb94f7fe640 1 -- 172.21.15.76:0/2166114087 >> v1:172.21.15.81:6789/0 conn(0x7fb95015e500 legacy=0x7fb9500aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb94f7fe640 1 -- 172.21.15.76:0/2166114087 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9500ae790 con 0x7fb95015e120 2024-03-08T06:01:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb94f7fe640 1 -- 172.21.15.76:0/2166114087 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fb940001670 con 0x7fb95015e120 2024-03-08T06:01:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb94f7fe640 1 -- 172.21.15.76:0/2166114087 >> v1:172.21.15.76:6789/0 conn(0x7fb95015e120 legacy=0x7fb9500ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb94f7fe640 1 --2- 172.21.15.76:0/2166114087 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb938003a70 0x7fb938003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb94f7fe640 1 --2- 172.21.15.76:0/2166114087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb938004800 0x7fb938004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb94f7fe640 1 --2- 172.21.15.76:0/2166114087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb938005210 0x7fb93800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb94f7fe640 1 -- 172.21.15.76:0/2166114087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9500ae4b0 con 0x7fb938005210 2024-03-08T06:01:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb94f7fe640 1 -- 172.21.15.76:0/2166114087 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93800c140 con 0x7fb938004800 2024-03-08T06:01:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb94f7fe640 1 -- 172.21.15.76:0/2166114087 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93800c340 con 0x7fb938003a70 2024-03-08T06:01:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb955477640 1 --2- 172.21.15.76:0/2166114087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb938005210 0x7fb93800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb94ffff640 1 --2- 172.21.15.76:0/2166114087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb938004800 0x7fb938004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb954c76640 1 --2- 172.21.15.76:0/2166114087 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb938003a70 0x7fb938003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb955477640 1 -- 172.21.15.76:0/2166114087 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb938003a70 msgr2=0x7fb938003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb955477640 1 --2- 172.21.15.76:0/2166114087 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb938003a70 0x7fb938003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb955477640 1 -- 172.21.15.76:0/2166114087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb938004800 msgr2=0x7fb938004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb955477640 1 --2- 172.21.15.76:0/2166114087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb938004800 0x7fb938004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.681+0000 7fb955477640 1 -- 172.21.15.76:0/2166114087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb938003520 con 0x7fb938005210 2024-03-08T06:01:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.682+0000 7fb955477640 1 -- 172.21.15.76:0/2166114087 reap_dead start 2024-03-08T06:01:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.682+0000 7fb955477640 1 --2- 172.21.15.76:0/2166114087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb938005210 0x7fb93800b9c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb9500723c0 tx=0x7fb938005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d02268b1c59fd3d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.682+0000 7fb94f7fe640 1 -- 172.21.15.76:0/2166114087 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948009bf0 con 0x7fb938005210 2024-03-08T06:01:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.682+0000 7fb94f7fe640 1 -- 172.21.15.76:0/2166114087 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94802cc50 con 0x7fb938005210 2024-03-08T06:01:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.682+0000 7fb94f7fe640 1 -- 172.21.15.76:0/2166114087 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948003990 con 0x7fb938005210 2024-03-08T06:01:23.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.683+0000 7fb956eda640 1 -- 172.21.15.76:0/2166114087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb938005210 msgr2=0x7fb93800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.683+0000 7fb956eda640 1 --2- 172.21.15.76:0/2166114087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb938005210 0x7fb93800b9c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb9500723c0 tx=0x7fb938005830 comp rx=0 tx=0).stop 2024-03-08T06:01:23.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.683+0000 7fb956eda640 1 -- 172.21.15.76:0/2166114087 shutdown_connections 2024-03-08T06:01:23.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.683+0000 7fb956eda640 1 --2- 172.21.15.76:0/2166114087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb938005210 0x7fb93800b9c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.683+0000 7fb956eda640 1 -- 172.21.15.76:0/2166114087 >> 172.21.15.76:0/2166114087 conn(0x7fb9500d0890 msgr2=0x7fb9500d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:23.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.683+0000 7fb956eda640 1 -- 172.21.15.76:0/2166114087 shutdown_connections 2024-03-08T06:01:23.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.683+0000 7fb956eda640 1 -- 172.21.15.76:0/2166114087 wait complete. 2024-03-08T06:01:23.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.684+0000 7fb956eda640 1 Processor -- start 2024-03-08T06:01:23.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.684+0000 7fb956eda640 1 -- start start 2024-03-08T06:01:23.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb956eda640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb938005210 0x7fb950150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb956eda640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb9500cc470 0x7fb9500c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb956eda640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb95015b9e0 0x7fb95015be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb956eda640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9500d1380 con 0x7fb95015b9e0 2024-03-08T06:01:23.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb956eda640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9500cf6b0 con 0x7fb938005210 2024-03-08T06:01:23.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb956eda640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9500d1500 con 0x7fb9500cc470 2024-03-08T06:01:23.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb955477640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb95015b9e0 0x7fb95015be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb955477640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb95015b9e0 0x7fb95015be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41810/0 (socket says 172.21.15.76:41810) 2024-03-08T06:01:23.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb955477640 1 -- 172.21.15.76:0/3418432883 learned_addr learned my addr 172.21.15.76:0/3418432883 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:23.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb94ffff640 1 --2- 172.21.15.76:0/3418432883 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb9500cc470 0x7fb9500c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb954c76640 1 --2- 172.21.15.76:0/3418432883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb938005210 0x7fb950150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb955477640 1 -- 172.21.15.76:0/3418432883 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb9500cc470 msgr2=0x7fb9500c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb955477640 1 --2- 172.21.15.76:0/3418432883 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb9500cc470 0x7fb9500c4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb955477640 1 -- 172.21.15.76:0/3418432883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb938005210 msgr2=0x7fb950150e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb955477640 1 --2- 172.21.15.76:0/3418432883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb938005210 0x7fb950150e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb955477640 1 -- 172.21.15.76:0/3418432883 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9500c48b0 con 0x7fb95015b9e0 2024-03-08T06:01:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb94ffff640 1 --2- 172.21.15.76:0/3418432883 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb9500cc470 0x7fb9500c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb954c76640 1 --2- 172.21.15.76:0/3418432883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb938005210 0x7fb950150e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.685+0000 7fb955477640 1 --2- 172.21.15.76:0/3418432883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb95015b9e0 0x7fb95015be60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb94802c600 tx=0x7fb948009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.686+0000 7fb94d7fa640 1 -- 172.21.15.76:0/3418432883 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948009bf0 con 0x7fb95015b9e0 2024-03-08T06:01:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.686+0000 7fb94d7fa640 1 -- 172.21.15.76:0/3418432883 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb948044070 con 0x7fb95015b9e0 2024-03-08T06:01:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.686+0000 7fb956eda640 1 -- 172.21.15.76:0/3418432883 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9500c4b90 con 0x7fb95015b9e0 2024-03-08T06:01:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.686+0000 7fb94d7fa640 1 -- 172.21.15.76:0/3418432883 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948004830 con 0x7fb95015b9e0 2024-03-08T06:01:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.686+0000 7fb956eda640 1 -- 172.21.15.76:0/3418432883 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9500c5150 con 0x7fb95015b9e0 2024-03-08T06:01:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.687+0000 7fb956eda640 1 -- 172.21.15.76:0/3418432883 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9500c5480 con 0x7fb95015b9e0 2024-03-08T06:01:23.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.687+0000 7fb94d7fa640 1 -- 172.21.15.76:0/3418432883 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb948005d10 con 0x7fb95015b9e0 2024-03-08T06:01:23.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.687+0000 7fb94d7fa640 1 --2- 172.21.15.76:0/3418432883 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb93c06b6a0 0x7fb93c06db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.688+0000 7fb94d7fa640 1 -- 172.21.15.76:0/3418432883 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb9480acad0 con 0x7fb95015b9e0 2024-03-08T06:01:23.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.688+0000 7fb954c76640 1 --2- 172.21.15.76:0/3418432883 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb93c06b6a0 0x7fb93c06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.688+0000 7fb954c76640 1 --2- 172.21.15.76:0/3418432883 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb93c06b6a0 0x7fb93c06db60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb940000fb0 tx=0x7fb940009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:23.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.692+0000 7fb94d7fa640 1 -- 172.21.15.76:0/3418432883 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb94803d330 con 0x7fb95015b9e0 2024-03-08T06:01:23.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.860+0000 7fb956eda640 1 -- 172.21.15.76:0/3418432883 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb95004f030 con 0x7fb95015b9e0 2024-03-08T06:01:23.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.861+0000 7fb94d7fa640 1 -- 172.21.15.76:0/3418432883 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb94807b550 con 0x7fb95015b9e0 2024-03-08T06:01:23.861 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:23.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.862+0000 7fb956eda640 1 -- 172.21.15.76:0/3418432883 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb93c06b6a0 msgr2=0x7fb93c06db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.862+0000 7fb956eda640 1 --2- 172.21.15.76:0/3418432883 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb93c06b6a0 0x7fb93c06db60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb940000fb0 tx=0x7fb940009040 comp rx=0 tx=0).stop 2024-03-08T06:01:23.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.862+0000 7fb956eda640 1 -- 172.21.15.76:0/3418432883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb95015b9e0 msgr2=0x7fb95015be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.862+0000 7fb956eda640 1 --2- 172.21.15.76:0/3418432883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb95015b9e0 0x7fb95015be60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb94802c600 tx=0x7fb948009750 comp rx=0 tx=0).stop 2024-03-08T06:01:23.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.863+0000 7fb956eda640 1 -- 172.21.15.76:0/3418432883 shutdown_connections 2024-03-08T06:01:23.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.863+0000 7fb956eda640 1 --2- 172.21.15.76:0/3418432883 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb93c06b6a0 0x7fb93c06db60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.863+0000 7fb956eda640 1 --2- 172.21.15.76:0/3418432883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb95015b9e0 0x7fb95015be60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.863+0000 7fb956eda640 1 --2- 172.21.15.76:0/3418432883 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb9500cc470 0x7fb9500c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.863+0000 7fb956eda640 1 --2- 172.21.15.76:0/3418432883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb938005210 0x7fb950150e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.863+0000 7fb956eda640 1 -- 172.21.15.76:0/3418432883 >> 172.21.15.76:0/3418432883 conn(0x7fb9500d0890 msgr2=0x7fb95015d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:23.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.863+0000 7fb956eda640 1 -- 172.21.15.76:0/3418432883 shutdown_connections 2024-03-08T06:01:23.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.863+0000 7fb956eda640 1 -- 172.21.15.76:0/3418432883 wait complete. 2024-03-08T06:01:23.874 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters' 2024-03-08T06:01:23.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.974+0000 7fe51bad4640 1 Processor -- start 2024-03-08T06:01:23.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.975+0000 7fe51bad4640 1 -- start start 2024-03-08T06:01:23.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.975+0000 7fe51bad4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe5140b6d00 0x7fe5140d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.975+0000 7fe51bad4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe5140cec10 0x7fe5140d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.975+0000 7fe51bad4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5140cf180 con 0x7fe51415dde0 2024-03-08T06:01:23.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.975+0000 7fe51bad4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5140d1ed0 con 0x7fe5140cec10 2024-03-08T06:01:23.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.975+0000 7fe51bad4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5140d21b0 con 0x7fe5140b6d00 2024-03-08T06:01:23.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.975+0000 7fe51a071640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe5140cec10 0x7fe5140d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.975+0000 7fe519870640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe51415dde0 0x7fe514162250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39568/0 (socket says 172.21.15.76:39568) 2024-03-08T06:01:23.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.976+0000 7fe519870640 1 -- 172.21.15.76:0/2671445662 learned_addr learned my addr 172.21.15.76:0/2671445662 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:23.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.976+0000 7fe51a071640 1 -- 172.21.15.76:0/2671445662 >> v2:172.21.15.81:3300/0 conn(0x7fe5140b6d00 msgr2=0x7fe5140d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:23.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.976+0000 7fe51a071640 1 --2- 172.21.15.76:0/2671445662 >> v2:172.21.15.81:3300/0 conn(0x7fe5140b6d00 0x7fe5140d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.976+0000 7fe51a071640 1 -- 172.21.15.76:0/2671445662 >> v1:172.21.15.76:6789/0 conn(0x7fe51415dde0 legacy=0x7fe514162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.976+0000 7fe51a071640 1 -- 172.21.15.76:0/2671445662 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe514168bd0 con 0x7fe5140cec10 2024-03-08T06:01:23.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.976+0000 7fe51886e640 1 -- 172.21.15.76:0/2671445662 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706466923 0 0) 0x7fe5140cf180 con 0x7fe51415dde0 2024-03-08T06:01:23.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.976+0000 7fe51a071640 1 --2- 172.21.15.76:0/2671445662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5140cec10 0x7fe5140d09c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe51408ce40 tx=0x7fe50c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6d4cbff9e2469d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:23.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.977+0000 7fe51886e640 1 -- 172.21.15.76:0/2671445662 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c0315a0 con 0x7fe5140cec10 2024-03-08T06:01:23.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.977+0000 7fe51886e640 1 -- 172.21.15.76:0/2671445662 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c037070 con 0x7fe5140cec10 2024-03-08T06:01:23.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.977+0000 7fe51886e640 1 -- 172.21.15.76:0/2671445662 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c031960 con 0x7fe5140cec10 2024-03-08T06:01:23.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.977+0000 7fe51bad4640 1 -- 172.21.15.76:0/2671445662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5140cec10 msgr2=0x7fe5140d09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.977+0000 7fe51bad4640 1 --2- 172.21.15.76:0/2671445662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5140cec10 0x7fe5140d09c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe51408ce40 tx=0x7fe50c02b840 comp rx=0 tx=0).stop 2024-03-08T06:01:23.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.977+0000 7fe51bad4640 1 -- 172.21.15.76:0/2671445662 shutdown_connections 2024-03-08T06:01:23.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.977+0000 7fe51bad4640 1 --2- 172.21.15.76:0/2671445662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5140cec10 0x7fe5140d09c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.977+0000 7fe51bad4640 1 --2- 172.21.15.76:0/2671445662 >> v2:172.21.15.81:3300/0 conn(0x7fe5140b6d00 0x7fe5140d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.977+0000 7fe51bad4640 1 -- 172.21.15.76:0/2671445662 >> 172.21.15.76:0/2671445662 conn(0x7fe5140b56e0 msgr2=0x7fe5140b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:23.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.977+0000 7fe51bad4640 1 -- 172.21.15.76:0/2671445662 shutdown_connections 2024-03-08T06:01:23.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.977+0000 7fe51bad4640 1 -- 172.21.15.76:0/2671445662 wait complete. 2024-03-08T06:01:23.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.978+0000 7fe51bad4640 1 Processor -- start 2024-03-08T06:01:23.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.978+0000 7fe51bad4640 1 -- start start 2024-03-08T06:01:23.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.979+0000 7fe51bad4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5140b6d00 0x7fe514154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.979+0000 7fe51bad4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5140cec10 0x7fe5140d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.979+0000 7fe51bad4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe51415dde0 0x7fe5140d40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.979+0000 7fe51bad4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5140d2d40 con 0x7fe51415dde0 2024-03-08T06:01:23.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.979+0000 7fe51bad4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5140d2bc0 con 0x7fe5140cec10 2024-03-08T06:01:23.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.979+0000 7fe51bad4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5140d2ec0 con 0x7fe5140b6d00 2024-03-08T06:01:23.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.979+0000 7fe51a071640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe51415dde0 0x7fe5140d40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.979+0000 7fe51a071640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe51415dde0 0x7fe5140d40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41818/0 (socket says 172.21.15.76:41818) 2024-03-08T06:01:23.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.979+0000 7fe51a071640 1 -- 172.21.15.76:0/234923173 learned_addr learned my addr 172.21.15.76:0/234923173 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:23.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.979+0000 7fe519870640 1 --2- 172.21.15.76:0/234923173 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5140b6d00 0x7fe514154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.979+0000 7fe51906f640 1 --2- 172.21.15.76:0/234923173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5140cec10 0x7fe5140d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.980+0000 7fe51a071640 1 -- 172.21.15.76:0/234923173 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5140b6d00 msgr2=0x7fe514154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.980+0000 7fe51a071640 1 --2- 172.21.15.76:0/234923173 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5140b6d00 0x7fe514154aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.980+0000 7fe51a071640 1 -- 172.21.15.76:0/234923173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5140cec10 msgr2=0x7fe5140d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:23.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.980+0000 7fe51a071640 1 --2- 172.21.15.76:0/234923173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5140cec10 0x7fe5140d3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:23.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.980+0000 7fe51a071640 1 -- 172.21.15.76:0/234923173 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5140cae40 con 0x7fe51415dde0 2024-03-08T06:01:23.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.980+0000 7fe51a071640 1 --2- 172.21.15.76:0/234923173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe51415dde0 0x7fe5140d40f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe50c02be50 tx=0x7fe50c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:23.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.980+0000 7fe50a7fc640 1 -- 172.21.15.76:0/234923173 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c032840 con 0x7fe51415dde0 2024-03-08T06:01:23.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.980+0000 7fe50a7fc640 1 -- 172.21.15.76:0/234923173 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c032a00 con 0x7fe51415dde0 2024-03-08T06:01:23.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.980+0000 7fe51bad4640 1 -- 172.21.15.76:0/234923173 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5140cb120 con 0x7fe51415dde0 2024-03-08T06:01:23.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.980+0000 7fe50a7fc640 1 -- 172.21.15.76:0/234923173 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c032df0 con 0x7fe51415dde0 2024-03-08T06:01:23.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.980+0000 7fe51bad4640 1 -- 172.21.15.76:0/234923173 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5140cb6e0 con 0x7fe51415dde0 2024-03-08T06:01:23.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.981+0000 7fe51bad4640 1 -- 172.21.15.76:0/234923173 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51404f030 con 0x7fe51415dde0 2024-03-08T06:01:23.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.981+0000 7fe50a7fc640 1 -- 172.21.15.76:0/234923173 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe50c04c020 con 0x7fe51415dde0 2024-03-08T06:01:23.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.982+0000 7fe50a7fc640 1 --2- 172.21.15.76:0/234923173 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe4f806b500 0x7fe4f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:23.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.982+0000 7fe519870640 1 --2- 172.21.15.76:0/234923173 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe4f806b500 0x7fe4f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:23.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.982+0000 7fe50a7fc640 1 -- 172.21.15.76:0/234923173 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe50c02f020 con 0x7fe51415dde0 2024-03-08T06:01:23.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.982+0000 7fe519870640 1 --2- 172.21.15.76:0/234923173 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe4f806b500 0x7fe4f806d9c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe5100012e0 tx=0x7fe510001630 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:23.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:23.986+0000 7fe50a7fc640 1 -- 172.21.15.76:0/234923173 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe50c040070 con 0x7fe51415dde0 2024-03-08T06:01:24.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.224+0000 7fe51bad4640 1 -- 172.21.15.76:0/234923173 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters"]} v 0) v1 -- 0x7fe5140cb8b0 con 0x7fe51415dde0 2024-03-08T06:01:24.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.353+0000 7fe50a7fc640 1 -- 172.21.15.76:0/234923173 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters"]}]=0 v68) v1 ==== 167+0+0 (secure 0 0 0) 0x7fe50c07ab90 con 0x7fe51415dde0 2024-03-08T06:01:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.355+0000 7fe51bad4640 1 -- 172.21.15.76:0/234923173 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe4f806b500 msgr2=0x7fe4f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.355+0000 7fe51bad4640 1 --2- 172.21.15.76:0/234923173 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe4f806b500 0x7fe4f806d9c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe5100012e0 tx=0x7fe510001630 comp rx=0 tx=0).stop 2024-03-08T06:01:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.355+0000 7fe51bad4640 1 -- 172.21.15.76:0/234923173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe51415dde0 msgr2=0x7fe5140d40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.355+0000 7fe51bad4640 1 --2- 172.21.15.76:0/234923173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe51415dde0 0x7fe5140d40f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe50c02be50 tx=0x7fe50c0046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:24.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.355+0000 7fe51bad4640 1 -- 172.21.15.76:0/234923173 shutdown_connections 2024-03-08T06:01:24.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.355+0000 7fe51bad4640 1 --2- 172.21.15.76:0/234923173 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe4f806b500 0x7fe4f806d9c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:24.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.355+0000 7fe51bad4640 1 --2- 172.21.15.76:0/234923173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe51415dde0 0x7fe5140d40f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:24.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.355+0000 7fe51bad4640 1 --2- 172.21.15.76:0/234923173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5140cec10 0x7fe5140d3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:24.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.355+0000 7fe51bad4640 1 --2- 172.21.15.76:0/234923173 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5140b6d00 0x7fe514154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:24.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.355+0000 7fe51bad4640 1 -- 172.21.15.76:0/234923173 >> 172.21.15.76:0/234923173 conn(0x7fe5140b56e0 msgr2=0x7fe5140d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:24.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.356+0000 7fe51bad4640 1 -- 172.21.15.76:0/234923173 shutdown_connections 2024-03-08T06:01:24.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.356+0000 7fe51bad4640 1 -- 172.21.15.76:0/234923173 wait complete. 2024-03-08T06:01:24.367 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-08T06:01:24.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.469+0000 7f0d24e55640 1 Processor -- start 2024-03-08T06:01:24.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.469+0000 7f0d24e55640 1 -- start start 2024-03-08T06:01:24.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.470+0000 7f0d24e55640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0d200b9840 0x7f0d200b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:24.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.470+0000 7f0d24e55640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d200d27d0 con 0x7f0d2015dbc0 2024-03-08T06:01:24.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.470+0000 7f0d24e55640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d200d2ab0 con 0x7f0d200b5b20 2024-03-08T06:01:24.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.470+0000 7f0d24e55640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200b5f90 con 0x7f0d200b9840 2024-03-08T06:01:24.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.470+0000 7f0d1e59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0d2015dbc0 0x7f0d200d1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39574/0 (socket says 172.21.15.76:39574) 2024-03-08T06:01:24.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.470+0000 7f0d1e59c640 1 -- 172.21.15.76:0/3544418917 learned_addr learned my addr 172.21.15.76:0/3544418917 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.470+0000 7f0d1d59a640 1 -- 172.21.15.76:0/3544418917 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491275331 0 0) 0x7f0d200d27d0 con 0x7f0d2015dbc0 2024-03-08T06:01:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1d59a640 1 -- 172.21.15.76:0/3544418917 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d08003520 con 0x7f0d2015dbc0 2024-03-08T06:01:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1dd9b640 1 --2- 172.21.15.76:0/3544418917 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0d200b9840 0x7f0d200b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1d59a640 1 -- 172.21.15.76:0/3544418917 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1170036659 0 0) 0x7f0d08003520 con 0x7f0d2015dbc0 2024-03-08T06:01:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1d59a640 1 -- 172.21.15.76:0/3544418917 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0d200b9840 msgr2=0x7f0d200b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1d59a640 1 --2- 172.21.15.76:0/3544418917 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0d200b9840 0x7f0d200b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1d59a640 1 -- 172.21.15.76:0/3544418917 >> v1:172.21.15.81:6789/0 conn(0x7f0d200b5b20 legacy=0x7f0d200b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1d59a640 1 -- 172.21.15.76:0/3544418917 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d200d2d90 con 0x7f0d2015dbc0 2024-03-08T06:01:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1d59a640 1 -- 172.21.15.76:0/3544418917 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f0d100019f0 con 0x7f0d2015dbc0 2024-03-08T06:01:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1d59a640 1 -- 172.21.15.76:0/3544418917 >> v1:172.21.15.76:6789/0 conn(0x7f0d2015dbc0 legacy=0x7f0d200d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1d59a640 1 --2- 172.21.15.76:0/3544418917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d08003a50 0x7f0d08003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1d59a640 1 --2- 172.21.15.76:0/3544418917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d08004800 0x7f0d08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1d59a640 1 --2- 172.21.15.76:0/3544418917 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0d200b9840 0x7f0d08005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1d59a640 1 -- 172.21.15.76:0/3544418917 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200b5f90 con 0x7f0d08004800 2024-03-08T06:01:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1d59a640 1 -- 172.21.15.76:0/3544418917 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0800bb80 con 0x7f0d08003a50 2024-03-08T06:01:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1d59a640 1 -- 172.21.15.76:0/3544418917 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0800bd80 con 0x7f0d200b9840 2024-03-08T06:01:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1dd9b640 1 --2- 172.21.15.76:0/3544418917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d08004800 0x7f0d08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1e59c640 1 --2- 172.21.15.76:0/3544418917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d08003a50 0x7f0d08003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1ed9d640 1 --2- 172.21.15.76:0/3544418917 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0d200b9840 0x7f0d08005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1dd9b640 1 -- 172.21.15.76:0/3544418917 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0d200b9840 msgr2=0x7f0d08005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1dd9b640 1 --2- 172.21.15.76:0/3544418917 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0d200b9840 0x7f0d08005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1dd9b640 1 -- 172.21.15.76:0/3544418917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d08003a50 msgr2=0x7f0d08003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1dd9b640 1 --2- 172.21.15.76:0/3544418917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d08003a50 0x7f0d08003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1dd9b640 1 -- 172.21.15.76:0/3544418917 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d08003520 con 0x7f0d08004800 2024-03-08T06:01:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.471+0000 7f0d1ed9d640 1 --2- 172.21.15.76:0/3544418917 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0d200b9840 0x7f0d08005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.472+0000 7f0d1dd9b640 1 --2- 172.21.15.76:0/3544418917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d08004800 0x7f0d08004c80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400ba30 tx=0x7f0d08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90426f5e3233aab0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.472+0000 7f0d1d59a640 1 -- 172.21.15.76:0/3544418917 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d14036020 con 0x7f0d08004800 2024-03-08T06:01:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.472+0000 7f0d1d59a640 1 -- 172.21.15.76:0/3544418917 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d14032640 con 0x7f0d08004800 2024-03-08T06:01:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.472+0000 7f0d1d59a640 1 -- 172.21.15.76:0/3544418917 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1400b600 con 0x7f0d08004800 2024-03-08T06:01:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.473+0000 7f0d24e55640 1 -- 172.21.15.76:0/3544418917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d08004800 msgr2=0x7f0d08004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.473+0000 7f0d24e55640 1 --2- 172.21.15.76:0/3544418917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d08004800 0x7f0d08004c80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400ba30 tx=0x7f0d08005830 comp rx=0 tx=0).stop 2024-03-08T06:01:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.473+0000 7f0d1ed9d640 1 -- 172.21.15.76:0/3544418917 reap_dead start 2024-03-08T06:01:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.473+0000 7f0d24e55640 1 -- 172.21.15.76:0/3544418917 shutdown_connections 2024-03-08T06:01:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.473+0000 7f0d24e55640 1 -- 172.21.15.76:0/3544418917 >> 172.21.15.76:0/3544418917 conn(0x7f0d200670d0 msgr2=0x7f0d200d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.473+0000 7f0d24e55640 1 -- 172.21.15.76:0/3544418917 shutdown_connections 2024-03-08T06:01:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.473+0000 7f0d24e55640 1 -- 172.21.15.76:0/3544418917 wait complete. 2024-03-08T06:01:24.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.474+0000 7f0d24e55640 1 Processor -- start 2024-03-08T06:01:24.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.474+0000 7f0d24e55640 1 -- start start 2024-03-08T06:01:24.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d24e55640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d08004800 0x7f0d201e6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:24.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d24e55640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0d201e6690 0x7f0d20145fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:24.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d24e55640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d201464e0 0x7f0d20146990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:24.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d24e55640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2015dfa0 con 0x7f0d08004800 2024-03-08T06:01:24.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d24e55640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200d3870 con 0x7f0d201464e0 2024-03-08T06:01:24.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d24e55640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2015e120 con 0x7f0d201e6690 2024-03-08T06:01:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d1e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d08004800 0x7f0d201e6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d1e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d08004800 0x7f0d201e6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41834/0 (socket says 172.21.15.76:41834) 2024-03-08T06:01:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d1e59c640 1 -- 172.21.15.76:0/1089074359 learned_addr learned my addr 172.21.15.76:0/1089074359 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d1ed9d640 1 --2- 172.21.15.76:0/1089074359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d201464e0 0x7f0d20146990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d1dd9b640 1 --2- 172.21.15.76:0/1089074359 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0d201e6690 0x7f0d20145fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d1e59c640 1 -- 172.21.15.76:0/1089074359 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0d201e6690 msgr2=0x7f0d20145fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d1e59c640 1 --2- 172.21.15.76:0/1089074359 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0d201e6690 0x7f0d20145fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d1e59c640 1 -- 172.21.15.76:0/1089074359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d201464e0 msgr2=0x7f0d20146990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d1e59c640 1 --2- 172.21.15.76:0/1089074359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d201464e0 0x7f0d20146990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d1e59c640 1 -- 172.21.15.76:0/1089074359 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d20156310 con 0x7f0d08004800 2024-03-08T06:01:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d1ed9d640 1 --2- 172.21.15.76:0/1089074359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d201464e0 0x7f0d20146990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d1e59c640 1 --2- 172.21.15.76:0/1089074359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d08004800 0x7f0d201e6150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d10001560 tx=0x7f0d1000a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d0effd640 1 -- 172.21.15.76:0/1089074359 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10017020 con 0x7f0d08004800 2024-03-08T06:01:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d0effd640 1 -- 172.21.15.76:0/1089074359 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d10007ab0 con 0x7f0d08004800 2024-03-08T06:01:24.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d24e55640 1 -- 172.21.15.76:0/1089074359 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d20159480 con 0x7f0d08004800 2024-03-08T06:01:24.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d0effd640 1 -- 172.21.15.76:0/1089074359 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10013600 con 0x7f0d08004800 2024-03-08T06:01:24.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.475+0000 7f0d24e55640 1 -- 172.21.15.76:0/1089074359 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d20143f40 con 0x7f0d08004800 2024-03-08T06:01:24.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.476+0000 7f0d24e55640 1 -- 172.21.15.76:0/1089074359 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2004f030 con 0x7f0d08004800 2024-03-08T06:01:24.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.477+0000 7f0d0effd640 1 -- 172.21.15.76:0/1089074359 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d1001a020 con 0x7f0d08004800 2024-03-08T06:01:24.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.477+0000 7f0d0effd640 1 --2- 172.21.15.76:0/1089074359 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0d0006b4d0 0x7f0d0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:24.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.477+0000 7f0d0effd640 1 -- 172.21.15.76:0/1089074359 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0d1008af50 con 0x7f0d08004800 2024-03-08T06:01:24.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.477+0000 7f0d1dd9b640 1 --2- 172.21.15.76:0/1089074359 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0d0006b4d0 0x7f0d0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:24.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.478+0000 7f0d1dd9b640 1 --2- 172.21.15.76:0/1089074359 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0d0006b4d0 0x7f0d0006d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400ba00 tx=0x7f0d14030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:24.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.482+0000 7f0d0effd640 1 -- 172.21.15.76:0/1089074359 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d10021020 con 0x7f0d08004800 2024-03-08T06:01:24.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.653+0000 7f0d24e55640 1 -- 172.21.15.76:0/1089074359 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0d201e6a70 con 0x7f0d08004800 2024-03-08T06:01:24.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.658+0000 7f0d0effd640 1 -- 172.21.15.76:0/1089074359 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0d10059410 con 0x7f0d08004800 2024-03-08T06:01:24.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.660+0000 7f0d24e55640 1 -- 172.21.15.76:0/1089074359 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0d0006b4d0 msgr2=0x7f0d0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:24.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.660+0000 7f0d24e55640 1 --2- 172.21.15.76:0/1089074359 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0d0006b4d0 0x7f0d0006d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400ba00 tx=0x7f0d14030000 comp rx=0 tx=0).stop 2024-03-08T06:01:24.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.660+0000 7f0d24e55640 1 -- 172.21.15.76:0/1089074359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d08004800 msgr2=0x7f0d201e6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:24.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.660+0000 7f0d24e55640 1 --2- 172.21.15.76:0/1089074359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d08004800 0x7f0d201e6150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d10001560 tx=0x7f0d1000a6f0 comp rx=0 tx=0).stop 2024-03-08T06:01:24.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.660+0000 7f0d24e55640 1 -- 172.21.15.76:0/1089074359 shutdown_connections 2024-03-08T06:01:24.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.660+0000 7f0d24e55640 1 --2- 172.21.15.76:0/1089074359 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0d0006b4d0 0x7f0d0006d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:24.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.660+0000 7f0d24e55640 1 --2- 172.21.15.76:0/1089074359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d201464e0 0x7f0d20146990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:24.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.660+0000 7f0d24e55640 1 --2- 172.21.15.76:0/1089074359 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0d201e6690 0x7f0d20145fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:24.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.660+0000 7f0d24e55640 1 --2- 172.21.15.76:0/1089074359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d08004800 0x7f0d201e6150 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:24.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.660+0000 7f0d24e55640 1 -- 172.21.15.76:0/1089074359 >> 172.21.15.76:0/1089074359 conn(0x7f0d200670d0 msgr2=0x7f0d200ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:24.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.660+0000 7f0d24e55640 1 -- 172.21.15.76:0/1089074359 shutdown_connections 2024-03-08T06:01:24.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:24.660+0000 7f0d24e55640 1 -- 172.21.15.76:0/1089074359 wait complete. 2024-03-08T06:01:24.672 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-08T06:01:24.672 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-08T06:01:24.718 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-08T06:01:24.774 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-08T06:01:24.774 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:01:24.774 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:24.790 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:01:24.790 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:24.845 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:01:24.845 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:24.900 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:25.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.040+0000 7fcc11484640 1 Processor -- start 2024-03-08T06:01:25.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.040+0000 7fcc11484640 1 -- start start 2024-03-08T06:01:25.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.041+0000 7fcc11484640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcc0c15dde0 0x7fcc0c162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.041+0000 7fcc11484640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcc0c0d0b90 0x7fcc0c0d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.041+0000 7fcc11484640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0c162790 con 0x7fcc0c0b6d00 2024-03-08T06:01:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.041+0000 7fcc11484640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0c0cf180 con 0x7fcc0c15dde0 2024-03-08T06:01:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.041+0000 7fcc11484640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0c0d1ed0 con 0x7fcc0c0d0b90 2024-03-08T06:01:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.041+0000 7fcc0affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcc0c15dde0 0x7fcc0c162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.041+0000 7fcc0affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcc0c15dde0 0x7fcc0c162250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41848/0 (socket says 172.21.15.76:41848) 2024-03-08T06:01:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.041+0000 7fcc0affd640 1 -- 172.21.15.76:0/3797424401 learned_addr learned my addr 172.21.15.76:0/3797424401 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.041+0000 7fcc0b7fe640 1 --2- 172.21.15.76:0/3797424401 >> v2:172.21.15.81:3300/0 conn(0x7fcc0c0d0b90 0x7fcc0c0d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.041+0000 7fcc0affd640 1 -- 172.21.15.76:0/3797424401 >> v2:172.21.15.81:3300/0 conn(0x7fcc0c0d0b90 msgr2=0x7fcc0c0d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.041+0000 7fcc0affd640 1 --2- 172.21.15.76:0/3797424401 >> v2:172.21.15.81:3300/0 conn(0x7fcc0c0d0b90 0x7fcc0c0d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.041+0000 7fcc0affd640 1 -- 172.21.15.76:0/3797424401 >> v1:172.21.15.81:6789/0 conn(0x7fcc0c0b6d00 legacy=0x7fcc0c0d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.041+0000 7fcc0affd640 1 -- 172.21.15.76:0/3797424401 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c0d21b0 con 0x7fcc0c15dde0 2024-03-08T06:01:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.041+0000 7fcc0b7fe640 1 --2- 172.21.15.76:0/3797424401 >> v2:172.21.15.81:3300/0 conn(0x7fcc0c0d0b90 0x7fcc0c0d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.042+0000 7fcc0affd640 1 --2- 172.21.15.76:0/3797424401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc0c15dde0 0x7fcc0c162250 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c162960 tx=0x7fcbfc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8321f300472ddef server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:25.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.042+0000 7fcc09ffb640 1 -- 172.21.15.76:0/3797424401 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc02c9e0 con 0x7fcc0c15dde0 2024-03-08T06:01:25.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.042+0000 7fcc09ffb640 1 -- 172.21.15.76:0/3797424401 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc02cb80 con 0x7fcc0c15dde0 2024-03-08T06:01:25.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.042+0000 7fcc09ffb640 1 -- 172.21.15.76:0/3797424401 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc0343f0 con 0x7fcc0c15dde0 2024-03-08T06:01:25.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.043+0000 7fcc11484640 1 -- 172.21.15.76:0/3797424401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc0c15dde0 msgr2=0x7fcc0c162250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.043+0000 7fcc11484640 1 --2- 172.21.15.76:0/3797424401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc0c15dde0 0x7fcc0c162250 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c162960 tx=0x7fcbfc02be40 comp rx=0 tx=0).stop 2024-03-08T06:01:25.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.043+0000 7fcc11484640 1 -- 172.21.15.76:0/3797424401 shutdown_connections 2024-03-08T06:01:25.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.043+0000 7fcc11484640 1 --2- 172.21.15.76:0/3797424401 >> v2:172.21.15.81:3300/0 conn(0x7fcc0c0d0b90 0x7fcc0c0d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.043+0000 7fcc11484640 1 --2- 172.21.15.76:0/3797424401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc0c15dde0 0x7fcc0c162250 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.043+0000 7fcc11484640 1 -- 172.21.15.76:0/3797424401 >> 172.21.15.76:0/3797424401 conn(0x7fcc0c0b56e0 msgr2=0x7fcc0c0b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:25.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.043+0000 7fcc11484640 1 -- 172.21.15.76:0/3797424401 shutdown_connections 2024-03-08T06:01:25.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.043+0000 7fcc11484640 1 -- 172.21.15.76:0/3797424401 wait complete. 2024-03-08T06:01:25.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.044+0000 7fcc11484640 1 Processor -- start 2024-03-08T06:01:25.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.044+0000 7fcc11484640 1 -- start start 2024-03-08T06:01:25.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc11484640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc0c0b6d00 0x7fcc0c1f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc11484640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc0c0d0b90 0x7fcc0c0ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc11484640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc0c15dde0 0x7fcc0c1427e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc11484640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0d2ef0 con 0x7fcc0c0b6d00 2024-03-08T06:01:25.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc11484640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0cee20 con 0x7fcc0c0d0b90 2024-03-08T06:01:25.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc11484640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0d3070 con 0x7fcc0c15dde0 2024-03-08T06:01:25.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc0affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc0c0b6d00 0x7fcc0c1f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc0affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc0c0b6d00 0x7fcc0c1f4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41850/0 (socket says 172.21.15.76:41850) 2024-03-08T06:01:25.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc0affd640 1 -- 172.21.15.76:0/13882991 learned_addr learned my addr 172.21.15.76:0/13882991 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:25.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc0a7fc640 1 --2- 172.21.15.76:0/13882991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc0c0d0b90 0x7fcc0c0ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc0b7fe640 1 --2- 172.21.15.76:0/13882991 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc0c15dde0 0x7fcc0c1427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc0affd640 1 -- 172.21.15.76:0/13882991 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc0c15dde0 msgr2=0x7fcc0c1427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc0affd640 1 --2- 172.21.15.76:0/13882991 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc0c15dde0 0x7fcc0c1427e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc0affd640 1 -- 172.21.15.76:0/13882991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc0c0d0b90 msgr2=0x7fcc0c0ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc0affd640 1 --2- 172.21.15.76:0/13882991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc0c0d0b90 0x7fcc0c0ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc0affd640 1 -- 172.21.15.76:0/13882991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c0ca560 con 0x7fcc0c0b6d00 2024-03-08T06:01:25.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc0b7fe640 1 --2- 172.21.15.76:0/13882991 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc0c15dde0 0x7fcc0c1427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:25.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc0affd640 1 --2- 172.21.15.76:0/13882991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc0c0b6d00 0x7fcc0c1f4c60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc02c450 tx=0x7fcbfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:25.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcbf37fe640 1 -- 172.21.15.76:0/13882991 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc043020 con 0x7fcc0c0b6d00 2024-03-08T06:01:25.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcbf37fe640 1 -- 172.21.15.76:0/13882991 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc03e450 con 0x7fcc0c0b6d00 2024-03-08T06:01:25.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcc11484640 1 -- 172.21.15.76:0/13882991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0c1f7600 con 0x7fcc0c0b6d00 2024-03-08T06:01:25.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.045+0000 7fcbf37fe640 1 -- 172.21.15.76:0/13882991 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc009510 con 0x7fcc0c0b6d00 2024-03-08T06:01:25.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.046+0000 7fcc11484640 1 -- 172.21.15.76:0/13882991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0c143310 con 0x7fcc0c0b6d00 2024-03-08T06:01:25.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.046+0000 7fcc11484640 1 -- 172.21.15.76:0/13882991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0c04f030 con 0x7fcc0c0b6d00 2024-03-08T06:01:25.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.047+0000 7fcbf37fe640 1 -- 172.21.15.76:0/13882991 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbfc005d10 con 0x7fcc0c0b6d00 2024-03-08T06:01:25.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.047+0000 7fcbf37fe640 1 --2- 172.21.15.76:0/13882991 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcbf806b4d0 0x7fcbf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.047+0000 7fcbf37fe640 1 -- 172.21.15.76:0/13882991 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fcbfc0ad380 con 0x7fcc0c0b6d00 2024-03-08T06:01:25.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.047+0000 7fcc0a7fc640 1 --2- 172.21.15.76:0/13882991 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcbf806b4d0 0x7fcbf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.048+0000 7fcc0a7fc640 1 --2- 172.21.15.76:0/13882991 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcbf806b4d0 0x7fcbf806d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcc00007fd0 tx=0x7fcc00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:25.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.052+0000 7fcbf37fe640 1 -- 172.21.15.76:0/13882991 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcbfc07bd70 con 0x7fcc0c0b6d00 2024-03-08T06:01:25.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.222+0000 7fcc11484640 1 -- 172.21.15.76:0/13882991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc0c0b41b0 con 0x7fcc0c0b6d00 2024-03-08T06:01:25.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.223+0000 7fcbf37fe640 1 -- 172.21.15.76:0/13882991 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18757 (secure 0 0 0) 0x7fcbfc07b670 con 0x7fcc0c0b6d00 2024-03-08T06:01:25.224 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:25.224 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":28,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:01:17.313272+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:25.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.225+0000 7fcc11484640 1 -- 172.21.15.76:0/13882991 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcbf806b4d0 msgr2=0x7fcbf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.225+0000 7fcc11484640 1 --2- 172.21.15.76:0/13882991 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcbf806b4d0 0x7fcbf806d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcc00007fd0 tx=0x7fcc00009040 comp rx=0 tx=0).stop 2024-03-08T06:01:25.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.225+0000 7fcc11484640 1 -- 172.21.15.76:0/13882991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc0c0b6d00 msgr2=0x7fcc0c1f4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.225+0000 7fcc11484640 1 --2- 172.21.15.76:0/13882991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc0c0b6d00 0x7fcc0c1f4c60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc02c450 tx=0x7fcbfc004a40 comp rx=0 tx=0).stop 2024-03-08T06:01:25.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.225+0000 7fcc11484640 1 -- 172.21.15.76:0/13882991 shutdown_connections 2024-03-08T06:01:25.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.225+0000 7fcc11484640 1 --2- 172.21.15.76:0/13882991 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fcbf806b4d0 0x7fcbf806d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.225+0000 7fcc11484640 1 --2- 172.21.15.76:0/13882991 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcc0c15dde0 0x7fcc0c1427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.225+0000 7fcc11484640 1 --2- 172.21.15.76:0/13882991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc0c0d0b90 0x7fcc0c0ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.225+0000 7fcc11484640 1 --2- 172.21.15.76:0/13882991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc0c0b6d00 0x7fcc0c1f4c60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.225+0000 7fcc11484640 1 -- 172.21.15.76:0/13882991 >> 172.21.15.76:0/13882991 conn(0x7fcc0c0b56e0 msgr2=0x7fcc0c0cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:25.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.225+0000 7fcc11484640 1 -- 172.21.15.76:0/13882991 shutdown_connections 2024-03-08T06:01:25.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.225+0000 7fcc11484640 1 -- 172.21.15.76:0/13882991 wait complete. 2024-03-08T06:01:25.236 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:00:33.926651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:00:34.474030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T06:00:38.207853+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T06:01:25.237 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:01:25.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.339+0000 7fe289e9d640 1 Processor -- start 2024-03-08T06:01:25.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.339+0000 7fe289e9d640 1 -- start start 2024-03-08T06:01:25.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.340+0000 7fe289e9d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe2840cc0d0 0x7fe2840cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.340+0000 7fe289e9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe28415af70 0x7fe28415b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.340+0000 7fe289e9d640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe28415f450 con 0x7fe284154df0 2024-03-08T06:01:25.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.340+0000 7fe289e9d640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2840af1e0 con 0x7fe28415af70 2024-03-08T06:01:25.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.340+0000 7fe289e9d640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2840b3c80 con 0x7fe2840cc0d0 2024-03-08T06:01:25.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.341+0000 7fe283fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe28415af70 0x7fe28415b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.341+0000 7fe282ffd640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe2840cc0d0 0x7fe2840cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.341+0000 7fe283fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe28415af70 0x7fe28415b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.76:53720/0 (socket says 172.21.15.76:53720) 2024-03-08T06:01:25.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.341+0000 7fe2837fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe284154df0 0x7fe2840b3100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:47906/0 (socket says 172.21.15.76:47906) 2024-03-08T06:01:25.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.341+0000 7fe283fff640 1 -- 172.21.15.76:0/1161073643 learned_addr learned my addr 172.21.15.76:0/1161073643 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:25.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.341+0000 7fe283fff640 1 -- 172.21.15.76:0/1161073643 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe2840cc0d0 msgr2=0x7fe2840cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:25.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.341+0000 7fe283fff640 1 --2- 172.21.15.76:0/1161073643 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe2840cc0d0 0x7fe2840cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.341+0000 7fe283fff640 1 -- 172.21.15.76:0/1161073643 >> v1:172.21.15.81:6789/0 conn(0x7fe284154df0 legacy=0x7fe2840b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.341+0000 7fe283fff640 1 -- 172.21.15.76:0/1161073643 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28415e690 con 0x7fe28415af70 2024-03-08T06:01:25.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.341+0000 7fe282ffd640 1 --2- 172.21.15.76:0/1161073643 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe2840cc0d0 0x7fe2840cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:25.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.342+0000 7fe283fff640 1 --2- 172.21.15.76:0/1161073643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe28415af70 0x7fe28415b390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009e50 tx=0x7fe27c02bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87a4b980008c75ae server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:25.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.342+0000 7fe2827fc640 1 -- 172.21.15.76:0/1161073643 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c02ca70 con 0x7fe28415af70 2024-03-08T06:01:25.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.342+0000 7fe2827fc640 1 -- 172.21.15.76:0/1161073643 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c02cc10 con 0x7fe28415af70 2024-03-08T06:01:25.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.342+0000 7fe2827fc640 1 -- 172.21.15.76:0/1161073643 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c0343f0 con 0x7fe28415af70 2024-03-08T06:01:25.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.342+0000 7fe289e9d640 1 -- 172.21.15.76:0/1161073643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe28415af70 msgr2=0x7fe28415b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.342+0000 7fe289e9d640 1 --2- 172.21.15.76:0/1161073643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe28415af70 0x7fe28415b390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009e50 tx=0x7fe27c02bed0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.342+0000 7fe289e9d640 1 -- 172.21.15.76:0/1161073643 shutdown_connections 2024-03-08T06:01:25.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.342+0000 7fe289e9d640 1 --2- 172.21.15.76:0/1161073643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe28415af70 0x7fe28415b390 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.342+0000 7fe289e9d640 1 --2- 172.21.15.76:0/1161073643 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe2840cc0d0 0x7fe2840cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.342+0000 7fe289e9d640 1 -- 172.21.15.76:0/1161073643 >> 172.21.15.76:0/1161073643 conn(0x7fe28406ac80 msgr2=0x7fe28406b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:25.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.342+0000 7fe289e9d640 1 -- 172.21.15.76:0/1161073643 shutdown_connections 2024-03-08T06:01:25.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.343+0000 7fe289e9d640 1 -- 172.21.15.76:0/1161073643 wait complete. 2024-03-08T06:01:25.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.344+0000 7fe289e9d640 1 Processor -- start 2024-03-08T06:01:25.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.344+0000 7fe289e9d640 1 -- start start 2024-03-08T06:01:25.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.344+0000 7fe289e9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe2840cc0d0 0x7fe284151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.344+0000 7fe289e9d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe284154df0 0x7fe2840a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe289e9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe28415af70 0x7fe284159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe289e9d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28415ef30 con 0x7fe28415af70 2024-03-08T06:01:25.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe289e9d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28415ec50 con 0x7fe2840cc0d0 2024-03-08T06:01:25.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe289e9d640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2841551d0 con 0x7fe284154df0 2024-03-08T06:01:25.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe283fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe28415af70 0x7fe284159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe283fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe28415af70 0x7fe284159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41858/0 (socket says 172.21.15.76:41858) 2024-03-08T06:01:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe283fff640 1 -- 172.21.15.76:0/763582196 learned_addr learned my addr 172.21.15.76:0/763582196 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe2837fe640 1 --2- 172.21.15.76:0/763582196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe2840cc0d0 0x7fe284151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe282ffd640 1 --2- 172.21.15.76:0/763582196 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe284154df0 0x7fe2840a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe283fff640 1 -- 172.21.15.76:0/763582196 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe284154df0 msgr2=0x7fe2840a8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe283fff640 1 --2- 172.21.15.76:0/763582196 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe284154df0 0x7fe2840a8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe283fff640 1 -- 172.21.15.76:0/763582196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe2840cc0d0 msgr2=0x7fe284151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe283fff640 1 --2- 172.21.15.76:0/763582196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe2840cc0d0 0x7fe284151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe283fff640 1 -- 172.21.15.76:0/763582196 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2840ab6b0 con 0x7fe28415af70 2024-03-08T06:01:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe2837fe640 1 --2- 172.21.15.76:0/763582196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe2840cc0d0 0x7fe284151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe283fff640 1 --2- 172.21.15.76:0/763582196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe28415af70 0x7fe284159930 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe27c036040 tx=0x7fe27c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe26bfff640 1 -- 172.21.15.76:0/763582196 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c043020 con 0x7fe28415af70 2024-03-08T06:01:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe26bfff640 1 -- 172.21.15.76:0/763582196 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c03e450 con 0x7fe28415af70 2024-03-08T06:01:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe26bfff640 1 -- 172.21.15.76:0/763582196 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c009510 con 0x7fe28415af70 2024-03-08T06:01:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe289e9d640 1 -- 172.21.15.76:0/763582196 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2840aa230 con 0x7fe28415af70 2024-03-08T06:01:25.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.345+0000 7fe289e9d640 1 -- 172.21.15.76:0/763582196 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2840a9c70 con 0x7fe28415af70 2024-03-08T06:01:25.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.346+0000 7fe289e9d640 1 -- 172.21.15.76:0/763582196 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe28404f030 con 0x7fe28415af70 2024-03-08T06:01:25.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.347+0000 7fe26bfff640 1 -- 172.21.15.76:0/763582196 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe27c046020 con 0x7fe28415af70 2024-03-08T06:01:25.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.347+0000 7fe26bfff640 1 --2- 172.21.15.76:0/763582196 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe27006b4d0 0x7fe27006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.347+0000 7fe26bfff640 1 -- 172.21.15.76:0/763582196 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe27c0acd00 con 0x7fe28415af70 2024-03-08T06:01:25.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.347+0000 7fe2837fe640 1 --2- 172.21.15.76:0/763582196 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe27006b4d0 0x7fe27006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.348+0000 7fe2837fe640 1 --2- 172.21.15.76:0/763582196 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe27006b4d0 0x7fe27006d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe274007fd0 tx=0x7fe274007d60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:25.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.351+0000 7fe26bfff640 1 -- 172.21.15.76:0/763582196 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe27c04d030 con 0x7fe28415af70 2024-03-08T06:01:25.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.523+0000 7fe289e9d640 1 -- 172.21.15.76:0/763582196 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2840ae9a0 con 0x7fe28415af70 2024-03-08T06:01:25.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.523+0000 7fe26bfff640 1 -- 172.21.15.76:0/763582196 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe27c07b810 con 0x7fe28415af70 2024-03-08T06:01:25.524 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:01:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.525+0000 7fe289e9d640 1 -- 172.21.15.76:0/763582196 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe27006b4d0 msgr2=0x7fe27006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.525+0000 7fe289e9d640 1 --2- 172.21.15.76:0/763582196 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe27006b4d0 0x7fe27006d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe274007fd0 tx=0x7fe274007d60 comp rx=0 tx=0).stop 2024-03-08T06:01:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.525+0000 7fe289e9d640 1 -- 172.21.15.76:0/763582196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe28415af70 msgr2=0x7fe284159930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.525+0000 7fe289e9d640 1 --2- 172.21.15.76:0/763582196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe28415af70 0x7fe284159930 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe27c036040 tx=0x7fe27c004a40 comp rx=0 tx=0).stop 2024-03-08T06:01:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.525+0000 7fe289e9d640 1 -- 172.21.15.76:0/763582196 shutdown_connections 2024-03-08T06:01:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.525+0000 7fe289e9d640 1 --2- 172.21.15.76:0/763582196 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe27006b4d0 0x7fe27006d990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.525+0000 7fe289e9d640 1 --2- 172.21.15.76:0/763582196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe28415af70 0x7fe284159930 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.525+0000 7fe289e9d640 1 --2- 172.21.15.76:0/763582196 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe284154df0 0x7fe2840a8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.525+0000 7fe289e9d640 1 --2- 172.21.15.76:0/763582196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe2840cc0d0 0x7fe284151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.525+0000 7fe289e9d640 1 -- 172.21.15.76:0/763582196 >> 172.21.15.76:0/763582196 conn(0x7fe28406ac80 msgr2=0x7fe28406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:25.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.525+0000 7fe289e9d640 1 -- 172.21.15.76:0/763582196 shutdown_connections 2024-03-08T06:01:25.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.525+0000 7fe289e9d640 1 -- 172.21.15.76:0/763582196 wait complete. 2024-03-08T06:01:25.536 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:01:25.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.640+0000 7face55d7640 1 Processor -- start 2024-03-08T06:01:25.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.640+0000 7face55d7640 1 -- start start 2024-03-08T06:01:25.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7face55d7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7face015dbc0 0x7face00d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7face55d7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face00d27d0 con 0x7face00b9840 2024-03-08T06:01:25.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7face55d7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face00d2ab0 con 0x7face00b5cf0 2024-03-08T06:01:25.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7face55d7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face00d2d90 con 0x7face015dbc0 2024-03-08T06:01:25.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facde7fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7face00b9840 0x7face00b55e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39580/0 (socket says 172.21.15.76:39580) 2024-03-08T06:01:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facde7fc640 1 -- 172.21.15.76:0/1441265677 learned_addr learned my addr 172.21.15.76:0/1441265677 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facdeffd640 1 --2- 172.21.15.76:0/1441265677 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7face015dbc0 0x7face00d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facddffb640 1 -- 172.21.15.76:0/1441265677 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997691866 0 0) 0x7face00d27d0 con 0x7face00b9840 2024-03-08T06:01:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facddffb640 1 -- 172.21.15.76:0/1441265677 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facc8003520 con 0x7face00b9840 2024-03-08T06:01:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facddffb640 1 -- 172.21.15.76:0/1441265677 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1944702119 0 0) 0x7facc8003520 con 0x7face00b9840 2024-03-08T06:01:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facddffb640 1 -- 172.21.15.76:0/1441265677 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7face015dbc0 msgr2=0x7face00d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facddffb640 1 --2- 172.21.15.76:0/1441265677 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7face015dbc0 0x7face00d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facddffb640 1 -- 172.21.15.76:0/1441265677 >> v1:172.21.15.81:6789/0 conn(0x7face00b5cf0 legacy=0x7face00b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facddffb640 1 -- 172.21.15.76:0/1441265677 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face00d3070 con 0x7face00b9840 2024-03-08T06:01:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facddffb640 1 -- 172.21.15.76:0/1441265677 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7facd4001670 con 0x7face00b9840 2024-03-08T06:01:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facddffb640 1 -- 172.21.15.76:0/1441265677 >> v1:172.21.15.76:6789/0 conn(0x7face00b9840 legacy=0x7face00b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facddffb640 1 --2- 172.21.15.76:0/1441265677 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7face015dbc0 0x7facc8003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facddffb640 1 --2- 172.21.15.76:0/1441265677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facc8003f90 0x7facc8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facddffb640 1 --2- 172.21.15.76:0/1441265677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facc8004d40 0x7facc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facddffb640 1 -- 172.21.15.76:0/1441265677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face00d2d90 con 0x7facc8004d40 2024-03-08T06:01:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facddffb640 1 -- 172.21.15.76:0/1441265677 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc800baa0 con 0x7facc8003f90 2024-03-08T06:01:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.641+0000 7facddffb640 1 -- 172.21.15.76:0/1441265677 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc800bca0 con 0x7face015dbc0 2024-03-08T06:01:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.642+0000 7facdf7fe640 1 --2- 172.21.15.76:0/1441265677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facc8004d40 0x7facc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.642+0000 7facdeffd640 1 --2- 172.21.15.76:0/1441265677 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7face015dbc0 0x7facc8003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.642+0000 7facde7fc640 1 --2- 172.21.15.76:0/1441265677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facc8003f90 0x7facc8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.642+0000 7facdf7fe640 1 -- 172.21.15.76:0/1441265677 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7face015dbc0 msgr2=0x7facc8003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.642+0000 7facdf7fe640 1 --2- 172.21.15.76:0/1441265677 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7face015dbc0 0x7facc8003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.642+0000 7facdf7fe640 1 -- 172.21.15.76:0/1441265677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facc8003f90 msgr2=0x7facc8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.642+0000 7facdf7fe640 1 --2- 172.21.15.76:0/1441265677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facc8003f90 0x7facc8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.642+0000 7facdf7fe640 1 -- 172.21.15.76:0/1441265677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc8003520 con 0x7facc8004d40 2024-03-08T06:01:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.643+0000 7facdf7fe640 1 --2- 172.21.15.76:0/1441265677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facc8004d40 0x7facc80051c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7facd8001f60 tx=0x7facc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad577e2460d9e30c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.643+0000 7facddffb640 1 -- 172.21.15.76:0/1441265677 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd8001b30 con 0x7facc8004d40 2024-03-08T06:01:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.643+0000 7facddffb640 1 -- 172.21.15.76:0/1441265677 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd802d710 con 0x7facc8004d40 2024-03-08T06:01:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.643+0000 7facddffb640 1 -- 172.21.15.76:0/1441265677 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd802db00 con 0x7facc8004d40 2024-03-08T06:01:25.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.643+0000 7face55d7640 1 -- 172.21.15.76:0/1441265677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facc8004d40 msgr2=0x7facc80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.643+0000 7face55d7640 1 --2- 172.21.15.76:0/1441265677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facc8004d40 0x7facc80051c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7facd8001f60 tx=0x7facc8005830 comp rx=0 tx=0).stop 2024-03-08T06:01:25.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.643+0000 7facdf7fe640 1 -- 172.21.15.76:0/1441265677 reap_dead start 2024-03-08T06:01:25.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.643+0000 7face55d7640 1 -- 172.21.15.76:0/1441265677 shutdown_connections 2024-03-08T06:01:25.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.643+0000 7face55d7640 1 -- 172.21.15.76:0/1441265677 >> 172.21.15.76:0/1441265677 conn(0x7face00670d0 msgr2=0x7face00d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:25.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.643+0000 7face55d7640 1 -- 172.21.15.76:0/1441265677 shutdown_connections 2024-03-08T06:01:25.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.643+0000 7face55d7640 1 -- 172.21.15.76:0/1441265677 wait complete. 2024-03-08T06:01:25.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.645+0000 7face55d7640 1 Processor -- start 2024-03-08T06:01:25.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.645+0000 7face55d7640 1 -- start start 2024-03-08T06:01:25.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.645+0000 7face55d7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7face015bc30 0x7face01e82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.645+0000 7face55d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7face01e87f0 0x7face01e3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.645+0000 7face55d7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7face01e36b0 0x7face01e5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.645+0000 7face55d7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face015e120 con 0x7face01e87f0 2024-03-08T06:01:25.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.645+0000 7face55d7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face015dfa0 con 0x7face01e36b0 2024-03-08T06:01:25.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.645+0000 7face55d7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face00d1ef0 con 0x7face015bc30 2024-03-08T06:01:25.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.645+0000 7facde7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7face01e87f0 0x7face01e3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.645+0000 7facde7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7face01e87f0 0x7face01e3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41876/0 (socket says 172.21.15.76:41876) 2024-03-08T06:01:25.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.645+0000 7facde7fc640 1 -- 172.21.15.76:0/2086090802 learned_addr learned my addr 172.21.15.76:0/2086090802 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:25.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.646+0000 7facdeffd640 1 --2- 172.21.15.76:0/2086090802 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7face015bc30 0x7face01e82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.646+0000 7facde7fc640 1 -- 172.21.15.76:0/2086090802 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7face015bc30 msgr2=0x7face01e82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.646+0000 7facdf7fe640 1 --2- 172.21.15.76:0/2086090802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7face01e36b0 0x7face01e5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.646+0000 7facde7fc640 1 --2- 172.21.15.76:0/2086090802 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7face015bc30 0x7face01e82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.646+0000 7facde7fc640 1 -- 172.21.15.76:0/2086090802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7face01e36b0 msgr2=0x7face01e5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.646+0000 7facde7fc640 1 --2- 172.21.15.76:0/2086090802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7face01e36b0 0x7face01e5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.646+0000 7facde7fc640 1 -- 172.21.15.76:0/2086090802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face01e3ea0 con 0x7face01e87f0 2024-03-08T06:01:25.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.646+0000 7facde7fc640 1 --2- 172.21.15.76:0/2086090802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7face01e87f0 0x7face01e3170 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7facd4007290 tx=0x7facd4007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:25.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.646+0000 7facc77fe640 1 -- 172.21.15.76:0/2086090802 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd400f960 con 0x7face01e87f0 2024-03-08T06:01:25.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.646+0000 7facc77fe640 1 -- 172.21.15.76:0/2086090802 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd4018070 con 0x7face01e87f0 2024-03-08T06:01:25.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.646+0000 7facc77fe640 1 -- 172.21.15.76:0/2086090802 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd400fd40 con 0x7face01e87f0 2024-03-08T06:01:25.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.646+0000 7face55d7640 1 -- 172.21.15.76:0/2086090802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face01e4180 con 0x7face01e87f0 2024-03-08T06:01:25.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.646+0000 7face55d7640 1 -- 172.21.15.76:0/2086090802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face01e4740 con 0x7face01e87f0 2024-03-08T06:01:25.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.647+0000 7face55d7640 1 -- 172.21.15.76:0/2086090802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face01e4940 con 0x7face01e87f0 2024-03-08T06:01:25.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.647+0000 7facc77fe640 1 -- 172.21.15.76:0/2086090802 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7facd401d460 con 0x7face01e87f0 2024-03-08T06:01:25.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.648+0000 7facc77fe640 1 --2- 172.21.15.76:0/2086090802 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faccc06b4d0 0x7faccc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.648+0000 7facdeffd640 1 --2- 172.21.15.76:0/2086090802 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faccc06b4d0 0x7faccc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.648+0000 7facc77fe640 1 -- 172.21.15.76:0/2086090802 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7facd408a640 con 0x7face01e87f0 2024-03-08T06:01:25.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.648+0000 7facdeffd640 1 --2- 172.21.15.76:0/2086090802 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faccc06b4d0 0x7faccc06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7face00cda50 tx=0x7facd0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:25.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.652+0000 7facc77fe640 1 -- 172.21.15.76:0/2086090802 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7facd40590c0 con 0x7face01e87f0 2024-03-08T06:01:25.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.820+0000 7face55d7640 1 -- 172.21.15.76:0/2086090802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7face004f030 con 0x7face01e87f0 2024-03-08T06:01:25.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.821+0000 7facc77fe640 1 -- 172.21.15.76:0/2086090802 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7facd40589c0 con 0x7face01e87f0 2024-03-08T06:01:25.821 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:25.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.823+0000 7face55d7640 1 -- 172.21.15.76:0/2086090802 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faccc06b4d0 msgr2=0x7faccc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.823+0000 7face55d7640 1 --2- 172.21.15.76:0/2086090802 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faccc06b4d0 0x7faccc06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7face00cda50 tx=0x7facd0008040 comp rx=0 tx=0).stop 2024-03-08T06:01:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.823+0000 7face55d7640 1 -- 172.21.15.76:0/2086090802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7face01e87f0 msgr2=0x7face01e3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.823+0000 7face55d7640 1 --2- 172.21.15.76:0/2086090802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7face01e87f0 0x7face01e3170 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7facd4007290 tx=0x7facd4007750 comp rx=0 tx=0).stop 2024-03-08T06:01:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.823+0000 7face55d7640 1 -- 172.21.15.76:0/2086090802 shutdown_connections 2024-03-08T06:01:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.823+0000 7face55d7640 1 --2- 172.21.15.76:0/2086090802 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faccc06b4d0 0x7faccc06d990 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.823+0000 7face55d7640 1 --2- 172.21.15.76:0/2086090802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7face01e36b0 0x7face01e5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.823+0000 7face55d7640 1 --2- 172.21.15.76:0/2086090802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7face01e87f0 0x7face01e3170 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.823+0000 7face55d7640 1 --2- 172.21.15.76:0/2086090802 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7face015bc30 0x7face01e82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.823+0000 7face55d7640 1 -- 172.21.15.76:0/2086090802 >> 172.21.15.76:0/2086090802 conn(0x7face00670d0 msgr2=0x7face00ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.823+0000 7face55d7640 1 -- 172.21.15.76:0/2086090802 shutdown_connections 2024-03-08T06:01:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.823+0000 7face55d7640 1 -- 172.21.15.76:0/2086090802 wait complete. 2024-03-08T06:01:25.834 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:01:25.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.937+0000 7f49c5647640 1 Processor -- start 2024-03-08T06:01:25.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.937+0000 7f49c5647640 1 -- start start 2024-03-08T06:01:25.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.938+0000 7f49c5647640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f49c015df30 0x7f49c00cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.938+0000 7f49c5647640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f49c00d0220 0x7f49c00b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.938+0000 7f49c5647640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c00b2160 con 0x7f49c00b1d50 2024-03-08T06:01:25.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.938+0000 7f49c5647640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c00cd440 con 0x7f49c015df30 2024-03-08T06:01:25.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.938+0000 7f49c5647640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c00cd690 con 0x7f49c00d0220 2024-03-08T06:01:25.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.938+0000 7f49beffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f49c015df30 0x7f49c00cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.938+0000 7f49bf7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f49c00b1d50 0x7f49c006ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39592/0 (socket says 172.21.15.76:39592) 2024-03-08T06:01:25.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.938+0000 7f49bf7fe640 1 -- 172.21.15.76:0/1827459120 learned_addr learned my addr 172.21.15.76:0/1827459120 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:25.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.938+0000 7f49beffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f49c015df30 0x7f49c00cfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41886/0 (socket says 172.21.15.76:41886) 2024-03-08T06:01:25.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.938+0000 7f49be7fc640 1 --2- 172.21.15.76:0/1827459120 >> v2:172.21.15.81:3300/0 conn(0x7f49c00d0220 0x7f49c00b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.939+0000 7f49beffd640 1 -- 172.21.15.76:0/1827459120 >> v2:172.21.15.81:3300/0 conn(0x7f49c00d0220 msgr2=0x7f49c00b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:25.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.939+0000 7f49beffd640 1 --2- 172.21.15.76:0/1827459120 >> v2:172.21.15.81:3300/0 conn(0x7f49c00d0220 0x7f49c00b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.939+0000 7f49beffd640 1 -- 172.21.15.76:0/1827459120 >> v1:172.21.15.76:6789/0 conn(0x7f49c00b1d50 legacy=0x7f49c006ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.939+0000 7f49beffd640 1 -- 172.21.15.76:0/1827459120 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00cd970 con 0x7f49c015df30 2024-03-08T06:01:25.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.939+0000 7f49bdffb640 1 -- 172.21.15.76:0/1827459120 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3699290411 0 0) 0x7f49b8001110 con 0x7f49c00b1d50 2024-03-08T06:01:25.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.939+0000 7f49beffd640 1 --2- 172.21.15.76:0/1827459120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49c015df30 0x7f49c00cfce0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f49c0072210 tx=0x7f49b002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b390dc4c889d98fe server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:25.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.939+0000 7f49bdffb640 1 -- 172.21.15.76:0/1827459120 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0039070 con 0x7f49c015df30 2024-03-08T06:01:25.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.939+0000 7f49bdffb640 1 -- 172.21.15.76:0/1827459120 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b002cc80 con 0x7f49c015df30 2024-03-08T06:01:25.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.939+0000 7f49bdffb640 1 -- 172.21.15.76:0/1827459120 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0034660 con 0x7f49c015df30 2024-03-08T06:01:25.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.940+0000 7f49c5647640 1 -- 172.21.15.76:0/1827459120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49c015df30 msgr2=0x7f49c00cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.940+0000 7f49c5647640 1 --2- 172.21.15.76:0/1827459120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49c015df30 0x7f49c00cfce0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f49c0072210 tx=0x7f49b002bed0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.940+0000 7f49c5647640 1 -- 172.21.15.76:0/1827459120 shutdown_connections 2024-03-08T06:01:25.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.940+0000 7f49c5647640 1 --2- 172.21.15.76:0/1827459120 >> v2:172.21.15.81:3300/0 conn(0x7f49c00d0220 0x7f49c00b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.940+0000 7f49c5647640 1 --2- 172.21.15.76:0/1827459120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49c015df30 0x7f49c00cfce0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.940+0000 7f49c5647640 1 -- 172.21.15.76:0/1827459120 >> 172.21.15.76:0/1827459120 conn(0x7f49c00d0850 msgr2=0x7f49c00d0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:25.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.940+0000 7f49c5647640 1 -- 172.21.15.76:0/1827459120 shutdown_connections 2024-03-08T06:01:25.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.940+0000 7f49c5647640 1 -- 172.21.15.76:0/1827459120 wait complete. 2024-03-08T06:01:25.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.941+0000 7f49c5647640 1 Processor -- start 2024-03-08T06:01:25.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.941+0000 7f49c5647640 1 -- start start 2024-03-08T06:01:25.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49c5647640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49c00b1d50 0x7f49c00b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49c5647640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f49c00d0220 0x7f49c00b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49c5647640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49c015df30 0x7f49c00b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49c5647640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c015e310 con 0x7f49c00b1d50 2024-03-08T06:01:25.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49c5647640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00cdf30 con 0x7f49c015df30 2024-03-08T06:01:25.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49c5647640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00cc300 con 0x7f49c00d0220 2024-03-08T06:01:25.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49beffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49c00b1d50 0x7f49c00b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49beffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49c00b1d50 0x7f49c00b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41892/0 (socket says 172.21.15.76:41892) 2024-03-08T06:01:25.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49beffd640 1 -- 172.21.15.76:0/1405831371 learned_addr learned my addr 172.21.15.76:0/1405831371 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:25.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49be7fc640 1 --2- 172.21.15.76:0/1405831371 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f49c00d0220 0x7f49c00b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49bf7fe640 1 --2- 172.21.15.76:0/1405831371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49c015df30 0x7f49c00b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49beffd640 1 -- 172.21.15.76:0/1405831371 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f49c00d0220 msgr2=0x7f49c00b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49beffd640 1 --2- 172.21.15.76:0/1405831371 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f49c00d0220 0x7f49c00b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49beffd640 1 -- 172.21.15.76:0/1405831371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49c015df30 msgr2=0x7f49c00b6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:25.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49beffd640 1 --2- 172.21.15.76:0/1405831371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49c015df30 0x7f49c00b6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:25.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49beffd640 1 -- 172.21.15.76:0/1405831371 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c0161290 con 0x7f49c00b1d50 2024-03-08T06:01:25.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49be7fc640 1 --2- 172.21.15.76:0/1405831371 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f49c00d0220 0x7f49c00b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:25.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.942+0000 7f49bf7fe640 1 --2- 172.21.15.76:0/1405831371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49c015df30 0x7f49c00b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:25.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.943+0000 7f49beffd640 1 --2- 172.21.15.76:0/1405831371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49c00b1d50 0x7f49c00b47e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f49b0037040 tx=0x7f49b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:25.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.943+0000 7f49a77fe640 1 -- 172.21.15.76:0/1405831371 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0039050 con 0x7f49c00b1d50 2024-03-08T06:01:25.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.943+0000 7f49a77fe640 1 -- 172.21.15.76:0/1405831371 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b0045070 con 0x7f49c00b1d50 2024-03-08T06:01:25.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.943+0000 7f49c5647640 1 -- 172.21.15.76:0/1405831371 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c00b3740 con 0x7f49c00b1d50 2024-03-08T06:01:25.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.943+0000 7f49a77fe640 1 -- 172.21.15.76:0/1405831371 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b003e850 con 0x7f49c00b1d50 2024-03-08T06:01:25.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.943+0000 7f49c5647640 1 -- 172.21.15.76:0/1405831371 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c00b3d00 con 0x7f49c00b1d50 2024-03-08T06:01:25.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.944+0000 7f49c5647640 1 -- 172.21.15.76:0/1405831371 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c00b4420 con 0x7f49c00b1d50 2024-03-08T06:01:25.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.944+0000 7f49a77fe640 1 -- 172.21.15.76:0/1405831371 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49b003e9f0 con 0x7f49c00b1d50 2024-03-08T06:01:25.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.945+0000 7f49a77fe640 1 --2- 172.21.15.76:0/1405831371 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f49ac06b500 0x7f49ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:25.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.945+0000 7f49a77fe640 1 -- 172.21.15.76:0/1405831371 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f49b00ad2b0 con 0x7f49c00b1d50 2024-03-08T06:01:25.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.945+0000 7f49be7fc640 1 --2- 172.21.15.76:0/1405831371 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f49ac06b500 0x7f49ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:25.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.945+0000 7f49be7fc640 1 --2- 172.21.15.76:0/1405831371 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f49ac06b500 0x7f49ac06d9c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f49c01551d0 tx=0x7f49b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:25.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:25.949+0000 7f49a77fe640 1 -- 172.21.15.76:0/1405831371 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f49b007bdc0 con 0x7f49c00b1d50 2024-03-08T06:01:26.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.117+0000 7f49c5647640 1 -- 172.21.15.76:0/1405831371 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f49c004f030 con 0x7f49c00b1d50 2024-03-08T06:01:26.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.117+0000 7f49a77fe640 1 -- 172.21.15.76:0/1405831371 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f49b007b6c0 con 0x7f49c00b1d50 2024-03-08T06:01:26.118 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:01:26.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.119+0000 7f49c5647640 1 -- 172.21.15.76:0/1405831371 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f49ac06b500 msgr2=0x7f49ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:26.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.119+0000 7f49c5647640 1 --2- 172.21.15.76:0/1405831371 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f49ac06b500 0x7f49ac06d9c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f49c01551d0 tx=0x7f49b4008040 comp rx=0 tx=0).stop 2024-03-08T06:01:26.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.119+0000 7f49c5647640 1 -- 172.21.15.76:0/1405831371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49c00b1d50 msgr2=0x7f49c00b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:26.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.119+0000 7f49c5647640 1 --2- 172.21.15.76:0/1405831371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49c00b1d50 0x7f49c00b47e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f49b0037040 tx=0x7f49b0004a40 comp rx=0 tx=0).stop 2024-03-08T06:01:26.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.119+0000 7f49c5647640 1 -- 172.21.15.76:0/1405831371 shutdown_connections 2024-03-08T06:01:26.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.119+0000 7f49c5647640 1 --2- 172.21.15.76:0/1405831371 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f49ac06b500 0x7f49ac06d9c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.119+0000 7f49c5647640 1 --2- 172.21.15.76:0/1405831371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49c015df30 0x7f49c00b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.119+0000 7f49c5647640 1 --2- 172.21.15.76:0/1405831371 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f49c00d0220 0x7f49c00b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.119+0000 7f49c5647640 1 --2- 172.21.15.76:0/1405831371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49c00b1d50 0x7f49c00b47e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.119+0000 7f49c5647640 1 -- 172.21.15.76:0/1405831371 >> 172.21.15.76:0/1405831371 conn(0x7f49c00d0850 msgr2=0x7f49c006f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:26.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.119+0000 7f49c5647640 1 -- 172.21.15.76:0/1405831371 shutdown_connections 2024-03-08T06:01:26.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.119+0000 7f49c5647640 1 -- 172.21.15.76:0/1405831371 wait complete. 2024-03-08T06:01:26.131 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T06:01:26.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.233+0000 7f1aae519640 1 Processor -- start 2024-03-08T06:01:26.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.234+0000 7f1aae519640 1 -- start start 2024-03-08T06:01:26.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.234+0000 7f1aae519640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1aa815dcb0 0x7f1aa80ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:26.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.234+0000 7f1aae519640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1aa80af950 0x7f1aa806fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:26.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.234+0000 7f1aae519640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1aa80afdb0 0x7f1aa8070040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:26.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.234+0000 7f1aae519640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa80cd580 con 0x7f1aa80afdb0 2024-03-08T06:01:26.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.234+0000 7f1aae519640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa80cd860 con 0x7f1aa815dcb0 2024-03-08T06:01:26.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.234+0000 7f1aae519640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa80cdb40 con 0x7f1aa80af950 2024-03-08T06:01:26.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.234+0000 7f1aacab6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1aa80afdb0 0x7f1aa8070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:26.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.235+0000 7f1aacab6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1aa80afdb0 0x7f1aa8070040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41894/0 (socket says 172.21.15.76:41894) 2024-03-08T06:01:26.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.235+0000 7f1aacab6640 1 -- 172.21.15.76:0/266143008 learned_addr learned my addr 172.21.15.76:0/266143008 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:26.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.235+0000 7f1aa7fff640 1 --2- 172.21.15.76:0/266143008 >> v2:172.21.15.81:3300/0 conn(0x7f1aa815dcb0 0x7f1aa80ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:26.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.235+0000 7f1aa77fe640 1 --2- 172.21.15.76:0/266143008 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1aa80af950 0x7f1aa806fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:26.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.235+0000 7f1aacab6640 1 -- 172.21.15.76:0/266143008 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1aa80af950 msgr2=0x7f1aa806fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:26.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.235+0000 7f1aacab6640 1 --2- 172.21.15.76:0/266143008 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1aa80af950 0x7f1aa806fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.235+0000 7f1aacab6640 1 -- 172.21.15.76:0/266143008 >> v2:172.21.15.81:3300/0 conn(0x7f1aa815dcb0 msgr2=0x7f1aa80ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:26.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.235+0000 7f1aacab6640 1 --2- 172.21.15.76:0/266143008 >> v2:172.21.15.81:3300/0 conn(0x7f1aa815dcb0 0x7f1aa80ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.235+0000 7f1aacab6640 1 -- 172.21.15.76:0/266143008 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa80cde20 con 0x7f1aa80afdb0 2024-03-08T06:01:26.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.235+0000 7f1aa7fff640 1 --2- 172.21.15.76:0/266143008 >> v2:172.21.15.81:3300/0 conn(0x7f1aa815dcb0 0x7f1aa80ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:26.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.235+0000 7f1aa77fe640 1 --2- 172.21.15.76:0/266143008 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1aa80af950 0x7f1aa806fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:26.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.236+0000 7f1aacab6640 1 --2- 172.21.15.76:0/266143008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa80afdb0 0x7f1aa8070040 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8073230 tx=0x7f1aa002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41878e4fcf031b4b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:26.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.236+0000 7f1aa6ffd640 1 -- 172.21.15.76:0/266143008 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa002ca70 con 0x7f1aa80afdb0 2024-03-08T06:01:26.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.236+0000 7f1aa6ffd640 1 -- 172.21.15.76:0/266143008 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa002cc10 con 0x7f1aa80afdb0 2024-03-08T06:01:26.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.236+0000 7f1aa6ffd640 1 -- 172.21.15.76:0/266143008 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa00343f0 con 0x7f1aa80afdb0 2024-03-08T06:01:26.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.236+0000 7f1aae519640 1 -- 172.21.15.76:0/266143008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa80afdb0 msgr2=0x7f1aa8070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:26.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.236+0000 7f1aae519640 1 --2- 172.21.15.76:0/266143008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa80afdb0 0x7f1aa8070040 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8073230 tx=0x7f1aa002bed0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.236+0000 7f1aae519640 1 -- 172.21.15.76:0/266143008 shutdown_connections 2024-03-08T06:01:26.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.236+0000 7f1aae519640 1 --2- 172.21.15.76:0/266143008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa80afdb0 0x7f1aa8070040 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.236+0000 7f1aae519640 1 --2- 172.21.15.76:0/266143008 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1aa80af950 0x7f1aa806fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.236+0000 7f1aae519640 1 --2- 172.21.15.76:0/266143008 >> v2:172.21.15.81:3300/0 conn(0x7f1aa815dcb0 0x7f1aa80ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.236+0000 7f1aae519640 1 -- 172.21.15.76:0/266143008 >> 172.21.15.76:0/266143008 conn(0x7f1aa80b2320 msgr2=0x7f1aa80b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:26.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.236+0000 7f1aae519640 1 -- 172.21.15.76:0/266143008 shutdown_connections 2024-03-08T06:01:26.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.236+0000 7f1aae519640 1 -- 172.21.15.76:0/266143008 wait complete. 2024-03-08T06:01:26.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.237+0000 7f1aae519640 1 Processor -- start 2024-03-08T06:01:26.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.238+0000 7f1aae519640 1 -- start start 2024-03-08T06:01:26.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.238+0000 7f1aae519640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1aa80af950 0x7f1aa80b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:26.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.238+0000 7f1aae519640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1aa80afdb0 0x7f1aa81f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:26.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.238+0000 7f1aae519640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa815dcb0 0x7f1aa815b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:26.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.238+0000 7f1aae519640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa80cd360 con 0x7f1aa815dcb0 2024-03-08T06:01:26.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.238+0000 7f1aae519640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa80cd1e0 con 0x7f1aa80afdb0 2024-03-08T06:01:26.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.238+0000 7f1aae519640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa815e090 con 0x7f1aa80af950 2024-03-08T06:01:26.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.238+0000 7f1aacab6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa815dcb0 0x7f1aa815b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:26.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.238+0000 7f1aacab6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa815dcb0 0x7f1aa815b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41906/0 (socket says 172.21.15.76:41906) 2024-03-08T06:01:26.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.238+0000 7f1aacab6640 1 -- 172.21.15.76:0/1714947516 learned_addr learned my addr 172.21.15.76:0/1714947516 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:26.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.239+0000 7f1aa77fe640 1 --2- 172.21.15.76:0/1714947516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1aa80afdb0 0x7f1aa81f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:26.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.239+0000 7f1aa7fff640 1 --2- 172.21.15.76:0/1714947516 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1aa80af950 0x7f1aa80b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:26.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.239+0000 7f1aacab6640 1 -- 172.21.15.76:0/1714947516 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1aa80af950 msgr2=0x7f1aa80b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:26.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.239+0000 7f1aacab6640 1 --2- 172.21.15.76:0/1714947516 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1aa80af950 0x7f1aa80b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.239+0000 7f1aacab6640 1 -- 172.21.15.76:0/1714947516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1aa80afdb0 msgr2=0x7f1aa81f3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:26.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.239+0000 7f1aacab6640 1 --2- 172.21.15.76:0/1714947516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1aa80afdb0 0x7f1aa81f3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.239+0000 7f1aacab6640 1 -- 172.21.15.76:0/1714947516 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa80b42f0 con 0x7f1aa815dcb0 2024-03-08T06:01:26.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.239+0000 7f1aa7fff640 1 --2- 172.21.15.76:0/1714947516 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1aa80af950 0x7f1aa80b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:26.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.239+0000 7f1aacab6640 1 --2- 172.21.15.76:0/1714947516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa815dcb0 0x7f1aa815b880 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0036040 tx=0x7f1aa0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:26.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.239+0000 7f1aa4ff9640 1 -- 172.21.15.76:0/1714947516 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0043020 con 0x7f1aa815dcb0 2024-03-08T06:01:26.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.239+0000 7f1aa4ff9640 1 -- 172.21.15.76:0/1714947516 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa003e450 con 0x7f1aa815dcb0 2024-03-08T06:01:26.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.239+0000 7f1aae519640 1 -- 172.21.15.76:0/1714947516 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa81f3cd0 con 0x7f1aa815dcb0 2024-03-08T06:01:26.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.239+0000 7f1aa4ff9640 1 -- 172.21.15.76:0/1714947516 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0009510 con 0x7f1aa815dcb0 2024-03-08T06:01:26.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.239+0000 7f1aae519640 1 -- 172.21.15.76:0/1714947516 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa80b35b0 con 0x7f1aa815dcb0 2024-03-08T06:01:26.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.240+0000 7f1aae519640 1 -- 172.21.15.76:0/1714947516 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa804f030 con 0x7f1aa815dcb0 2024-03-08T06:01:26.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.240+0000 7f1aa4ff9640 1 -- 172.21.15.76:0/1714947516 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1aa0046020 con 0x7f1aa815dcb0 2024-03-08T06:01:26.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.241+0000 7f1aa4ff9640 1 --2- 172.21.15.76:0/1714947516 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1a9406b4d0 0x7f1a9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:26.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.241+0000 7f1aa4ff9640 1 -- 172.21.15.76:0/1714947516 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1aa00acef0 con 0x7f1aa815dcb0 2024-03-08T06:01:26.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.241+0000 7f1aa7fff640 1 --2- 172.21.15.76:0/1714947516 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1a9406b4d0 0x7f1a9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:26.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.241+0000 7f1aa7fff640 1 --2- 172.21.15.76:0/1714947516 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1a9406b4d0 0x7f1a9406d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80c8050 tx=0x7f1a98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:26.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.245+0000 7f1aa4ff9640 1 -- 172.21.15.76:0/1714947516 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1aa004d030 con 0x7f1aa815dcb0 2024-03-08T06:01:26.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.417+0000 7f1aae519640 1 -- 172.21.15.76:0/1714947516 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1aa80c7050 con 0x7f1aa815dcb0 2024-03-08T06:01:26.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.417+0000 7f1aa4ff9640 1 -- 172.21.15.76:0/1714947516 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1aa007b970 con 0x7f1aa815dcb0 2024-03-08T06:01:26.418 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.419+0000 7f1aae519640 1 -- 172.21.15.76:0/1714947516 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1a9406b4d0 msgr2=0x7f1a9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.419+0000 7f1aae519640 1 --2- 172.21.15.76:0/1714947516 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1a9406b4d0 0x7f1a9406d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80c8050 tx=0x7f1a98008040 comp rx=0 tx=0).stop 2024-03-08T06:01:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.419+0000 7f1aae519640 1 -- 172.21.15.76:0/1714947516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa815dcb0 msgr2=0x7f1aa815b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.419+0000 7f1aae519640 1 --2- 172.21.15.76:0/1714947516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa815dcb0 0x7f1aa815b880 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0036040 tx=0x7f1aa0004a40 comp rx=0 tx=0).stop 2024-03-08T06:01:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.419+0000 7f1aae519640 1 -- 172.21.15.76:0/1714947516 shutdown_connections 2024-03-08T06:01:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.419+0000 7f1aae519640 1 --2- 172.21.15.76:0/1714947516 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1a9406b4d0 0x7f1a9406d990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.419+0000 7f1aae519640 1 --2- 172.21.15.76:0/1714947516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa815dcb0 0x7f1aa815b880 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.419+0000 7f1aae519640 1 --2- 172.21.15.76:0/1714947516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1aa80afdb0 0x7f1aa81f3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.419+0000 7f1aae519640 1 --2- 172.21.15.76:0/1714947516 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1aa80af950 0x7f1aa80b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.419+0000 7f1aae519640 1 -- 172.21.15.76:0/1714947516 >> 172.21.15.76:0/1714947516 conn(0x7f1aa80b2320 msgr2=0x7f1aa80af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:26.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.419+0000 7f1aae519640 1 -- 172.21.15.76:0/1714947516 shutdown_connections 2024-03-08T06:01:26.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.419+0000 7f1aae519640 1 -- 172.21.15.76:0/1714947516 wait complete. 2024-03-08T06:01:26.431 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T06:01:26.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.531+0000 7eff4aa15640 1 Processor -- start 2024-03-08T06:01:26.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.531+0000 7eff4aa15640 1 -- start start 2024-03-08T06:01:26.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.532+0000 7eff4aa15640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7eff4415dbc0 0x7eff440d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:26.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.532+0000 7eff4aa15640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7eff440b5cf0 0x7eff440b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:26.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.532+0000 7eff4aa15640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff440d27d0 con 0x7eff440b9840 2024-03-08T06:01:26.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.532+0000 7eff4aa15640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff440d2ab0 con 0x7eff440b5cf0 2024-03-08T06:01:26.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.532+0000 7eff4aa15640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff440d2d90 con 0x7eff4415dbc0 2024-03-08T06:01:26.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.532+0000 7eff48fb2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7eff440b5cf0 0x7eff440b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:26.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.532+0000 7eff48fb2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7eff440b5cf0 0x7eff440b6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41920/0 (socket says 172.21.15.76:41920) 2024-03-08T06:01:26.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.532+0000 7eff48fb2640 1 -- 172.21.15.76:0/248478774 learned_addr learned my addr 172.21.15.76:0/248478774 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:26.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.533+0000 7eff43fff640 1 --2- 172.21.15.76:0/248478774 >> v2:172.21.15.81:3300/0 conn(0x7eff4415dbc0 0x7eff440d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:26.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.533+0000 7eff48fb2640 1 -- 172.21.15.76:0/248478774 >> v2:172.21.15.81:3300/0 conn(0x7eff4415dbc0 msgr2=0x7eff440d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:26.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.533+0000 7eff48fb2640 1 --2- 172.21.15.76:0/248478774 >> v2:172.21.15.81:3300/0 conn(0x7eff4415dbc0 0x7eff440d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.533+0000 7eff48fb2640 1 -- 172.21.15.76:0/248478774 >> v1:172.21.15.81:6789/0 conn(0x7eff440b9840 legacy=0x7eff440b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:26.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.533+0000 7eff48fb2640 1 -- 172.21.15.76:0/248478774 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff440d3070 con 0x7eff440b5cf0 2024-03-08T06:01:26.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.533+0000 7eff48fb2640 1 --2- 172.21.15.76:0/248478774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff440b5cf0 0x7eff440b6c90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7eff4406d280 tx=0x7eff3c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c2267bb7d2fb0e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:26.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.534+0000 7eff42ffd640 1 -- 172.21.15.76:0/248478774 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c02ca80 con 0x7eff440b5cf0 2024-03-08T06:01:26.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.534+0000 7eff42ffd640 1 -- 172.21.15.76:0/248478774 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3c02cc20 con 0x7eff440b5cf0 2024-03-08T06:01:26.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.534+0000 7eff42ffd640 1 -- 172.21.15.76:0/248478774 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0343f0 con 0x7eff440b5cf0 2024-03-08T06:01:26.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.534+0000 7eff4aa15640 1 -- 172.21.15.76:0/248478774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff440b5cf0 msgr2=0x7eff440b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:26.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.534+0000 7eff4aa15640 1 --2- 172.21.15.76:0/248478774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff440b5cf0 0x7eff440b6c90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7eff4406d280 tx=0x7eff3c02bee0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.534+0000 7eff4aa15640 1 -- 172.21.15.76:0/248478774 shutdown_connections 2024-03-08T06:01:26.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.534+0000 7eff4aa15640 1 --2- 172.21.15.76:0/248478774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff440b5cf0 0x7eff440b6c90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.534+0000 7eff4aa15640 1 --2- 172.21.15.76:0/248478774 >> v2:172.21.15.81:3300/0 conn(0x7eff4415dbc0 0x7eff440d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.534+0000 7eff4aa15640 1 -- 172.21.15.76:0/248478774 >> 172.21.15.76:0/248478774 conn(0x7eff440670d0 msgr2=0x7eff440d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:26.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.534+0000 7eff4aa15640 1 -- 172.21.15.76:0/248478774 shutdown_connections 2024-03-08T06:01:26.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.534+0000 7eff4aa15640 1 -- 172.21.15.76:0/248478774 wait complete. 2024-03-08T06:01:26.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.535+0000 7eff4aa15640 1 Processor -- start 2024-03-08T06:01:26.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.536+0000 7eff4aa15640 1 -- start start 2024-03-08T06:01:26.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.536+0000 7eff4aa15640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff440b5cf0 0x7eff440ced80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:26.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.536+0000 7eff4aa15640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff440b9840 0x7eff44158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:26.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.536+0000 7eff4aa15640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7eff4415dbc0 0x7eff44159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:26.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.536+0000 7eff4aa15640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff440d25b0 con 0x7eff440b5cf0 2024-03-08T06:01:26.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.536+0000 7eff4aa15640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff440d2430 con 0x7eff440b9840 2024-03-08T06:01:26.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.536+0000 7eff4aa15640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4415dfa0 con 0x7eff4415dbc0 2024-03-08T06:01:26.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.536+0000 7eff43fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff440b5cf0 0x7eff440ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:26.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.536+0000 7eff43fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff440b5cf0 0x7eff440ced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41934/0 (socket says 172.21.15.76:41934) 2024-03-08T06:01:26.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.536+0000 7eff43fff640 1 -- 172.21.15.76:0/566995085 learned_addr learned my addr 172.21.15.76:0/566995085 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:26.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.537+0000 7eff437fe640 1 --2- 172.21.15.76:0/566995085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff440b9840 0x7eff44158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:26.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.537+0000 7eff48fb2640 1 --2- 172.21.15.76:0/566995085 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7eff4415dbc0 0x7eff44159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:26.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.537+0000 7eff43fff640 1 -- 172.21.15.76:0/566995085 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7eff4415dbc0 msgr2=0x7eff44159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:26.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.537+0000 7eff43fff640 1 --2- 172.21.15.76:0/566995085 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7eff4415dbc0 0x7eff44159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.537+0000 7eff43fff640 1 -- 172.21.15.76:0/566995085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff440b9840 msgr2=0x7eff44158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:26.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.537+0000 7eff43fff640 1 --2- 172.21.15.76:0/566995085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff440b9840 0x7eff44158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.537+0000 7eff43fff640 1 -- 172.21.15.76:0/566995085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff441f34d0 con 0x7eff440b5cf0 2024-03-08T06:01:26.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.537+0000 7eff43fff640 1 --2- 172.21.15.76:0/566995085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff440b5cf0 0x7eff440ced80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7eff3400d620 tx=0x7eff3400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:26.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.537+0000 7eff40ff9640 1 -- 172.21.15.76:0/566995085 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3400b620 con 0x7eff440b5cf0 2024-03-08T06:01:26.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.537+0000 7eff40ff9640 1 -- 172.21.15.76:0/566995085 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff34019070 con 0x7eff440b5cf0 2024-03-08T06:01:26.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.537+0000 7eff40ff9640 1 -- 172.21.15.76:0/566995085 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3400ba00 con 0x7eff440b5cf0 2024-03-08T06:01:26.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.537+0000 7eff4aa15640 1 -- 172.21.15.76:0/566995085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff441f6dc0 con 0x7eff440b5cf0 2024-03-08T06:01:26.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.537+0000 7eff4aa15640 1 -- 172.21.15.76:0/566995085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff441f7380 con 0x7eff440b5cf0 2024-03-08T06:01:26.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.538+0000 7eff4aa15640 1 -- 172.21.15.76:0/566995085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff4404f030 con 0x7eff440b5cf0 2024-03-08T06:01:26.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.538+0000 7eff40ff9640 1 -- 172.21.15.76:0/566995085 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff3400bba0 con 0x7eff440b5cf0 2024-03-08T06:01:26.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.539+0000 7eff40ff9640 1 --2- 172.21.15.76:0/566995085 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7eff3006b4d0 0x7eff3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:26.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.539+0000 7eff40ff9640 1 -- 172.21.15.76:0/566995085 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7eff3408aa20 con 0x7eff440b5cf0 2024-03-08T06:01:26.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.539+0000 7eff437fe640 1 --2- 172.21.15.76:0/566995085 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7eff3006b4d0 0x7eff3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:26.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.539+0000 7eff437fe640 1 --2- 172.21.15.76:0/566995085 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7eff3006b4d0 0x7eff3006d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7eff440c9c20 tx=0x7eff38009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:26.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.543+0000 7eff40ff9640 1 -- 172.21.15.76:0/566995085 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7eff340594a0 con 0x7eff440b5cf0 2024-03-08T06:01:26.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.726+0000 7eff4aa15640 1 -- 172.21.15.76:0/566995085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7eff4415b770 con 0x7eff440b5cf0 2024-03-08T06:01:26.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.726+0000 7eff40ff9640 1 -- 172.21.15.76:0/566995085 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7eff34058da0 con 0x7eff440b5cf0 2024-03-08T06:01:26.726 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-03-08T06:01:26.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.728+0000 7eff4aa15640 1 -- 172.21.15.76:0/566995085 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7eff3006b4d0 msgr2=0x7eff3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:26.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.728+0000 7eff4aa15640 1 --2- 172.21.15.76:0/566995085 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7eff3006b4d0 0x7eff3006d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7eff440c9c20 tx=0x7eff38009040 comp rx=0 tx=0).stop 2024-03-08T06:01:26.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.728+0000 7eff4aa15640 1 -- 172.21.15.76:0/566995085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff440b5cf0 msgr2=0x7eff440ced80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:26.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.728+0000 7eff4aa15640 1 --2- 172.21.15.76:0/566995085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff440b5cf0 0x7eff440ced80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7eff3400d620 tx=0x7eff3400dae0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.728+0000 7eff4aa15640 1 -- 172.21.15.76:0/566995085 shutdown_connections 2024-03-08T06:01:26.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.728+0000 7eff4aa15640 1 --2- 172.21.15.76:0/566995085 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7eff3006b4d0 0x7eff3006d990 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.728+0000 7eff4aa15640 1 --2- 172.21.15.76:0/566995085 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7eff4415dbc0 0x7eff44159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.728+0000 7eff4aa15640 1 --2- 172.21.15.76:0/566995085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff440b9840 0x7eff44158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.728+0000 7eff4aa15640 1 --2- 172.21.15.76:0/566995085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff440b5cf0 0x7eff440ced80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:26.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.728+0000 7eff4aa15640 1 -- 172.21.15.76:0/566995085 >> 172.21.15.76:0/566995085 conn(0x7eff440670d0 msgr2=0x7eff440c83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:26.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.728+0000 7eff4aa15640 1 -- 172.21.15.76:0/566995085 shutdown_connections 2024-03-08T06:01:26.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:26.728+0000 7eff4aa15640 1 -- 172.21.15.76:0/566995085 wait complete. 2024-03-08T06:01:26.739 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-08T06:01:26.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:26.740 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-08T06:01:26.768 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-03-08T06:01:26.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:26.768 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-08T06:01:26.823 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:01:26.823 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:26.842 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:01:26.842 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:26.899 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:01:26.899 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:26.956 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:27.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.057+0000 7f8dde16c640 1 Processor -- start 2024-03-08T06:01:27.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.057+0000 7f8dde16c640 1 -- start start 2024-03-08T06:01:27.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.058+0000 7f8dde16c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8dd815e040 0x7f8dd8099b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.058+0000 7f8dde16c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd80cfcc0 con 0x7f8dd806bfa0 2024-03-08T06:01:27.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.058+0000 7f8dde16c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd80cff60 con 0x7f8dd815dc60 2024-03-08T06:01:27.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.058+0000 7f8dde16c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd80b4360 con 0x7f8dd815e040 2024-03-08T06:01:27.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.058+0000 7f8dd6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8dd815e040 0x7f8dd8099b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.058+0000 7f8dd6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8dd815e040 0x7f8dd8099b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41938/0 (socket says 172.21.15.76:41938) 2024-03-08T06:01:27.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.058+0000 7f8dd7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8dd806bfa0 0x7f8dd80cbfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39600/0 (socket says 172.21.15.76:39600) 2024-03-08T06:01:27.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.058+0000 7f8dd6ffd640 1 -- 172.21.15.76:0/419367272 learned_addr learned my addr 172.21.15.76:0/419367272 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:27.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.058+0000 7f8dd6ffd640 1 -- 172.21.15.76:0/419367272 >> v1:172.21.15.81:6789/0 conn(0x7f8dd815dc60 legacy=0x7f8dd80d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:27.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.058+0000 7f8dd6ffd640 1 -- 172.21.15.76:0/419367272 >> v1:172.21.15.76:6789/0 conn(0x7f8dd806bfa0 legacy=0x7f8dd80cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.058+0000 7f8dd6ffd640 1 -- 172.21.15.76:0/419367272 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd80d0240 con 0x7f8dd815e040 2024-03-08T06:01:27.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.058+0000 7f8dd67fc640 1 -- 172.21.15.76:0/419367272 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829925125 0 0) 0x7f8dd80cfcc0 con 0x7f8dd806bfa0 2024-03-08T06:01:27.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.059+0000 7f8dd6ffd640 1 --2- 172.21.15.76:0/419367272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dd815e040 0x7f8dd8099b40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009870 tx=0x7f8dcc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528d022384dc2409 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:27.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.059+0000 7f8dd67fc640 1 -- 172.21.15.76:0/419367272 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc034070 con 0x7f8dd815e040 2024-03-08T06:01:27.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.059+0000 7f8dd67fc640 1 -- 172.21.15.76:0/419367272 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc030860 con 0x7f8dd815e040 2024-03-08T06:01:27.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.059+0000 7f8dd67fc640 1 -- 172.21.15.76:0/419367272 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc030c30 con 0x7f8dd815e040 2024-03-08T06:01:27.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.060+0000 7f8dde16c640 1 -- 172.21.15.76:0/419367272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dd815e040 msgr2=0x7f8dd8099b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.060+0000 7f8dde16c640 1 --2- 172.21.15.76:0/419367272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dd815e040 0x7f8dd8099b40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009870 tx=0x7f8dcc02b9a0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.060+0000 7f8dde16c640 1 -- 172.21.15.76:0/419367272 shutdown_connections 2024-03-08T06:01:27.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.060+0000 7f8dde16c640 1 --2- 172.21.15.76:0/419367272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dd815e040 0x7f8dd8099b40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.060+0000 7f8dde16c640 1 -- 172.21.15.76:0/419367272 >> 172.21.15.76:0/419367272 conn(0x7f8dd80b3440 msgr2=0x7f8dd80b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:27.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.060+0000 7f8dde16c640 1 -- 172.21.15.76:0/419367272 shutdown_connections 2024-03-08T06:01:27.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.060+0000 7f8dde16c640 1 -- 172.21.15.76:0/419367272 wait complete. 2024-03-08T06:01:27.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.061+0000 7f8dde16c640 1 Processor -- start 2024-03-08T06:01:27.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.061+0000 7f8dde16c640 1 -- start start 2024-03-08T06:01:27.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.062+0000 7f8dde16c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8dd806bfa0 0x7f8dd80b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.062+0000 7f8dde16c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dd815dc60 0x7f8dd80c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.062+0000 7f8dde16c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8dd815e040 0x7f8dd8156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.062+0000 7f8dde16c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd80d0ae0 con 0x7f8dd815dc60 2024-03-08T06:01:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.062+0000 7f8dde16c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd80d0800 con 0x7f8dd806bfa0 2024-03-08T06:01:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.062+0000 7f8dde16c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd809a890 con 0x7f8dd815e040 2024-03-08T06:01:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.062+0000 7f8dd6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dd815dc60 0x7f8dd80c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.062+0000 7f8dd6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dd815dc60 0x7f8dd80c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41954/0 (socket says 172.21.15.76:41954) 2024-03-08T06:01:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.062+0000 7f8dd6ffd640 1 -- 172.21.15.76:0/1979994571 learned_addr learned my addr 172.21.15.76:0/1979994571 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.062+0000 7f8dd77fe640 1 --2- 172.21.15.76:0/1979994571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8dd806bfa0 0x7f8dd80b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.062+0000 7f8dd7fff640 1 --2- 172.21.15.76:0/1979994571 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8dd815e040 0x7f8dd8156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.062+0000 7f8dd6ffd640 1 -- 172.21.15.76:0/1979994571 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8dd815e040 msgr2=0x7f8dd8156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.062+0000 7f8dd6ffd640 1 --2- 172.21.15.76:0/1979994571 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8dd815e040 0x7f8dd8156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.062+0000 7f8dd6ffd640 1 -- 172.21.15.76:0/1979994571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8dd806bfa0 msgr2=0x7f8dd80b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.062+0000 7f8dd6ffd640 1 --2- 172.21.15.76:0/1979994571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8dd806bfa0 0x7f8dd80b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.062+0000 7f8dd6ffd640 1 -- 172.21.15.76:0/1979994571 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd8156590 con 0x7f8dd815dc60 2024-03-08T06:01:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.063+0000 7f8dd6ffd640 1 --2- 172.21.15.76:0/1979994571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dd815dc60 0x7f8dd80c6330 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009870 tx=0x7f8dcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.063+0000 7f8dbffff640 1 -- 172.21.15.76:0/1979994571 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc034040 con 0x7f8dd815dc60 2024-03-08T06:01:27.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.063+0000 7f8dbffff640 1 -- 172.21.15.76:0/1979994571 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc0433f0 con 0x7f8dd815dc60 2024-03-08T06:01:27.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.063+0000 7f8dde16c640 1 -- 172.21.15.76:0/1979994571 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd80b6030 con 0x7f8dd815dc60 2024-03-08T06:01:27.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.063+0000 7f8dbffff640 1 -- 172.21.15.76:0/1979994571 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0437e0 con 0x7f8dd815dc60 2024-03-08T06:01:27.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.063+0000 7f8dde16c640 1 -- 172.21.15.76:0/1979994571 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd80b65f0 con 0x7f8dd815dc60 2024-03-08T06:01:27.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.064+0000 7f8dde16c640 1 -- 172.21.15.76:0/1979994571 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd804f030 con 0x7f8dd815dc60 2024-03-08T06:01:27.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.064+0000 7f8dbffff640 1 -- 172.21.15.76:0/1979994571 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8dcc04a060 con 0x7f8dd815dc60 2024-03-08T06:01:27.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.064+0000 7f8dbffff640 1 --2- 172.21.15.76:0/1979994571 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8dc406b4d0 0x7f8dc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.065+0000 7f8dbffff640 1 -- 172.21.15.76:0/1979994571 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8dcc0ac7e0 con 0x7f8dd815dc60 2024-03-08T06:01:27.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.065+0000 7f8dd77fe640 1 --2- 172.21.15.76:0/1979994571 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8dc406b4d0 0x7f8dc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.065+0000 7f8dd77fe640 1 --2- 172.21.15.76:0/1979994571 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8dc406b4d0 0x7f8dc406d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80c8790 tx=0x7f8dc8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:27.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.069+0000 7f8dbffff640 1 -- 172.21.15.76:0/1979994571 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8dcc03e070 con 0x7f8dd815dc60 2024-03-08T06:01:27.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.239+0000 7f8dde16c640 1 -- 172.21.15.76:0/1979994571 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8dd809a670 con 0x7f8dd815dc60 2024-03-08T06:01:27.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.240+0000 7f8dbffff640 1 -- 172.21.15.76:0/1979994571 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18757 (secure 0 0 0) 0x7f8dcc07b2f0 con 0x7f8dd815dc60 2024-03-08T06:01:27.241 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:27.241 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":28,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:01:17.313272+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:27.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.242+0000 7f8dde16c640 1 -- 172.21.15.76:0/1979994571 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8dc406b4d0 msgr2=0x7f8dc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.242+0000 7f8dde16c640 1 --2- 172.21.15.76:0/1979994571 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8dc406b4d0 0x7f8dc406d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80c8790 tx=0x7f8dc8009040 comp rx=0 tx=0).stop 2024-03-08T06:01:27.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.242+0000 7f8dde16c640 1 -- 172.21.15.76:0/1979994571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dd815dc60 msgr2=0x7f8dd80c6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.242+0000 7f8dde16c640 1 --2- 172.21.15.76:0/1979994571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dd815dc60 0x7f8dd80c6330 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009870 tx=0x7f8dcc0046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.242+0000 7f8dde16c640 1 -- 172.21.15.76:0/1979994571 shutdown_connections 2024-03-08T06:01:27.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.242+0000 7f8dde16c640 1 --2- 172.21.15.76:0/1979994571 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8dc406b4d0 0x7f8dc406d990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.242+0000 7f8dde16c640 1 --2- 172.21.15.76:0/1979994571 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8dd815e040 0x7f8dd8156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.242+0000 7f8dde16c640 1 --2- 172.21.15.76:0/1979994571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dd815dc60 0x7f8dd80c6330 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.242+0000 7f8dde16c640 1 --2- 172.21.15.76:0/1979994571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8dd806bfa0 0x7f8dd80b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.242+0000 7f8dde16c640 1 -- 172.21.15.76:0/1979994571 >> 172.21.15.76:0/1979994571 conn(0x7f8dd80b3440 msgr2=0x7f8dd815ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:27.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.243+0000 7f8dde16c640 1 -- 172.21.15.76:0/1979994571 shutdown_connections 2024-03-08T06:01:27.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.243+0000 7f8dde16c640 1 -- 172.21.15.76:0/1979994571 wait complete. 2024-03-08T06:01:27.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:00:33.926651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:00:34.474030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T06:00:38.207853+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T06:01:27.254 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:01:27.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.359+0000 7fc72bcc2640 1 Processor -- start 2024-03-08T06:01:27.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.359+0000 7fc72bcc2640 1 -- start start 2024-03-08T06:01:27.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.360+0000 7fc72bcc2640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc72415dbc0 0x7fc7240d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.360+0000 7fc72bcc2640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc7240b5cf0 0x7fc7240b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.360+0000 7fc72bcc2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7240d27d0 con 0x7fc7240b9840 2024-03-08T06:01:27.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.360+0000 7fc72bcc2640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7240d2ab0 con 0x7fc7240b5cf0 2024-03-08T06:01:27.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.360+0000 7fc72bcc2640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240d2d90 con 0x7fc72415dbc0 2024-03-08T06:01:27.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.360+0000 7fc72925d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc7240b9840 0x7fc7240b55e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:47944/0 (socket says 172.21.15.76:47944) 2024-03-08T06:01:27.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.360+0000 7fc729a5e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc72415dbc0 0x7fc7240d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.360+0000 7fc72925d640 1 -- 172.21.15.76:0/3460530825 learned_addr learned my addr 172.21.15.76:0/3460530825 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:27.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.360+0000 7fc72a25f640 1 --2- 172.21.15.76:0/3460530825 >> v2:172.21.15.81:3300/0 conn(0x7fc7240b5cf0 0x7fc7240b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.360+0000 7fc72a25f640 1 -- 172.21.15.76:0/3460530825 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc72415dbc0 msgr2=0x7fc7240d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:27.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.360+0000 7fc72a25f640 1 --2- 172.21.15.76:0/3460530825 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc72415dbc0 0x7fc7240d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.360+0000 7fc72a25f640 1 -- 172.21.15.76:0/3460530825 >> v1:172.21.15.81:6789/0 conn(0x7fc7240b9840 legacy=0x7fc7240b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.360+0000 7fc72a25f640 1 -- 172.21.15.76:0/3460530825 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7240d3070 con 0x7fc7240b5cf0 2024-03-08T06:01:27.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.360+0000 7fc729a5e640 1 --2- 172.21.15.76:0/3460530825 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc72415dbc0 0x7fc7240d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:27.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.361+0000 7fc72a25f640 1 --2- 172.21.15.76:0/3460530825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7240b5cf0 0x7fc7240b6c90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009d90 tx=0x7fc71c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=377ab843d5c4cbdd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:27.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.363+0000 7fc728a5c640 1 -- 172.21.15.76:0/3460530825 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c039070 con 0x7fc7240b5cf0 2024-03-08T06:01:27.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.363+0000 7fc728a5c640 1 -- 172.21.15.76:0/3460530825 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71c02cc50 con 0x7fc7240b5cf0 2024-03-08T06:01:27.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.363+0000 7fc728a5c640 1 -- 172.21.15.76:0/3460530825 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c034440 con 0x7fc7240b5cf0 2024-03-08T06:01:27.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.364+0000 7fc72bcc2640 1 -- 172.21.15.76:0/3460530825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7240b5cf0 msgr2=0x7fc7240b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.364+0000 7fc72bcc2640 1 --2- 172.21.15.76:0/3460530825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7240b5cf0 0x7fc7240b6c90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009d90 tx=0x7fc71c02bea0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.364+0000 7fc72bcc2640 1 -- 172.21.15.76:0/3460530825 shutdown_connections 2024-03-08T06:01:27.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.364+0000 7fc72bcc2640 1 --2- 172.21.15.76:0/3460530825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7240b5cf0 0x7fc7240b6c90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.364+0000 7fc72bcc2640 1 --2- 172.21.15.76:0/3460530825 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc72415dbc0 0x7fc7240d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.364+0000 7fc72bcc2640 1 -- 172.21.15.76:0/3460530825 >> 172.21.15.76:0/3460530825 conn(0x7fc7240670d0 msgr2=0x7fc7240d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:27.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.364+0000 7fc72bcc2640 1 -- 172.21.15.76:0/3460530825 shutdown_connections 2024-03-08T06:01:27.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.364+0000 7fc72bcc2640 1 -- 172.21.15.76:0/3460530825 wait complete. 2024-03-08T06:01:27.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.365+0000 7fc72bcc2640 1 Processor -- start 2024-03-08T06:01:27.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.365+0000 7fc72bcc2640 1 -- start start 2024-03-08T06:01:27.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.366+0000 7fc72bcc2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7240b5cf0 0x7fc7241f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.366+0000 7fc72bcc2640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc7240b9840 0x7fc7241581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.366+0000 7fc72bcc2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc72415dbc0 0x7fc724152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.366+0000 7fc72bcc2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240d25b0 con 0x7fc7240b5cf0 2024-03-08T06:01:27.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.366+0000 7fc72bcc2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240d2430 con 0x7fc72415dbc0 2024-03-08T06:01:27.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.366+0000 7fc72bcc2640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72415dfa0 con 0x7fc7240b9840 2024-03-08T06:01:27.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.366+0000 7fc729a5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7240b5cf0 0x7fc7241f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.366+0000 7fc729a5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7240b5cf0 0x7fc7241f2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41962/0 (socket says 172.21.15.76:41962) 2024-03-08T06:01:27.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.366+0000 7fc729a5e640 1 -- 172.21.15.76:0/1504810279 learned_addr learned my addr 172.21.15.76:0/1504810279 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:27.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.366+0000 7fc72a25f640 1 --2- 172.21.15.76:0/1504810279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc72415dbc0 0x7fc724152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.366+0000 7fc72925d640 1 --2- 172.21.15.76:0/1504810279 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc7240b9840 0x7fc7241581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.367+0000 7fc729a5e640 1 -- 172.21.15.76:0/1504810279 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc7240b9840 msgr2=0x7fc7241581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.367+0000 7fc729a5e640 1 --2- 172.21.15.76:0/1504810279 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc7240b9840 0x7fc7241581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.367+0000 7fc729a5e640 1 -- 172.21.15.76:0/1504810279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc72415dbc0 msgr2=0x7fc724152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.367+0000 7fc729a5e640 1 --2- 172.21.15.76:0/1504810279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc72415dbc0 0x7fc724152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.367+0000 7fc729a5e640 1 -- 172.21.15.76:0/1504810279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc724156e90 con 0x7fc7240b5cf0 2024-03-08T06:01:27.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.367+0000 7fc72925d640 1 --2- 172.21.15.76:0/1504810279 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc7240b9840 0x7fc7241581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:27.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.367+0000 7fc729a5e640 1 --2- 172.21.15.76:0/1504810279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7240b5cf0 0x7fc7241f2b60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc72000e960 tx=0x7fc72000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:27.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.367+0000 7fc71a7fc640 1 -- 172.21.15.76:0/1504810279 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720017070 con 0x7fc7240b5cf0 2024-03-08T06:01:27.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.367+0000 7fc71a7fc640 1 -- 172.21.15.76:0/1504810279 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72000ca00 con 0x7fc7240b5cf0 2024-03-08T06:01:27.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.367+0000 7fc72bcc2640 1 -- 172.21.15.76:0/1504810279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc724157130 con 0x7fc7240b5cf0 2024-03-08T06:01:27.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.367+0000 7fc71a7fc640 1 -- 172.21.15.76:0/1504810279 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72000cdd0 con 0x7fc7240b5cf0 2024-03-08T06:01:27.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.367+0000 7fc72bcc2640 1 -- 172.21.15.76:0/1504810279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7241579d0 con 0x7fc7240b5cf0 2024-03-08T06:01:27.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.368+0000 7fc72bcc2640 1 -- 172.21.15.76:0/1504810279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc72404f030 con 0x7fc7240b5cf0 2024-03-08T06:01:27.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.368+0000 7fc71a7fc640 1 -- 172.21.15.76:0/1504810279 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc72001a070 con 0x7fc7240b5cf0 2024-03-08T06:01:27.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.369+0000 7fc71a7fc640 1 --2- 172.21.15.76:0/1504810279 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc70806b4d0 0x7fc70806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.369+0000 7fc71a7fc640 1 -- 172.21.15.76:0/1504810279 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc72008a430 con 0x7fc7240b5cf0 2024-03-08T06:01:27.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.369+0000 7fc72925d640 1 --2- 172.21.15.76:0/1504810279 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc70806b4d0 0x7fc70806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.369+0000 7fc72925d640 1 --2- 172.21.15.76:0/1504810279 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc70806b4d0 0x7fc70806d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc724151240 tx=0x7fc714009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:27.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.373+0000 7fc71a7fc640 1 -- 172.21.15.76:0/1504810279 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc720058860 con 0x7fc7240b5cf0 2024-03-08T06:01:27.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.543+0000 7fc72bcc2640 1 -- 172.21.15.76:0/1504810279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc72406b2b0 con 0x7fc7240b5cf0 2024-03-08T06:01:27.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.544+0000 7fc71a7fc640 1 -- 172.21.15.76:0/1504810279 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc72005c900 con 0x7fc7240b5cf0 2024-03-08T06:01:27.544 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:01:27.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.545+0000 7fc72bcc2640 1 -- 172.21.15.76:0/1504810279 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc70806b4d0 msgr2=0x7fc70806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.545+0000 7fc72bcc2640 1 --2- 172.21.15.76:0/1504810279 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc70806b4d0 0x7fc70806d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc724151240 tx=0x7fc714009040 comp rx=0 tx=0).stop 2024-03-08T06:01:27.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.545+0000 7fc72bcc2640 1 -- 172.21.15.76:0/1504810279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7240b5cf0 msgr2=0x7fc7241f2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.545+0000 7fc72bcc2640 1 --2- 172.21.15.76:0/1504810279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7240b5cf0 0x7fc7241f2b60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc72000e960 tx=0x7fc72000ee20 comp rx=0 tx=0).stop 2024-03-08T06:01:27.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.545+0000 7fc72bcc2640 1 -- 172.21.15.76:0/1504810279 shutdown_connections 2024-03-08T06:01:27.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.545+0000 7fc72bcc2640 1 --2- 172.21.15.76:0/1504810279 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc70806b4d0 0x7fc70806d990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.545+0000 7fc72bcc2640 1 --2- 172.21.15.76:0/1504810279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc72415dbc0 0x7fc724152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.545+0000 7fc72bcc2640 1 --2- 172.21.15.76:0/1504810279 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc7240b9840 0x7fc7241581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.545+0000 7fc72bcc2640 1 --2- 172.21.15.76:0/1504810279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7240b5cf0 0x7fc7241f2b60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.545+0000 7fc72bcc2640 1 -- 172.21.15.76:0/1504810279 >> 172.21.15.76:0/1504810279 conn(0x7fc7240670d0 msgr2=0x7fc7240c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:27.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.546+0000 7fc72bcc2640 1 -- 172.21.15.76:0/1504810279 shutdown_connections 2024-03-08T06:01:27.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.546+0000 7fc72bcc2640 1 -- 172.21.15.76:0/1504810279 wait complete. 2024-03-08T06:01:27.557 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:01:27.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.659+0000 7f8edc181640 1 Processor -- start 2024-03-08T06:01:27.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.659+0000 7f8edc181640 1 -- start start 2024-03-08T06:01:27.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.660+0000 7f8edc181640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8ed415dfb0 0x7f8ed415e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.660+0000 7f8edc181640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed4161840 con 0x7f8ed40ce5e0 2024-03-08T06:01:27.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.660+0000 7f8edc181640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed415eb30 con 0x7f8ed406d8a0 2024-03-08T06:01:27.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.660+0000 7f8edc181640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed4161b90 con 0x7f8ed415dfb0 2024-03-08T06:01:27.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.660+0000 7f8eda71e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8ed415dfb0 0x7f8ed415e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.660+0000 7f8ed9f1d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8ed40ce5e0 0x7f8ed40ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39610/0 (socket says 172.21.15.76:39610) 2024-03-08T06:01:27.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.660+0000 7f8ed9f1d640 1 -- 172.21.15.76:0/3472401796 learned_addr learned my addr 172.21.15.76:0/3472401796 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:27.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.660+0000 7f8eda71e640 1 -- 172.21.15.76:0/3472401796 >> v1:172.21.15.81:6789/0 conn(0x7f8ed406d8a0 legacy=0x7f8ed4162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.660+0000 7f8eda71e640 1 -- 172.21.15.76:0/3472401796 >> v1:172.21.15.76:6789/0 conn(0x7f8ed40ce5e0 legacy=0x7f8ed40ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.660+0000 7f8ed8f1b640 1 -- 172.21.15.76:0/3472401796 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131783348 0 0) 0x7f8ed4161840 con 0x7f8ed40ce5e0 2024-03-08T06:01:27.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.660+0000 7f8eda71e640 1 -- 172.21.15.76:0/3472401796 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed415ee10 con 0x7f8ed415dfb0 2024-03-08T06:01:27.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.661+0000 7f8eda71e640 1 --2- 172.21.15.76:0/3472401796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed415dfb0 0x7f8ed415e440 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009d90 tx=0x7f8ecc02bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ac54c4ab335edb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:27.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.661+0000 7f8ed8f1b640 1 -- 172.21.15.76:0/3472401796 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc039070 con 0x7f8ed415dfb0 2024-03-08T06:01:27.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.661+0000 7f8ed8f1b640 1 -- 172.21.15.76:0/3472401796 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ecc02cd80 con 0x7f8ed415dfb0 2024-03-08T06:01:27.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.661+0000 7f8ed8f1b640 1 -- 172.21.15.76:0/3472401796 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc034660 con 0x7f8ed415dfb0 2024-03-08T06:01:27.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.662+0000 7f8edc181640 1 -- 172.21.15.76:0/3472401796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed415dfb0 msgr2=0x7f8ed415e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.662+0000 7f8edc181640 1 --2- 172.21.15.76:0/3472401796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed415dfb0 0x7f8ed415e440 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009d90 tx=0x7f8ecc02bfd0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.662+0000 7f8edc181640 1 -- 172.21.15.76:0/3472401796 shutdown_connections 2024-03-08T06:01:27.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.662+0000 7f8edc181640 1 --2- 172.21.15.76:0/3472401796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed415dfb0 0x7f8ed415e440 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.662+0000 7f8edc181640 1 -- 172.21.15.76:0/3472401796 >> 172.21.15.76:0/3472401796 conn(0x7f8ed406ae60 msgr2=0x7f8ed406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:27.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.662+0000 7f8edc181640 1 -- 172.21.15.76:0/3472401796 shutdown_connections 2024-03-08T06:01:27.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.662+0000 7f8edc181640 1 -- 172.21.15.76:0/3472401796 wait complete. 2024-03-08T06:01:27.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.663+0000 7f8edc181640 1 Processor -- start 2024-03-08T06:01:27.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.663+0000 7f8edc181640 1 -- start start 2024-03-08T06:01:27.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8edc181640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ed406d8a0 0x7f8ed414deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8edc181640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8ed40ce5e0 0x7f8ed41f7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8edc181640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed415dfb0 0x7f8ed41f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8edc181640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed406bfa0 con 0x7f8ed415dfb0 2024-03-08T06:01:27.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8edc181640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed4162730 con 0x7f8ed406d8a0 2024-03-08T06:01:27.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8edc181640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed406c120 con 0x7f8ed40ce5e0 2024-03-08T06:01:27.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8eda71e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed415dfb0 0x7f8ed41f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8eda71e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed415dfb0 0x7f8ed41f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41984/0 (socket says 172.21.15.76:41984) 2024-03-08T06:01:27.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8eda71e640 1 -- 172.21.15.76:0/3415306626 learned_addr learned my addr 172.21.15.76:0/3415306626 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:27.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8ed9f1d640 1 --2- 172.21.15.76:0/3415306626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ed406d8a0 0x7f8ed414deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8ed971c640 1 --2- 172.21.15.76:0/3415306626 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8ed40ce5e0 0x7f8ed41f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8eda71e640 1 -- 172.21.15.76:0/3415306626 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8ed40ce5e0 msgr2=0x7f8ed41f7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8eda71e640 1 --2- 172.21.15.76:0/3415306626 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8ed40ce5e0 0x7f8ed41f7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8eda71e640 1 -- 172.21.15.76:0/3415306626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ed406d8a0 msgr2=0x7f8ed414deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8eda71e640 1 --2- 172.21.15.76:0/3415306626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ed406d8a0 0x7f8ed414deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8eda71e640 1 -- 172.21.15.76:0/3415306626 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed41f7fd0 con 0x7f8ed415dfb0 2024-03-08T06:01:27.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8ed971c640 1 --2- 172.21.15.76:0/3415306626 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8ed40ce5e0 0x7f8ed41f7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:27.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8eda71e640 1 --2- 172.21.15.76:0/3415306626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed415dfb0 0x7f8ed41f78c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009d90 tx=0x7f8ecc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:27.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8eca7fc640 1 -- 172.21.15.76:0/3415306626 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc039050 con 0x7f8ed415dfb0 2024-03-08T06:01:27.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8eca7fc640 1 -- 172.21.15.76:0/3415306626 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ecc045070 con 0x7f8ed415dfb0 2024-03-08T06:01:27.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8eca7fc640 1 -- 172.21.15.76:0/3415306626 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc03e830 con 0x7f8ed415dfb0 2024-03-08T06:01:27.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8edc181640 1 -- 172.21.15.76:0/3415306626 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed40b4680 con 0x7f8ed415dfb0 2024-03-08T06:01:27.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.664+0000 7f8edc181640 1 -- 172.21.15.76:0/3415306626 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed40ac950 con 0x7f8ed415dfb0 2024-03-08T06:01:27.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.665+0000 7f8edc181640 1 -- 172.21.15.76:0/3415306626 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed404f030 con 0x7f8ed415dfb0 2024-03-08T06:01:27.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.666+0000 7f8eca7fc640 1 -- 172.21.15.76:0/3415306626 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ecc03e9d0 con 0x7f8ed415dfb0 2024-03-08T06:01:27.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.666+0000 7f8eca7fc640 1 --2- 172.21.15.76:0/3415306626 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8eb806b500 0x7f8eb806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.666+0000 7f8ed9f1d640 1 --2- 172.21.15.76:0/3415306626 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8eb806b500 0x7f8eb806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.666+0000 7f8eca7fc640 1 -- 172.21.15.76:0/3415306626 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8ecc0ad270 con 0x7f8ed415dfb0 2024-03-08T06:01:27.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.667+0000 7f8ed9f1d640 1 --2- 172.21.15.76:0/3415306626 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8eb806b500 0x7f8eb806d9c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0007f70 tx=0x7f8ed0007f00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:27.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.671+0000 7f8eca7fc640 1 -- 172.21.15.76:0/3415306626 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ecc07bea0 con 0x7f8ed415dfb0 2024-03-08T06:01:27.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.839+0000 7f8edc181640 1 -- 172.21.15.76:0/3415306626 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8ed40b4330 con 0x7f8ed415dfb0 2024-03-08T06:01:27.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.840+0000 7f8eca7fc640 1 -- 172.21.15.76:0/3415306626 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8ecc07b7a0 con 0x7f8ed415dfb0 2024-03-08T06:01:27.840 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:27.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.841+0000 7f8edc181640 1 -- 172.21.15.76:0/3415306626 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8eb806b500 msgr2=0x7f8eb806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.841+0000 7f8edc181640 1 --2- 172.21.15.76:0/3415306626 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8eb806b500 0x7f8eb806d9c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0007f70 tx=0x7f8ed0007f00 comp rx=0 tx=0).stop 2024-03-08T06:01:27.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.841+0000 7f8edc181640 1 -- 172.21.15.76:0/3415306626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed415dfb0 msgr2=0x7f8ed41f78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.841+0000 7f8edc181640 1 --2- 172.21.15.76:0/3415306626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed415dfb0 0x7f8ed41f78c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009d90 tx=0x7f8ecc004a40 comp rx=0 tx=0).stop 2024-03-08T06:01:27.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.841+0000 7f8edc181640 1 -- 172.21.15.76:0/3415306626 shutdown_connections 2024-03-08T06:01:27.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.841+0000 7f8edc181640 1 --2- 172.21.15.76:0/3415306626 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8eb806b500 0x7f8eb806d9c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.841+0000 7f8edc181640 1 --2- 172.21.15.76:0/3415306626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed415dfb0 0x7f8ed41f78c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.841+0000 7f8edc181640 1 --2- 172.21.15.76:0/3415306626 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8ed40ce5e0 0x7f8ed41f7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.841+0000 7f8edc181640 1 --2- 172.21.15.76:0/3415306626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ed406d8a0 0x7f8ed414deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.841+0000 7f8edc181640 1 -- 172.21.15.76:0/3415306626 >> 172.21.15.76:0/3415306626 conn(0x7f8ed406ae60 msgr2=0x7f8ed406f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:27.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.841+0000 7f8edc181640 1 -- 172.21.15.76:0/3415306626 shutdown_connections 2024-03-08T06:01:27.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.842+0000 7f8edc181640 1 -- 172.21.15.76:0/3415306626 wait complete. 2024-03-08T06:01:27.853 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:01:27.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.973+0000 7ff0e5ed4640 1 Processor -- start 2024-03-08T06:01:27.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.974+0000 7ff0e5ed4640 1 -- start start 2024-03-08T06:01:27.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.974+0000 7ff0e5ed4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff0e015e120 0x7ff0e00cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.974+0000 7ff0e5ed4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e00b1550 con 0x7ff0e00cf010 2024-03-08T06:01:27.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.974+0000 7ff0e5ed4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e00ae1d0 con 0x7ff0e015e500 2024-03-08T06:01:27.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.974+0000 7ff0e5ed4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e00ae4b0 con 0x7ff0e015e120 2024-03-08T06:01:27.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.974+0000 7ff0dffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff0e00cf010 0x7ff0e00aecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39622/0 (socket says 172.21.15.76:39622) 2024-03-08T06:01:27.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.975+0000 7ff0dffff640 1 -- 172.21.15.76:0/1212629779 learned_addr learned my addr 172.21.15.76:0/1212629779 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:27.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.975+0000 7ff0df7fe640 1 --2- 172.21.15.76:0/1212629779 >> v2:172.21.15.81:3300/0 conn(0x7ff0e015e120 0x7ff0e00cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.975+0000 7ff0deffd640 1 -- 172.21.15.76:0/1212629779 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134079477 0 0) 0x7ff0e00b1550 con 0x7ff0e00cf010 2024-03-08T06:01:27.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.975+0000 7ff0deffd640 1 -- 172.21.15.76:0/1212629779 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0cc003520 con 0x7ff0e00cf010 2024-03-08T06:01:27.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.975+0000 7ff0deffd640 1 -- 172.21.15.76:0/1212629779 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1469615047 0 0) 0x7ff0e00ae1d0 con 0x7ff0e015e500 2024-03-08T06:01:27.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.975+0000 7ff0deffd640 1 -- 172.21.15.76:0/1212629779 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0e00b1550 con 0x7ff0e015e500 2024-03-08T06:01:27.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.975+0000 7ff0deffd640 1 -- 172.21.15.76:0/1212629779 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 583990934 0 0) 0x7ff0cc003520 con 0x7ff0e00cf010 2024-03-08T06:01:27.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.975+0000 7ff0df7fe640 1 -- 172.21.15.76:0/1212629779 >> v1:172.21.15.81:6789/0 conn(0x7ff0e015e500 legacy=0x7ff0e00ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.975+0000 7ff0df7fe640 1 -- 172.21.15.76:0/1212629779 >> v1:172.21.15.76:6789/0 conn(0x7ff0e00cf010 legacy=0x7ff0e00aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.975+0000 7ff0df7fe640 1 -- 172.21.15.76:0/1212629779 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e00ae790 con 0x7ff0e015e120 2024-03-08T06:01:27.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.976+0000 7ff0df7fe640 1 --2- 172.21.15.76:0/1212629779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0e015e120 0x7ff0e00cb270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00097a0 tx=0x7ff0d002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54f6a6104647d005 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:27.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.976+0000 7ff0deffd640 1 -- 172.21.15.76:0/1212629779 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0038070 con 0x7ff0e015e120 2024-03-08T06:01:27.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.976+0000 7ff0deffd640 1 -- 172.21.15.76:0/1212629779 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d0031500 con 0x7ff0e015e120 2024-03-08T06:01:27.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.976+0000 7ff0deffd640 1 -- 172.21.15.76:0/1212629779 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d80019f0 con 0x7ff0e015e120 2024-03-08T06:01:27.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.976+0000 7ff0e5ed4640 1 -- 172.21.15.76:0/1212629779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0e015e120 msgr2=0x7ff0e00cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.976+0000 7ff0e5ed4640 1 --2- 172.21.15.76:0/1212629779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0e015e120 0x7ff0e00cb270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00097a0 tx=0x7ff0d002b820 comp rx=0 tx=0).stop 2024-03-08T06:01:27.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.977+0000 7ff0e5ed4640 1 -- 172.21.15.76:0/1212629779 shutdown_connections 2024-03-08T06:01:27.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.977+0000 7ff0e5ed4640 1 --2- 172.21.15.76:0/1212629779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0e015e120 0x7ff0e00cb270 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.977+0000 7ff0e5ed4640 1 -- 172.21.15.76:0/1212629779 >> 172.21.15.76:0/1212629779 conn(0x7ff0e00d0890 msgr2=0x7ff0e00d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:27.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.977+0000 7ff0e5ed4640 1 -- 172.21.15.76:0/1212629779 shutdown_connections 2024-03-08T06:01:27.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.977+0000 7ff0e5ed4640 1 -- 172.21.15.76:0/1212629779 wait complete. 2024-03-08T06:01:27.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.978+0000 7ff0e5ed4640 1 Processor -- start 2024-03-08T06:01:27.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.978+0000 7ff0e5ed4640 1 -- start start 2024-03-08T06:01:27.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0e5ed4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0e00cf010 0x7ff0e00c3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0e5ed4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0e015e120 0x7ff0e015cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0e5ed4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0e015e500 0x7ff0e015d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0e5ed4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00d1380 con 0x7ff0e015e500 2024-03-08T06:01:27.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0e5ed4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00cf6f0 con 0x7ff0e00cf010 2024-03-08T06:01:27.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0e5ed4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00d1500 con 0x7ff0e015e120 2024-03-08T06:01:27.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0e015e500 0x7ff0e015d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0e015e500 0x7ff0e015d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41986/0 (socket says 172.21.15.76:41986) 2024-03-08T06:01:27.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0dffff640 1 -- 172.21.15.76:0/2333370534 learned_addr learned my addr 172.21.15.76:0/2333370534 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:27.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0df7fe640 1 --2- 172.21.15.76:0/2333370534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0e00cf010 0x7ff0e00c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0d6ffd640 1 --2- 172.21.15.76:0/2333370534 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0e015e120 0x7ff0e015cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0dffff640 1 -- 172.21.15.76:0/2333370534 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0e015e120 msgr2=0x7ff0e015cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0dffff640 1 --2- 172.21.15.76:0/2333370534 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0e015e120 0x7ff0e015cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0dffff640 1 -- 172.21.15.76:0/2333370534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0e00cf010 msgr2=0x7ff0e00c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:27.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0dffff640 1 --2- 172.21.15.76:0/2333370534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0e00cf010 0x7ff0e00c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:27.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0dffff640 1 -- 172.21.15.76:0/2333370534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e014b9d0 con 0x7ff0e015e500 2024-03-08T06:01:27.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0d6ffd640 1 --2- 172.21.15.76:0/2333370534 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0e015e120 0x7ff0e015cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:27.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0dffff640 1 --2- 172.21.15.76:0/2333370534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0e015e500 0x7ff0e015d0b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8001350 tx=0x7ff0d800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:27.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0dcff9640 1 -- 172.21.15.76:0/2333370534 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8017070 con 0x7ff0e015e500 2024-03-08T06:01:27.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0dcff9640 1 -- 172.21.15.76:0/2333370534 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d8012400 con 0x7ff0e015e500 2024-03-08T06:01:27.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0e5ed4640 1 -- 172.21.15.76:0/2333370534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e00c7bb0 con 0x7ff0e015e500 2024-03-08T06:01:27.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0dcff9640 1 -- 172.21.15.76:0/2333370534 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d80127b0 con 0x7ff0e015e500 2024-03-08T06:01:27.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.979+0000 7ff0e5ed4640 1 -- 172.21.15.76:0/2333370534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e00c8170 con 0x7ff0e015e500 2024-03-08T06:01:27.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.980+0000 7ff0e5ed4640 1 -- 172.21.15.76:0/2333370534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e00d0170 con 0x7ff0e015e500 2024-03-08T06:01:27.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.981+0000 7ff0dcff9640 1 -- 172.21.15.76:0/2333370534 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0d801a070 con 0x7ff0e015e500 2024-03-08T06:01:27.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.981+0000 7ff0dcff9640 1 --2- 172.21.15.76:0/2333370534 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff0c006b4d0 0x7ff0c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:27.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.981+0000 7ff0dcff9640 1 -- 172.21.15.76:0/2333370534 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff0d808a430 con 0x7ff0e015e500 2024-03-08T06:01:27.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.981+0000 7ff0df7fe640 1 --2- 172.21.15.76:0/2333370534 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff0c006b4d0 0x7ff0c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:27.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.982+0000 7ff0df7fe640 1 --2- 172.21.15.76:0/2333370534 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff0c006b4d0 0x7ff0c006d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0035000 tx=0x7ff0d0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:27.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:27.986+0000 7ff0dcff9640 1 -- 172.21.15.76:0/2333370534 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0e00d0170 con 0x7ff0e015e500 2024-03-08T06:01:28.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.155+0000 7ff0e5ed4640 1 -- 172.21.15.76:0/2333370534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0e00d1140 con 0x7ff0e015e500 2024-03-08T06:01:28.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.156+0000 7ff0dcff9640 1 -- 172.21.15.76:0/2333370534 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff0d8058860 con 0x7ff0e015e500 2024-03-08T06:01:28.156 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:01:28.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.157+0000 7ff0e5ed4640 1 -- 172.21.15.76:0/2333370534 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff0c006b4d0 msgr2=0x7ff0c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:28.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.157+0000 7ff0e5ed4640 1 --2- 172.21.15.76:0/2333370534 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff0c006b4d0 0x7ff0c006d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0035000 tx=0x7ff0d0008030 comp rx=0 tx=0).stop 2024-03-08T06:01:28.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.157+0000 7ff0e5ed4640 1 -- 172.21.15.76:0/2333370534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0e015e500 msgr2=0x7ff0e015d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:28.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.157+0000 7ff0e5ed4640 1 --2- 172.21.15.76:0/2333370534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0e015e500 0x7ff0e015d0b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8001350 tx=0x7ff0d800a430 comp rx=0 tx=0).stop 2024-03-08T06:01:28.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.158+0000 7ff0e5ed4640 1 -- 172.21.15.76:0/2333370534 shutdown_connections 2024-03-08T06:01:28.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.158+0000 7ff0e5ed4640 1 --2- 172.21.15.76:0/2333370534 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff0c006b4d0 0x7ff0c006d990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.158+0000 7ff0e5ed4640 1 --2- 172.21.15.76:0/2333370534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0e015e500 0x7ff0e015d0b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.158+0000 7ff0e5ed4640 1 --2- 172.21.15.76:0/2333370534 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0e015e120 0x7ff0e015cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.158+0000 7ff0e5ed4640 1 --2- 172.21.15.76:0/2333370534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0e00cf010 0x7ff0e00c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.158+0000 7ff0e5ed4640 1 -- 172.21.15.76:0/2333370534 >> 172.21.15.76:0/2333370534 conn(0x7ff0e00d0890 msgr2=0x7ff0e006f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:28.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.158+0000 7ff0e5ed4640 1 -- 172.21.15.76:0/2333370534 shutdown_connections 2024-03-08T06:01:28.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.158+0000 7ff0e5ed4640 1 -- 172.21.15.76:0/2333370534 wait complete. 2024-03-08T06:01:28.169 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T06:01:28.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.273+0000 7f4948d6c640 1 Processor -- start 2024-03-08T06:01:28.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.273+0000 7f4948d6c640 1 -- start start 2024-03-08T06:01:28.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.274+0000 7f4948d6c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f49440b9840 0x7f49440b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:28.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.274+0000 7f4948d6c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f49440b5b20 0x7f49440b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:28.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.274+0000 7f4948d6c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49440b5f40 con 0x7f494415dbc0 2024-03-08T06:01:28.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.274+0000 7f4948d6c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49440d27d0 con 0x7f49440b9840 2024-03-08T06:01:28.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.274+0000 7f4948d6c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49440d2ab0 con 0x7f49440b5b20 2024-03-08T06:01:28.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.274+0000 7f493bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f49440b9840 0x7f49440b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:28.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.274+0000 7f494259c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f494415dbc0 0x7f49440d1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39636/0 (socket says 172.21.15.76:39636) 2024-03-08T06:01:28.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.274+0000 7f494259c640 1 -- 172.21.15.76:0/3201106901 learned_addr learned my addr 172.21.15.76:0/3201106901 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:28.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.274+0000 7f4942d9d640 1 --2- 172.21.15.76:0/3201106901 >> v2:172.21.15.81:3300/0 conn(0x7f49440b5b20 0x7f49440b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:28.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.274+0000 7f4941d9b640 1 -- 172.21.15.76:0/3201106901 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513123641 0 0) 0x7f49440b5f40 con 0x7f494415dbc0 2024-03-08T06:01:28.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.274+0000 7f4941d9b640 1 -- 172.21.15.76:0/3201106901 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4930003520 con 0x7f494415dbc0 2024-03-08T06:01:28.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.274+0000 7f493bfff640 1 -- 172.21.15.76:0/3201106901 >> v2:172.21.15.81:3300/0 conn(0x7f49440b5b20 msgr2=0x7f49440b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:28.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.274+0000 7f493bfff640 1 --2- 172.21.15.76:0/3201106901 >> v2:172.21.15.81:3300/0 conn(0x7f49440b5b20 0x7f49440b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.274+0000 7f493bfff640 1 -- 172.21.15.76:0/3201106901 >> v1:172.21.15.76:6789/0 conn(0x7f494415dbc0 legacy=0x7f49440d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:28.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.274+0000 7f493bfff640 1 -- 172.21.15.76:0/3201106901 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440d2d90 con 0x7f49440b9840 2024-03-08T06:01:28.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.275+0000 7f493bfff640 1 --2- 172.21.15.76:0/3201106901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b9840 0x7f49440b55e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f494406d410 tx=0x7f492c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad4404d7e43f4ba server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:28.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.275+0000 7f4941d9b640 1 -- 172.21.15.76:0/3201106901 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c035020 con 0x7f49440b9840 2024-03-08T06:01:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.275+0000 7f4941d9b640 1 -- 172.21.15.76:0/3201106901 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c031830 con 0x7f49440b9840 2024-03-08T06:01:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.275+0000 7f4941d9b640 1 -- 172.21.15.76:0/3201106901 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c031c00 con 0x7f49440b9840 2024-03-08T06:01:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.276+0000 7f4948d6c640 1 -- 172.21.15.76:0/3201106901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b9840 msgr2=0x7f49440b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.276+0000 7f4948d6c640 1 --2- 172.21.15.76:0/3201106901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b9840 0x7f49440b55e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f494406d410 tx=0x7f492c02b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.276+0000 7f4948d6c640 1 -- 172.21.15.76:0/3201106901 shutdown_connections 2024-03-08T06:01:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.276+0000 7f4948d6c640 1 --2- 172.21.15.76:0/3201106901 >> v2:172.21.15.81:3300/0 conn(0x7f49440b5b20 0x7f49440b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.276+0000 7f4948d6c640 1 --2- 172.21.15.76:0/3201106901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b9840 0x7f49440b55e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.276+0000 7f4948d6c640 1 -- 172.21.15.76:0/3201106901 >> 172.21.15.76:0/3201106901 conn(0x7f49440670d0 msgr2=0x7f49440d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.276+0000 7f4948d6c640 1 -- 172.21.15.76:0/3201106901 shutdown_connections 2024-03-08T06:01:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.276+0000 7f4948d6c640 1 -- 172.21.15.76:0/3201106901 wait complete. 2024-03-08T06:01:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.277+0000 7f4948d6c640 1 Processor -- start 2024-03-08T06:01:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.277+0000 7f4948d6c640 1 -- start start 2024-03-08T06:01:28.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.278+0000 7f4948d6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b5b20 0x7f49440cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:28.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.278+0000 7f4948d6c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49440b9840 0x7f494414d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:28.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.278+0000 7f4948d6c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f494415dbc0 0x7f49441f6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:28.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.278+0000 7f4948d6c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494415dfa0 con 0x7f49440b5b20 2024-03-08T06:01:28.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.278+0000 7f4948d6c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49440d3870 con 0x7f49440b9840 2024-03-08T06:01:28.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.278+0000 7f4948d6c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494415e120 con 0x7f494415dbc0 2024-03-08T06:01:28.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.278+0000 7f494259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b5b20 0x7f49440cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:28.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.278+0000 7f494259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b5b20 0x7f49440cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42010/0 (socket says 172.21.15.76:42010) 2024-03-08T06:01:28.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.278+0000 7f494259c640 1 -- 172.21.15.76:0/4245080184 learned_addr learned my addr 172.21.15.76:0/4245080184 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:28.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.278+0000 7f494259c640 1 -- 172.21.15.76:0/4245080184 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f494415dbc0 msgr2=0x7f49441f6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:28.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.278+0000 7f494259c640 1 --2- 172.21.15.76:0/4245080184 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f494415dbc0 0x7f49441f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.278+0000 7f494259c640 1 -- 172.21.15.76:0/4245080184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49440b9840 msgr2=0x7f494414d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:28.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.278+0000 7f494259c640 1 --2- 172.21.15.76:0/4245080184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49440b9840 0x7f494414d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.278+0000 7f494259c640 1 -- 172.21.15.76:0/4245080184 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49441f6aa0 con 0x7f49440b5b20 2024-03-08T06:01:28.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.278+0000 7f494259c640 1 --2- 172.21.15.76:0/4245080184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b5b20 0x7f49440cac30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f493400c3c0 tx=0x7f493400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:28.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.279+0000 7f493affd640 1 -- 172.21.15.76:0/4245080184 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934017070 con 0x7f49440b5b20 2024-03-08T06:01:28.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.279+0000 7f493affd640 1 -- 172.21.15.76:0/4245080184 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4934005ce0 con 0x7f49440b5b20 2024-03-08T06:01:28.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.279+0000 7f4948d6c640 1 -- 172.21.15.76:0/4245080184 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49440cb2f0 con 0x7f49440b5b20 2024-03-08T06:01:28.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.279+0000 7f493affd640 1 -- 172.21.15.76:0/4245080184 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49340072d0 con 0x7f49440b5b20 2024-03-08T06:01:28.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.279+0000 7f4948d6c640 1 -- 172.21.15.76:0/4245080184 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49440cb8b0 con 0x7f49440b5b20 2024-03-08T06:01:28.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.280+0000 7f4948d6c640 1 -- 172.21.15.76:0/4245080184 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494404f030 con 0x7f49440b5b20 2024-03-08T06:01:28.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.280+0000 7f493affd640 1 -- 172.21.15.76:0/4245080184 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f493401a070 con 0x7f49440b5b20 2024-03-08T06:01:28.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.281+0000 7f493affd640 1 --2- 172.21.15.76:0/4245080184 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f492406b500 0x7f492406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:28.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.281+0000 7f493bfff640 1 --2- 172.21.15.76:0/4245080184 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f492406b500 0x7f492406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:28.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.281+0000 7f493affd640 1 -- 172.21.15.76:0/4245080184 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f4934089740 con 0x7f49440b5b20 2024-03-08T06:01:28.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.281+0000 7f493bfff640 1 --2- 172.21.15.76:0/4245080184 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f492406b500 0x7f492406d9c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f492c036000 tx=0x7f492c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:28.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.285+0000 7f493affd640 1 -- 172.21.15.76:0/4245080184 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4934057c00 con 0x7f49440b5b20 2024-03-08T06:01:28.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.456+0000 7f4948d6c640 1 -- 172.21.15.76:0/4245080184 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f49440b7800 con 0x7f49440b5b20 2024-03-08T06:01:28.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.456+0000 7f493affd640 1 -- 172.21.15.76:0/4245080184 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f493405bca0 con 0x7f49440b5b20 2024-03-08T06:01:28.457 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:28.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.458+0000 7f4948d6c640 1 -- 172.21.15.76:0/4245080184 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f492406b500 msgr2=0x7f492406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:28.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.458+0000 7f4948d6c640 1 --2- 172.21.15.76:0/4245080184 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f492406b500 0x7f492406d9c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f492c036000 tx=0x7f492c030000 comp rx=0 tx=0).stop 2024-03-08T06:01:28.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.458+0000 7f4948d6c640 1 -- 172.21.15.76:0/4245080184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b5b20 msgr2=0x7f49440cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:28.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.458+0000 7f4948d6c640 1 --2- 172.21.15.76:0/4245080184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b5b20 0x7f49440cac30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f493400c3c0 tx=0x7f493400c880 comp rx=0 tx=0).stop 2024-03-08T06:01:28.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.458+0000 7f4948d6c640 1 -- 172.21.15.76:0/4245080184 shutdown_connections 2024-03-08T06:01:28.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.458+0000 7f4948d6c640 1 --2- 172.21.15.76:0/4245080184 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f492406b500 0x7f492406d9c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.458+0000 7f4948d6c640 1 --2- 172.21.15.76:0/4245080184 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f494415dbc0 0x7f49441f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.458+0000 7f4948d6c640 1 --2- 172.21.15.76:0/4245080184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49440b9840 0x7f494414d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.458+0000 7f4948d6c640 1 --2- 172.21.15.76:0/4245080184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b5b20 0x7f49440cac30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.458+0000 7f4948d6c640 1 -- 172.21.15.76:0/4245080184 >> 172.21.15.76:0/4245080184 conn(0x7f49440670d0 msgr2=0x7f49440b6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:28.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.458+0000 7f4948d6c640 1 -- 172.21.15.76:0/4245080184 shutdown_connections 2024-03-08T06:01:28.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.458+0000 7f4948d6c640 1 -- 172.21.15.76:0/4245080184 wait complete. 2024-03-08T06:01:28.470 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T06:01:28.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.571+0000 7f0ab99e9640 1 Processor -- start 2024-03-08T06:01:28.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.571+0000 7f0ab99e9640 1 -- start start 2024-03-08T06:01:28.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.572+0000 7f0ab99e9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0ab406ac80 0x7f0ab406b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:28.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.572+0000 7f0ab99e9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0ab40ace90 0x7f0ab40ad320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:28.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.572+0000 7f0ab99e9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab406b720 con 0x7f0ab40b3c50 2024-03-08T06:01:28.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.572+0000 7f0ab99e9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab406ba00 con 0x7f0ab406ac80 2024-03-08T06:01:28.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.572+0000 7f0ab99e9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab406bce0 con 0x7f0ab40ace90 2024-03-08T06:01:28.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.572+0000 7f0ab27fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0ab406ac80 0x7f0ab406b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:28.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.572+0000 7f0ab2ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0ab40b3c50 0x7f0ab40b4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46706/0 (socket says 172.21.15.76:46706) 2024-03-08T06:01:28.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.572+0000 7f0ab27fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0ab406ac80 0x7f0ab406b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53154/0 (socket says 172.21.15.76:53154) 2024-03-08T06:01:28.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.572+0000 7f0ab2ffd640 1 -- 172.21.15.76:0/872198974 learned_addr learned my addr 172.21.15.76:0/872198974 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:28.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.572+0000 7f0ab37fe640 1 --2- 172.21.15.76:0/872198974 >> v2:172.21.15.81:3300/0 conn(0x7f0ab40ace90 0x7f0ab40ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:28.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.572+0000 7f0ab27fc640 1 -- 172.21.15.76:0/872198974 >> v2:172.21.15.81:3300/0 conn(0x7f0ab40ace90 msgr2=0x7f0ab40ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:28.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.573+0000 7f0ab27fc640 1 --2- 172.21.15.76:0/872198974 >> v2:172.21.15.81:3300/0 conn(0x7f0ab40ace90 0x7f0ab40ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.573+0000 7f0ab27fc640 1 -- 172.21.15.76:0/872198974 >> v1:172.21.15.76:6789/0 conn(0x7f0ab40b3c50 legacy=0x7f0ab40b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:28.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.573+0000 7f0ab1ffb640 1 -- 172.21.15.76:0/872198974 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520003382 0 0) 0x7f0ab406b720 con 0x7f0ab40b3c50 2024-03-08T06:01:28.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.573+0000 7f0ab27fc640 1 -- 172.21.15.76:0/872198974 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab406bfc0 con 0x7f0ab406ac80 2024-03-08T06:01:28.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.573+0000 7f0ab37fe640 1 --2- 172.21.15.76:0/872198974 >> v2:172.21.15.81:3300/0 conn(0x7f0ab40ace90 0x7f0ab40ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:28.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.573+0000 7f0ab27fc640 1 --2- 172.21.15.76:0/872198974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ab406ac80 0x7f0ab406b0f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80097a0 tx=0x7f0aa802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b6a0fb252ba490 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:28.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.573+0000 7f0ab1ffb640 1 -- 172.21.15.76:0/872198974 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8035020 con 0x7f0ab406ac80 2024-03-08T06:01:28.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.573+0000 7f0ab1ffb640 1 -- 172.21.15.76:0/872198974 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa8031830 con 0x7f0ab406ac80 2024-03-08T06:01:28.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.574+0000 7f0ab1ffb640 1 -- 172.21.15.76:0/872198974 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8031c00 con 0x7f0ab406ac80 2024-03-08T06:01:28.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.574+0000 7f0ab99e9640 1 -- 172.21.15.76:0/872198974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ab406ac80 msgr2=0x7f0ab406b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:28.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.574+0000 7f0ab99e9640 1 --2- 172.21.15.76:0/872198974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ab406ac80 0x7f0ab406b0f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80097a0 tx=0x7f0aa802b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.574+0000 7f0ab99e9640 1 -- 172.21.15.76:0/872198974 shutdown_connections 2024-03-08T06:01:28.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.574+0000 7f0ab99e9640 1 --2- 172.21.15.76:0/872198974 >> v2:172.21.15.81:3300/0 conn(0x7f0ab40ace90 0x7f0ab40ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.574+0000 7f0ab99e9640 1 --2- 172.21.15.76:0/872198974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ab406ac80 0x7f0ab406b0f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.574+0000 7f0ab99e9640 1 -- 172.21.15.76:0/872198974 >> 172.21.15.76:0/872198974 conn(0x7f0ab406dd40 msgr2=0x7f0ab406e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:28.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.574+0000 7f0ab99e9640 1 -- 172.21.15.76:0/872198974 shutdown_connections 2024-03-08T06:01:28.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.574+0000 7f0ab99e9640 1 -- 172.21.15.76:0/872198974 wait complete. 2024-03-08T06:01:28.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.575+0000 7f0ab99e9640 1 Processor -- start 2024-03-08T06:01:28.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.575+0000 7f0ab99e9640 1 -- start start 2024-03-08T06:01:28.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.576+0000 7f0ab99e9640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0ab40ace90 0x7f0ab41f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:28.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.576+0000 7f0ab99e9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ab40b3c50 0x7f0ab414f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:28.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.576+0000 7f0ab99e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ab4150080 0x7f0ab41f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:28.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.576+0000 7f0ab99e9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab41612c0 con 0x7f0ab40b3c50 2024-03-08T06:01:28.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.576+0000 7f0ab99e9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab4070500 con 0x7f0ab4150080 2024-03-08T06:01:28.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.576+0000 7f0ab99e9640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab4161440 con 0x7f0ab40ace90 2024-03-08T06:01:28.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.576+0000 7f0ab27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ab40b3c50 0x7f0ab414f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:28.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.576+0000 7f0ab27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ab40b3c50 0x7f0ab414f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53164/0 (socket says 172.21.15.76:53164) 2024-03-08T06:01:28.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.576+0000 7f0ab27fc640 1 -- 172.21.15.76:0/4224099295 learned_addr learned my addr 172.21.15.76:0/4224099295 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:28.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.576+0000 7f0ab37fe640 1 --2- 172.21.15.76:0/4224099295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ab4150080 0x7f0ab41f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:28.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.576+0000 7f0ab27fc640 1 -- 172.21.15.76:0/4224099295 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0ab40ace90 msgr2=0x7f0ab41f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:28.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.576+0000 7f0ab2ffd640 1 --2- 172.21.15.76:0/4224099295 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0ab40ace90 0x7f0ab41f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:28.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.577+0000 7f0ab27fc640 1 --2- 172.21.15.76:0/4224099295 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0ab40ace90 0x7f0ab41f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.577+0000 7f0ab27fc640 1 -- 172.21.15.76:0/4224099295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ab4150080 msgr2=0x7f0ab41f3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:28.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.577+0000 7f0ab27fc640 1 --2- 172.21.15.76:0/4224099295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ab4150080 0x7f0ab41f3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.577+0000 7f0ab27fc640 1 -- 172.21.15.76:0/4224099295 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab41f45a0 con 0x7f0ab40b3c50 2024-03-08T06:01:28.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.577+0000 7f0ab27fc640 1 --2- 172.21.15.76:0/4224099295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ab40b3c50 0x7f0ab414f6f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80098c0 tx=0x7f0aa80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:28.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.577+0000 7f0a9bfff640 1 -- 172.21.15.76:0/4224099295 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8041070 con 0x7f0ab40b3c50 2024-03-08T06:01:28.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.577+0000 7f0a9bfff640 1 -- 172.21.15.76:0/4224099295 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa8004230 con 0x7f0ab40b3c50 2024-03-08T06:01:28.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.577+0000 7f0ab99e9640 1 -- 172.21.15.76:0/4224099295 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab41f4880 con 0x7f0ab40b3c50 2024-03-08T06:01:28.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.577+0000 7f0a9bfff640 1 -- 172.21.15.76:0/4224099295 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8035040 con 0x7f0ab40b3c50 2024-03-08T06:01:28.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.577+0000 7f0ab99e9640 1 -- 172.21.15.76:0/4224099295 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab414fc30 con 0x7f0ab40b3c50 2024-03-08T06:01:28.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.578+0000 7f0ab99e9640 1 -- 172.21.15.76:0/4224099295 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab4160a20 con 0x7f0ab40b3c50 2024-03-08T06:01:28.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.578+0000 7f0a9bfff640 1 -- 172.21.15.76:0/4224099295 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0aa8038020 con 0x7f0ab40b3c50 2024-03-08T06:01:28.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.579+0000 7f0a9bfff640 1 --2- 172.21.15.76:0/4224099295 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0aa006b4d0 0x7f0aa006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:28.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.579+0000 7f0a9bfff640 1 -- 172.21.15.76:0/4224099295 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0aa80abd70 con 0x7f0ab40b3c50 2024-03-08T06:01:28.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.579+0000 7f0ab2ffd640 1 --2- 172.21.15.76:0/4224099295 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0aa006b4d0 0x7f0aa006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:28.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.579+0000 7f0ab2ffd640 1 --2- 172.21.15.76:0/4224099295 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0aa006b4d0 0x7f0aa006d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0aa40012e0 tx=0x7f0aa4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:28.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.583+0000 7f0a9bfff640 1 -- 172.21.15.76:0/4224099295 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0aa807a880 con 0x7f0ab40b3c50 2024-03-08T06:01:28.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.767+0000 7f0ab99e9640 1 -- 172.21.15.76:0/4224099295 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0ab40afb40 con 0x7f0ab40b3c50 2024-03-08T06:01:28.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.767+0000 7f0a9bfff640 1 -- 172.21.15.76:0/4224099295 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0aa807a180 con 0x7f0ab40b3c50 2024-03-08T06:01:28.768 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-03-08T06:01:28.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.769+0000 7f0ab99e9640 1 -- 172.21.15.76:0/4224099295 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0aa006b4d0 msgr2=0x7f0aa006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:28.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.769+0000 7f0ab99e9640 1 --2- 172.21.15.76:0/4224099295 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0aa006b4d0 0x7f0aa006d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0aa40012e0 tx=0x7f0aa4009040 comp rx=0 tx=0).stop 2024-03-08T06:01:28.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.769+0000 7f0ab99e9640 1 -- 172.21.15.76:0/4224099295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ab40b3c50 msgr2=0x7f0ab414f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:28.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.769+0000 7f0ab99e9640 1 --2- 172.21.15.76:0/4224099295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ab40b3c50 0x7f0ab414f6f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80098c0 tx=0x7f0aa80046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.769+0000 7f0ab99e9640 1 -- 172.21.15.76:0/4224099295 shutdown_connections 2024-03-08T06:01:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.769+0000 7f0ab99e9640 1 --2- 172.21.15.76:0/4224099295 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0aa006b4d0 0x7f0aa006d990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.769+0000 7f0ab99e9640 1 --2- 172.21.15.76:0/4224099295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ab4150080 0x7f0ab41f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.769+0000 7f0ab99e9640 1 --2- 172.21.15.76:0/4224099295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ab40b3c50 0x7f0ab414f6f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.769+0000 7f0ab99e9640 1 --2- 172.21.15.76:0/4224099295 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0ab40ace90 0x7f0ab41f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.769+0000 7f0ab99e9640 1 -- 172.21.15.76:0/4224099295 >> 172.21.15.76:0/4224099295 conn(0x7f0ab406dd40 msgr2=0x7f0ab40afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.770+0000 7f0ab99e9640 1 -- 172.21.15.76:0/4224099295 shutdown_connections 2024-03-08T06:01:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:28.770+0000 7f0ab99e9640 1 -- 172.21.15.76:0/4224099295 wait complete. 2024-03-08T06:01:28.782 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-08T06:01:28.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:28.782 DEBUG:teuthology.orchestra.run.smithi081:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-08T06:01:28.815 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-03-08T06:01:28.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:28.816 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-08T06:01:28.951 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:01:28.951 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:28.967 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:01:28.967 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:29.022 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:01:29.022 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:29.077 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:29.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.216+0000 7fe1d08fb640 1 Processor -- start 2024-03-08T06:01:29.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.216+0000 7fe1d08fb640 1 -- start start 2024-03-08T06:01:29.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.217+0000 7fe1d08fb640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1cc0cfa00 0x7fe1cc0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.217+0000 7fe1d08fb640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe1cc06eee0 0x7fe1cc0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.217+0000 7fe1d08fb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1cc0d0150 con 0x7fe1cc06ac80 2024-03-08T06:01:29.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.217+0000 7fe1d08fb640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1cc0aff00 con 0x7fe1cc06eee0 2024-03-08T06:01:29.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.217+0000 7fe1d08fb640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0d0430 con 0x7fe1cc0cfa00 2024-03-08T06:01:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1ca59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe1cc06ac80 0x7fe1cc06b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46710/0 (socket says 172.21.15.76:46710) 2024-03-08T06:01:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1ca59c640 1 -- 172.21.15.76:0/1441586400 learned_addr learned my addr 172.21.15.76:0/1441586400 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1cad9d640 1 --2- 172.21.15.76:0/1441586400 >> v2:172.21.15.81:3300/0 conn(0x7fe1cc06eee0 0x7fe1cc0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c9d9b640 1 --2- 172.21.15.76:0/1441586400 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1cc0cfa00 0x7fe1cc0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c959a640 1 -- 172.21.15.76:0/1441586400 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 674447544 0 0) 0x7fe1cc0d0150 con 0x7fe1cc06ac80 2024-03-08T06:01:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c959a640 1 -- 172.21.15.76:0/1441586400 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1b4003520 con 0x7fe1cc06ac80 2024-03-08T06:01:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c959a640 1 -- 172.21.15.76:0/1441586400 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1186037480 0 0) 0x7fe1b4003520 con 0x7fe1cc06ac80 2024-03-08T06:01:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c959a640 1 -- 172.21.15.76:0/1441586400 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1cc0cfa00 msgr2=0x7fe1cc0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c959a640 1 --2- 172.21.15.76:0/1441586400 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1cc0cfa00 0x7fe1cc0af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c959a640 1 -- 172.21.15.76:0/1441586400 >> v2:172.21.15.81:3300/0 conn(0x7fe1cc06eee0 msgr2=0x7fe1cc0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c959a640 1 --2- 172.21.15.76:0/1441586400 >> v2:172.21.15.81:3300/0 conn(0x7fe1cc06eee0 0x7fe1cc0caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c959a640 1 -- 172.21.15.76:0/1441586400 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc06e6e0 con 0x7fe1cc06ac80 2024-03-08T06:01:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c959a640 1 -- 172.21.15.76:0/1441586400 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fe1c0001670 con 0x7fe1cc06ac80 2024-03-08T06:01:29.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c959a640 1 -- 172.21.15.76:0/1441586400 >> v1:172.21.15.76:6789/0 conn(0x7fe1cc06ac80 legacy=0x7fe1cc06b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:29.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c959a640 1 --2- 172.21.15.76:0/1441586400 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1cc0cfa00 0x7fe1b4003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c959a640 1 --2- 172.21.15.76:0/1441586400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1b4003fa0 0x7fe1b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c959a640 1 --2- 172.21.15.76:0/1441586400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b4004d40 0x7fe1b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c959a640 1 -- 172.21.15.76:0/1441586400 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0aff00 con 0x7fe1b4003fa0 2024-03-08T06:01:29.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c959a640 1 -- 172.21.15.76:0/1441586400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0d0430 con 0x7fe1b4004d40 2024-03-08T06:01:29.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c959a640 1 -- 172.21.15.76:0/1441586400 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b400b9f0 con 0x7fe1cc0cfa00 2024-03-08T06:01:29.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1c9d9b640 1 --2- 172.21.15.76:0/1441586400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1b4003fa0 0x7fe1b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1ca59c640 1 --2- 172.21.15.76:0/1441586400 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1cc0cfa00 0x7fe1b4003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.218+0000 7fe1cad9d640 1 --2- 172.21.15.76:0/1441586400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b4004d40 0x7fe1b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.219+0000 7fe1c9d9b640 1 -- 172.21.15.76:0/1441586400 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1cc0cfa00 msgr2=0x7fe1b4003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:29.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.219+0000 7fe1c9d9b640 1 --2- 172.21.15.76:0/1441586400 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1cc0cfa00 0x7fe1b4003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.219+0000 7fe1c9d9b640 1 -- 172.21.15.76:0/1441586400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b4004d40 msgr2=0x7fe1b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:29.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.219+0000 7fe1c9d9b640 1 --2- 172.21.15.76:0/1441586400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b4004d40 0x7fe1b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.219+0000 7fe1c9d9b640 1 -- 172.21.15.76:0/1441586400 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b4003520 con 0x7fe1b4003fa0 2024-03-08T06:01:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.219+0000 7fe1c9d9b640 1 --2- 172.21.15.76:0/1441586400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1b4003fa0 0x7fe1b4004800 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009ac0 tx=0x7fe1b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fe2c8c0e46e6ca server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.219+0000 7fe1c959a640 1 -- 172.21.15.76:0/1441586400 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc039070 con 0x7fe1b4003fa0 2024-03-08T06:01:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.219+0000 7fe1c959a640 1 -- 172.21.15.76:0/1441586400 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc02c6b0 con 0x7fe1b4003fa0 2024-03-08T06:01:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.219+0000 7fe1c959a640 1 -- 172.21.15.76:0/1441586400 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc009690 con 0x7fe1b4003fa0 2024-03-08T06:01:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.220+0000 7fe1d08fb640 1 -- 172.21.15.76:0/1441586400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1b4003fa0 msgr2=0x7fe1b4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.220+0000 7fe1d08fb640 1 --2- 172.21.15.76:0/1441586400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1b4003fa0 0x7fe1b4004800 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009ac0 tx=0x7fe1b4005830 comp rx=0 tx=0).stop 2024-03-08T06:01:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.220+0000 7fe1cad9d640 1 -- 172.21.15.76:0/1441586400 reap_dead start 2024-03-08T06:01:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.220+0000 7fe1d08fb640 1 -- 172.21.15.76:0/1441586400 shutdown_connections 2024-03-08T06:01:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.220+0000 7fe1d08fb640 1 -- 172.21.15.76:0/1441586400 >> 172.21.15.76:0/1441586400 conn(0x7fe1cc099a60 msgr2=0x7fe1cc0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.220+0000 7fe1d08fb640 1 -- 172.21.15.76:0/1441586400 shutdown_connections 2024-03-08T06:01:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.220+0000 7fe1d08fb640 1 -- 172.21.15.76:0/1441586400 wait complete. 2024-03-08T06:01:29.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.221+0000 7fe1d08fb640 1 Processor -- start 2024-03-08T06:01:29.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.221+0000 7fe1d08fb640 1 -- start start 2024-03-08T06:01:29.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.222+0000 7fe1d08fb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b4003fa0 0x7fe1cc153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.222+0000 7fe1d08fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1cc160d20 0x7fe1cc151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.222+0000 7fe1d08fb640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1cc0c6480 0x7fe1cc0c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.222+0000 7fe1d08fb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc15e280 con 0x7fe1cc160d20 2024-03-08T06:01:29.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.222+0000 7fe1d08fb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0cfe60 con 0x7fe1b4003fa0 2024-03-08T06:01:29.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.222+0000 7fe1d08fb640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc06f370 con 0x7fe1cc0c6480 2024-03-08T06:01:29.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.222+0000 7fe1c9d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1cc160d20 0x7fe1cc151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.222+0000 7fe1c9d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1cc160d20 0x7fe1cc151d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53190/0 (socket says 172.21.15.76:53190) 2024-03-08T06:01:29.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.222+0000 7fe1c9d9b640 1 -- 172.21.15.76:0/3196256105 learned_addr learned my addr 172.21.15.76:0/3196256105 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:29.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.222+0000 7fe1ca59c640 1 --2- 172.21.15.76:0/3196256105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b4003fa0 0x7fe1cc153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.222+0000 7fe1cad9d640 1 --2- 172.21.15.76:0/3196256105 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1cc0c6480 0x7fe1cc0c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.222+0000 7fe1c9d9b640 1 -- 172.21.15.76:0/3196256105 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1cc0c6480 msgr2=0x7fe1cc0c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:29.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.223+0000 7fe1c9d9b640 1 --2- 172.21.15.76:0/3196256105 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1cc0c6480 0x7fe1cc0c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.223+0000 7fe1c9d9b640 1 -- 172.21.15.76:0/3196256105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b4003fa0 msgr2=0x7fe1cc153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:29.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.223+0000 7fe1c9d9b640 1 --2- 172.21.15.76:0/3196256105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b4003fa0 0x7fe1cc153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.223+0000 7fe1c9d9b640 1 -- 172.21.15.76:0/3196256105 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc0b3d10 con 0x7fe1cc160d20 2024-03-08T06:01:29.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.223+0000 7fe1cad9d640 1 --2- 172.21.15.76:0/3196256105 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1cc0c6480 0x7fe1cc0c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:29.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.223+0000 7fe1c9d9b640 1 --2- 172.21.15.76:0/3196256105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1cc160d20 0x7fe1cc151d40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc037040 tx=0x7fe1bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:29.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.223+0000 7fe1baffd640 1 -- 172.21.15.76:0/3196256105 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc042020 con 0x7fe1cc160d20 2024-03-08T06:01:29.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.223+0000 7fe1baffd640 1 -- 172.21.15.76:0/3196256105 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc03a040 con 0x7fe1cc160d20 2024-03-08T06:01:29.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.223+0000 7fe1d08fb640 1 -- 172.21.15.76:0/3196256105 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc152670 con 0x7fe1cc160d20 2024-03-08T06:01:29.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.223+0000 7fe1baffd640 1 -- 172.21.15.76:0/3196256105 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc039070 con 0x7fe1cc160d20 2024-03-08T06:01:29.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.223+0000 7fe1d08fb640 1 -- 172.21.15.76:0/3196256105 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc152c30 con 0x7fe1cc160d20 2024-03-08T06:01:29.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.224+0000 7fe1d08fb640 1 -- 172.21.15.76:0/3196256105 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc04f030 con 0x7fe1cc160d20 2024-03-08T06:01:29.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.224+0000 7fe1baffd640 1 -- 172.21.15.76:0/3196256105 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1bc045020 con 0x7fe1cc160d20 2024-03-08T06:01:29.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.225+0000 7fe1baffd640 1 --2- 172.21.15.76:0/3196256105 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe1ac06b500 0x7fe1ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.225+0000 7fe1baffd640 1 -- 172.21.15.76:0/3196256105 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe1bc0ab840 con 0x7fe1cc160d20 2024-03-08T06:01:29.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.225+0000 7fe1ca59c640 1 --2- 172.21.15.76:0/3196256105 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe1ac06b500 0x7fe1ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.225+0000 7fe1ca59c640 1 --2- 172.21.15.76:0/3196256105 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe1ac06b500 0x7fe1ac06d9c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0007fd0 tx=0x7fe1c0007970 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:29.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.229+0000 7fe1baffd640 1 -- 172.21.15.76:0/3196256105 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1bc04c070 con 0x7fe1cc160d20 2024-03-08T06:01:29.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.401+0000 7fe1d08fb640 1 -- 172.21.15.76:0/3196256105 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1cc0cd040 con 0x7fe1cc160d20 2024-03-08T06:01:29.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.402+0000 7fe1baffd640 1 -- 172.21.15.76:0/3196256105 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18757 (secure 0 0 0) 0x7fe1bc07a350 con 0x7fe1cc160d20 2024-03-08T06:01:29.403 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:29.403 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":28,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:01:17.313272+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:29.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.404+0000 7fe1d08fb640 1 -- 172.21.15.76:0/3196256105 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe1ac06b500 msgr2=0x7fe1ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:29.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.404+0000 7fe1d08fb640 1 --2- 172.21.15.76:0/3196256105 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe1ac06b500 0x7fe1ac06d9c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0007fd0 tx=0x7fe1c0007970 comp rx=0 tx=0).stop 2024-03-08T06:01:29.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.404+0000 7fe1d08fb640 1 -- 172.21.15.76:0/3196256105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1cc160d20 msgr2=0x7fe1cc151d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:29.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.404+0000 7fe1d08fb640 1 --2- 172.21.15.76:0/3196256105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1cc160d20 0x7fe1cc151d40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc037040 tx=0x7fe1bc004730 comp rx=0 tx=0).stop 2024-03-08T06:01:29.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.404+0000 7fe1d08fb640 1 -- 172.21.15.76:0/3196256105 shutdown_connections 2024-03-08T06:01:29.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.404+0000 7fe1d08fb640 1 --2- 172.21.15.76:0/3196256105 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe1ac06b500 0x7fe1ac06d9c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.404+0000 7fe1d08fb640 1 --2- 172.21.15.76:0/3196256105 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe1cc0c6480 0x7fe1cc0c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.404+0000 7fe1d08fb640 1 --2- 172.21.15.76:0/3196256105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1cc160d20 0x7fe1cc151d40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.404+0000 7fe1d08fb640 1 --2- 172.21.15.76:0/3196256105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b4003fa0 0x7fe1cc153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.404+0000 7fe1d08fb640 1 -- 172.21.15.76:0/3196256105 >> 172.21.15.76:0/3196256105 conn(0x7fe1cc099a60 msgr2=0x7fe1cc0ae830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:29.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.404+0000 7fe1d08fb640 1 -- 172.21.15.76:0/3196256105 shutdown_connections 2024-03-08T06:01:29.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.404+0000 7fe1d08fb640 1 -- 172.21.15.76:0/3196256105 wait complete. 2024-03-08T06:01:29.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:00:33.926651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:00:34.474030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T06:00:38.207853+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T06:01:29.416 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:01:29.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.519+0000 7feb0d876640 1 Processor -- start 2024-03-08T06:01:29.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.519+0000 7feb0d876640 1 -- start start 2024-03-08T06:01:29.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.520+0000 7feb0d876640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7feb0815dbc0 0x7feb080d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.520+0000 7feb0d876640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feb080b9840 0x7feb080b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.520+0000 7feb0d876640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7feb080b5b20 0x7feb0806c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.520+0000 7feb0d876640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb080b5f90 con 0x7feb080b9840 2024-03-08T06:01:29.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.520+0000 7feb0d876640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb080d27d0 con 0x7feb0815dbc0 2024-03-08T06:01:29.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.520+0000 7feb0d876640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb080d2ab0 con 0x7feb080b5b20 2024-03-08T06:01:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.520+0000 7feb067fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feb080b9840 0x7feb080b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.520+0000 7feb067fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feb080b9840 0x7feb080b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53200/0 (socket says 172.21.15.76:53200) 2024-03-08T06:01:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.520+0000 7feb067fc640 1 -- 172.21.15.76:0/270099144 learned_addr learned my addr 172.21.15.76:0/270099144 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.520+0000 7feb06ffd640 1 --2- 172.21.15.76:0/270099144 >> v2:172.21.15.81:3300/0 conn(0x7feb0815dbc0 0x7feb080d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.520+0000 7feb077fe640 1 --2- 172.21.15.76:0/270099144 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7feb080b5b20 0x7feb0806c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.520+0000 7feb067fc640 1 -- 172.21.15.76:0/270099144 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7feb080b5b20 msgr2=0x7feb0806c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.520+0000 7feb067fc640 1 --2- 172.21.15.76:0/270099144 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7feb080b5b20 0x7feb0806c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.520+0000 7feb067fc640 1 -- 172.21.15.76:0/270099144 >> v2:172.21.15.81:3300/0 conn(0x7feb0815dbc0 msgr2=0x7feb080d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.520+0000 7feb067fc640 1 --2- 172.21.15.76:0/270099144 >> v2:172.21.15.81:3300/0 conn(0x7feb0815dbc0 0x7feb080d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.521+0000 7feb067fc640 1 -- 172.21.15.76:0/270099144 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb080d2d90 con 0x7feb080b9840 2024-03-08T06:01:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.521+0000 7feb067fc640 1 --2- 172.21.15.76:0/270099144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb080b9840 0x7feb080b55e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feb0806d280 tx=0x7feafc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e620d44f52285c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.521+0000 7feb05ffb640 1 -- 172.21.15.76:0/270099144 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc035020 con 0x7feb080b9840 2024-03-08T06:01:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.521+0000 7feb05ffb640 1 -- 172.21.15.76:0/270099144 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc031870 con 0x7feb080b9840 2024-03-08T06:01:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.521+0000 7feb05ffb640 1 -- 172.21.15.76:0/270099144 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc031c40 con 0x7feb080b9840 2024-03-08T06:01:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.522+0000 7feb0d876640 1 -- 172.21.15.76:0/270099144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb080b9840 msgr2=0x7feb080b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.522+0000 7feb0d876640 1 --2- 172.21.15.76:0/270099144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb080b9840 0x7feb080b55e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feb0806d280 tx=0x7feafc02b910 comp rx=0 tx=0).stop 2024-03-08T06:01:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.522+0000 7feb0d876640 1 -- 172.21.15.76:0/270099144 shutdown_connections 2024-03-08T06:01:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.522+0000 7feb0d876640 1 --2- 172.21.15.76:0/270099144 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7feb080b5b20 0x7feb0806c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.522+0000 7feb0d876640 1 --2- 172.21.15.76:0/270099144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb080b9840 0x7feb080b55e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.522+0000 7feb0d876640 1 --2- 172.21.15.76:0/270099144 >> v2:172.21.15.81:3300/0 conn(0x7feb0815dbc0 0x7feb080d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.522+0000 7feb0d876640 1 -- 172.21.15.76:0/270099144 >> 172.21.15.76:0/270099144 conn(0x7feb080670d0 msgr2=0x7feb080d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.522+0000 7feb0d876640 1 -- 172.21.15.76:0/270099144 shutdown_connections 2024-03-08T06:01:29.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.522+0000 7feb0d876640 1 -- 172.21.15.76:0/270099144 wait complete. 2024-03-08T06:01:29.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.523+0000 7feb0d876640 1 Processor -- start 2024-03-08T06:01:29.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.523+0000 7feb0d876640 1 -- start start 2024-03-08T06:01:29.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.524+0000 7feb0d876640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb080b5b20 0x7feb080cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.524+0000 7feb0d876640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7feb080b9840 0x7feb080cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.524+0000 7feb0d876640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb0815dbc0 0x7feb08157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.524+0000 7feb0d876640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb080d2430 con 0x7feb0815dbc0 2024-03-08T06:01:29.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.524+0000 7feb0d876640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb080d3870 con 0x7feb080b5b20 2024-03-08T06:01:29.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.524+0000 7feb0d876640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb080d25b0 con 0x7feb080b9840 2024-03-08T06:01:29.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.524+0000 7feb077fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb0815dbc0 0x7feb08157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.524+0000 7feb077fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb0815dbc0 0x7feb08157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53204/0 (socket says 172.21.15.76:53204) 2024-03-08T06:01:29.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.524+0000 7feb077fe640 1 -- 172.21.15.76:0/2085458647 learned_addr learned my addr 172.21.15.76:0/2085458647 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:29.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.525+0000 7feb067fc640 1 --2- 172.21.15.76:0/2085458647 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7feb080b9840 0x7feb080cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.525+0000 7feb06ffd640 1 --2- 172.21.15.76:0/2085458647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb080b5b20 0x7feb080cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.525+0000 7feb077fe640 1 -- 172.21.15.76:0/2085458647 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7feb080b9840 msgr2=0x7feb080cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:29.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.525+0000 7feb077fe640 1 --2- 172.21.15.76:0/2085458647 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7feb080b9840 0x7feb080cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.525+0000 7feb077fe640 1 -- 172.21.15.76:0/2085458647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb080b5b20 msgr2=0x7feb080cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:29.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.525+0000 7feb077fe640 1 --2- 172.21.15.76:0/2085458647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb080b5b20 0x7feb080cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.525+0000 7feb077fe640 1 -- 172.21.15.76:0/2085458647 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb081f3730 con 0x7feb0815dbc0 2024-03-08T06:01:29.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.525+0000 7feb067fc640 1 --2- 172.21.15.76:0/2085458647 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7feb080b9840 0x7feb080cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:29.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.525+0000 7feb06ffd640 1 --2- 172.21.15.76:0/2085458647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb080b5b20 0x7feb080cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:29.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.525+0000 7feb077fe640 1 --2- 172.21.15.76:0/2085458647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb0815dbc0 0x7feb08157e90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feb0000d800 tx=0x7feb0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:29.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.525+0000 7feaeffff640 1 -- 172.21.15.76:0/2085458647 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00018020 con 0x7feb0815dbc0 2024-03-08T06:01:29.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.525+0000 7feaeffff640 1 -- 172.21.15.76:0/2085458647 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb000096c0 con 0x7feb0815dbc0 2024-03-08T06:01:29.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.525+0000 7feb0d876640 1 -- 172.21.15.76:0/2085458647 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb080cb2c0 con 0x7feb0815dbc0 2024-03-08T06:01:29.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.525+0000 7feaeffff640 1 -- 172.21.15.76:0/2085458647 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00009a90 con 0x7feb0815dbc0 2024-03-08T06:01:29.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.525+0000 7feb0d876640 1 -- 172.21.15.76:0/2085458647 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb080cb880 con 0x7feb0815dbc0 2024-03-08T06:01:29.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.526+0000 7feb0d876640 1 -- 172.21.15.76:0/2085458647 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0804f030 con 0x7feb0815dbc0 2024-03-08T06:01:29.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.526+0000 7feaeffff640 1 -- 172.21.15.76:0/2085458647 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb0001b020 con 0x7feb0815dbc0 2024-03-08T06:01:29.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.527+0000 7feaeffff640 1 --2- 172.21.15.76:0/2085458647 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7feaf406b4d0 0x7feaf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.527+0000 7feaeffff640 1 -- 172.21.15.76:0/2085458647 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7feb0008a830 con 0x7feb0815dbc0 2024-03-08T06:01:29.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.527+0000 7feb06ffd640 1 --2- 172.21.15.76:0/2085458647 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7feaf406b4d0 0x7feaf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.527+0000 7feb06ffd640 1 --2- 172.21.15.76:0/2085458647 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7feaf406b4d0 0x7feaf406d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009f50 tx=0x7feaf800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:29.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.531+0000 7feaeffff640 1 -- 172.21.15.76:0/2085458647 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb00022030 con 0x7feb0815dbc0 2024-03-08T06:01:29.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.698+0000 7feb0d876640 1 -- 172.21.15.76:0/2085458647 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb080cec20 con 0x7feb0815dbc0 2024-03-08T06:01:29.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.699+0000 7feaeffff640 1 -- 172.21.15.76:0/2085458647 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb000592b0 con 0x7feb0815dbc0 2024-03-08T06:01:29.699 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:01:29.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.701+0000 7feb0d876640 1 -- 172.21.15.76:0/2085458647 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7feaf406b4d0 msgr2=0x7feaf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:29.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.701+0000 7feb0d876640 1 --2- 172.21.15.76:0/2085458647 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7feaf406b4d0 0x7feaf406d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009f50 tx=0x7feaf800b040 comp rx=0 tx=0).stop 2024-03-08T06:01:29.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.701+0000 7feb0d876640 1 -- 172.21.15.76:0/2085458647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb0815dbc0 msgr2=0x7feb08157e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:29.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.701+0000 7feb0d876640 1 --2- 172.21.15.76:0/2085458647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb0815dbc0 0x7feb08157e90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feb0000d800 tx=0x7feb0000dcc0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.701+0000 7feb0d876640 1 -- 172.21.15.76:0/2085458647 shutdown_connections 2024-03-08T06:01:29.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.701+0000 7feb0d876640 1 --2- 172.21.15.76:0/2085458647 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7feaf406b4d0 0x7feaf406d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.701+0000 7feb0d876640 1 --2- 172.21.15.76:0/2085458647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb0815dbc0 0x7feb08157e90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.701+0000 7feb0d876640 1 --2- 172.21.15.76:0/2085458647 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7feb080b9840 0x7feb080cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.701+0000 7feb0d876640 1 --2- 172.21.15.76:0/2085458647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb080b5b20 0x7feb080cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.701+0000 7feb0d876640 1 -- 172.21.15.76:0/2085458647 >> 172.21.15.76:0/2085458647 conn(0x7feb080670d0 msgr2=0x7feb080c9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:29.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.701+0000 7feb0d876640 1 -- 172.21.15.76:0/2085458647 shutdown_connections 2024-03-08T06:01:29.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.701+0000 7feb0d876640 1 -- 172.21.15.76:0/2085458647 wait complete. 2024-03-08T06:01:29.712 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:01:29.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.814+0000 7f15a361c640 1 Processor -- start 2024-03-08T06:01:29.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.815+0000 7f15a361c640 1 -- start start 2024-03-08T06:01:29.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.815+0000 7f15a361c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f159c0cfbb0 0x7f159c06b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.815+0000 7f15a361c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f159c0caee0 0x7f159c0cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.815+0000 7f15a361c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f159c0d04d0 con 0x7f159c15e1b0 2024-03-08T06:01:29.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.815+0000 7f15a361c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f159c0cb360 con 0x7f159c0cfbb0 2024-03-08T06:01:29.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.815+0000 7f15a361c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159c15e9d0 con 0x7f159c0caee0 2024-03-08T06:01:29.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.815+0000 7f15a13b8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f159c0cfbb0 0x7f159c06b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.815+0000 7f15a1bb9640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f159c15e1b0 0x7f159c0cd4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46714/0 (socket says 172.21.15.76:46714) 2024-03-08T06:01:29.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.815+0000 7f15a1bb9640 1 -- 172.21.15.76:0/2164350940 learned_addr learned my addr 172.21.15.76:0/2164350940 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:29.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.816+0000 7f15a0bb7640 1 --2- 172.21.15.76:0/2164350940 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f159c0caee0 0x7f159c0cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.816+0000 7f15a13b8640 1 -- 172.21.15.76:0/2164350940 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f159c0caee0 msgr2=0x7f159c0cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:29.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.816+0000 7f15a13b8640 1 --2- 172.21.15.76:0/2164350940 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f159c0caee0 0x7f159c0cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.816+0000 7f15a13b8640 1 -- 172.21.15.76:0/2164350940 >> v1:172.21.15.76:6789/0 conn(0x7f159c15e1b0 legacy=0x7f159c0cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:29.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.816+0000 7f1593fff640 1 -- 172.21.15.76:0/2164350940 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251430192 0 0) 0x7f159c0d04d0 con 0x7f159c15e1b0 2024-03-08T06:01:29.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.816+0000 7f15a13b8640 1 -- 172.21.15.76:0/2164350940 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c15ecb0 con 0x7f159c0cfbb0 2024-03-08T06:01:29.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.816+0000 7f15a0bb7640 1 --2- 172.21.15.76:0/2164350940 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f159c0caee0 0x7f159c0cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:29.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.816+0000 7f15a13b8640 1 --2- 172.21.15.76:0/2164350940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f159c0cfbb0 0x7f159c06b690 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1598009d90 tx=0x7f159802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387f8d9c1a568058 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:29.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.817+0000 7f1593fff640 1 -- 172.21.15.76:0/2164350940 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598039070 con 0x7f159c0cfbb0 2024-03-08T06:01:29.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.817+0000 7f1593fff640 1 -- 172.21.15.76:0/2164350940 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159802cc50 con 0x7f159c0cfbb0 2024-03-08T06:01:29.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.817+0000 7f1593fff640 1 -- 172.21.15.76:0/2164350940 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598034440 con 0x7f159c0cfbb0 2024-03-08T06:01:29.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.817+0000 7f15a361c640 1 -- 172.21.15.76:0/2164350940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f159c0cfbb0 msgr2=0x7f159c06b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:29.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.817+0000 7f15a361c640 1 --2- 172.21.15.76:0/2164350940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f159c0cfbb0 0x7f159c06b690 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1598009d90 tx=0x7f159802bea0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.817+0000 7f15a361c640 1 -- 172.21.15.76:0/2164350940 shutdown_connections 2024-03-08T06:01:29.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.817+0000 7f15a361c640 1 --2- 172.21.15.76:0/2164350940 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f159c0caee0 0x7f159c0cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.817+0000 7f15a361c640 1 --2- 172.21.15.76:0/2164350940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f159c0cfbb0 0x7f159c06b690 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.817+0000 7f15a361c640 1 -- 172.21.15.76:0/2164350940 >> 172.21.15.76:0/2164350940 conn(0x7f159c0d0890 msgr2=0x7f159c0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:29.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.817+0000 7f15a361c640 1 -- 172.21.15.76:0/2164350940 shutdown_connections 2024-03-08T06:01:29.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.817+0000 7f15a361c640 1 -- 172.21.15.76:0/2164350940 wait complete. 2024-03-08T06:01:29.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.818+0000 7f15a361c640 1 Processor -- start 2024-03-08T06:01:29.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.818+0000 7f15a361c640 1 -- start start 2024-03-08T06:01:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.819+0000 7f15a361c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f159c0caee0 0x7f159c14ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.819+0000 7f15a361c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f159c0cfbb0 0x7f159c15b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.819+0000 7f15a361c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f159c15e1b0 0x7f159c0c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.819+0000 7f15a361c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c15e630 con 0x7f159c0cfbb0 2024-03-08T06:01:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.819+0000 7f15a361c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c15f270 con 0x7f159c0caee0 2024-03-08T06:01:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.819+0000 7f15a361c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159c15e7b0 con 0x7f159c15e1b0 2024-03-08T06:01:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.819+0000 7f15a0bb7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f159c0cfbb0 0x7f159c15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.819+0000 7f15a0bb7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f159c0cfbb0 0x7f159c15b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53222/0 (socket says 172.21.15.76:53222) 2024-03-08T06:01:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.819+0000 7f15a0bb7640 1 -- 172.21.15.76:0/502064526 learned_addr learned my addr 172.21.15.76:0/502064526 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.819+0000 7f15a13b8640 1 --2- 172.21.15.76:0/502064526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f159c0caee0 0x7f159c14ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.819+0000 7f15a1bb9640 1 --2- 172.21.15.76:0/502064526 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f159c15e1b0 0x7f159c0c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.819+0000 7f15a0bb7640 1 -- 172.21.15.76:0/502064526 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f159c15e1b0 msgr2=0x7f159c0c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.819+0000 7f15a0bb7640 1 --2- 172.21.15.76:0/502064526 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f159c15e1b0 0x7f159c0c6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.820+0000 7f15a0bb7640 1 -- 172.21.15.76:0/502064526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f159c0caee0 msgr2=0x7f159c14ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.820+0000 7f15a0bb7640 1 --2- 172.21.15.76:0/502064526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f159c0caee0 0x7f159c14ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.820+0000 7f15a0bb7640 1 -- 172.21.15.76:0/502064526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c15c790 con 0x7f159c0cfbb0 2024-03-08T06:01:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.820+0000 7f15a0bb7640 1 --2- 172.21.15.76:0/502064526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f159c0cfbb0 0x7f159c15b610 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c7f0 tx=0x7f158c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.820+0000 7f1591ffb640 1 -- 172.21.15.76:0/502064526 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c004230 con 0x7f159c0cfbb0 2024-03-08T06:01:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.820+0000 7f1591ffb640 1 -- 172.21.15.76:0/502064526 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158c0043d0 con 0x7f159c0cfbb0 2024-03-08T06:01:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.820+0000 7f15a361c640 1 -- 172.21.15.76:0/502064526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159c15bef0 con 0x7f159c0cfbb0 2024-03-08T06:01:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.820+0000 7f1591ffb640 1 -- 172.21.15.76:0/502064526 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c007a70 con 0x7f159c0cfbb0 2024-03-08T06:01:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.820+0000 7f15a361c640 1 -- 172.21.15.76:0/502064526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159c15c4b0 con 0x7f159c0cfbb0 2024-03-08T06:01:29.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.821+0000 7f15a361c640 1 -- 172.21.15.76:0/502064526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159c0b3640 con 0x7f159c0cfbb0 2024-03-08T06:01:29.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.821+0000 7f1591ffb640 1 -- 172.21.15.76:0/502064526 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f158c007c10 con 0x7f159c0cfbb0 2024-03-08T06:01:29.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.822+0000 7f1591ffb640 1 --2- 172.21.15.76:0/502064526 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f158006b4d0 0x7f158006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:29.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.822+0000 7f1591ffb640 1 -- 172.21.15.76:0/502064526 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f158c08a5d0 con 0x7f159c0cfbb0 2024-03-08T06:01:29.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.822+0000 7f15a13b8640 1 --2- 172.21.15.76:0/502064526 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f158006b4d0 0x7f158006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:29.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.822+0000 7f15a13b8640 1 --2- 172.21.15.76:0/502064526 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f158006b4d0 0x7f158006d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1598036000 tx=0x7f1598030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:29.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.826+0000 7f1591ffb640 1 -- 172.21.15.76:0/502064526 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f158c015020 con 0x7f159c0cfbb0 2024-03-08T06:01:29.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.996+0000 7f15a361c640 1 -- 172.21.15.76:0/502064526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f159c04f030 con 0x7f159c0cfbb0 2024-03-08T06:01:29.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.997+0000 7f1591ffb640 1 -- 172.21.15.76:0/502064526 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f158c058a90 con 0x7f159c0cfbb0 2024-03-08T06:01:29.997 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:29.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.999+0000 7f15a361c640 1 -- 172.21.15.76:0/502064526 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f158006b4d0 msgr2=0x7f158006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:29.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.999+0000 7f15a361c640 1 --2- 172.21.15.76:0/502064526 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f158006b4d0 0x7f158006d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1598036000 tx=0x7f1598030000 comp rx=0 tx=0).stop 2024-03-08T06:01:30.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.999+0000 7f15a361c640 1 -- 172.21.15.76:0/502064526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f159c0cfbb0 msgr2=0x7f159c15b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.999+0000 7f15a361c640 1 --2- 172.21.15.76:0/502064526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f159c0cfbb0 0x7f159c15b610 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c7f0 tx=0x7f158c00ccb0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.999+0000 7f15a361c640 1 -- 172.21.15.76:0/502064526 shutdown_connections 2024-03-08T06:01:30.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.999+0000 7f15a361c640 1 --2- 172.21.15.76:0/502064526 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f158006b4d0 0x7f158006d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.999+0000 7f15a361c640 1 --2- 172.21.15.76:0/502064526 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f159c15e1b0 0x7f159c0c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.999+0000 7f15a361c640 1 --2- 172.21.15.76:0/502064526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f159c0cfbb0 0x7f159c15b610 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.999+0000 7f15a361c640 1 --2- 172.21.15.76:0/502064526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f159c0caee0 0x7f159c14ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.999+0000 7f15a361c640 1 -- 172.21.15.76:0/502064526 >> 172.21.15.76:0/502064526 conn(0x7f159c0d0890 msgr2=0x7f159c0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:30.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.999+0000 7f15a361c640 1 -- 172.21.15.76:0/502064526 shutdown_connections 2024-03-08T06:01:30.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:29.999+0000 7f15a361c640 1 -- 172.21.15.76:0/502064526 wait complete. 2024-03-08T06:01:30.010 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:01:30.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.113+0000 7fe9524b5640 1 Processor -- start 2024-03-08T06:01:30.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.113+0000 7fe9524b5640 1 -- start start 2024-03-08T06:01:30.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe9524b5640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe94c145240 0x7fe94c0cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe9524b5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe94c0b40d0 0x7fe94c0b4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe9524b5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe94c15c060 con 0x7fe94c071e10 2024-03-08T06:01:30.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe9524b5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94c0b4a80 con 0x7fe94c0b40d0 2024-03-08T06:01:30.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe9524b5640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c15c390 con 0x7fe94c145240 2024-03-08T06:01:30.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe94b7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe94c071e10 0x7fe94c15c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46718/0 (socket says 172.21.15.76:46718) 2024-03-08T06:01:30.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe94b7fe640 1 -- 172.21.15.76:0/390959931 learned_addr learned my addr 172.21.15.76:0/390959931 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe950a52640 1 --2- 172.21.15.76:0/390959931 >> v2:172.21.15.81:3300/0 conn(0x7fe94c0b40d0 0x7fe94c0b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe94bfff640 1 --2- 172.21.15.76:0/390959931 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe94c145240 0x7fe94c0cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe94affd640 1 -- 172.21.15.76:0/390959931 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716680248 0 0) 0x7fe94c15c060 con 0x7fe94c071e10 2024-03-08T06:01:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe94affd640 1 -- 172.21.15.76:0/390959931 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe934003520 con 0x7fe94c071e10 2024-03-08T06:01:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe94affd640 1 -- 172.21.15.76:0/390959931 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 85192786 0 0) 0x7fe934003520 con 0x7fe94c071e10 2024-03-08T06:01:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe94affd640 1 -- 172.21.15.76:0/390959931 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe94c145240 msgr2=0x7fe94c0cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe94affd640 1 --2- 172.21.15.76:0/390959931 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe94c145240 0x7fe94c0cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe94affd640 1 -- 172.21.15.76:0/390959931 >> v2:172.21.15.81:3300/0 conn(0x7fe94c0b40d0 msgr2=0x7fe94c0b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe94affd640 1 --2- 172.21.15.76:0/390959931 >> v2:172.21.15.81:3300/0 conn(0x7fe94c0b40d0 0x7fe94c0b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe94affd640 1 -- 172.21.15.76:0/390959931 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c145620 con 0x7fe94c071e10 2024-03-08T06:01:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.114+0000 7fe94affd640 1 -- 172.21.15.76:0/390959931 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fe940001690 con 0x7fe94c071e10 2024-03-08T06:01:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.115+0000 7fe94affd640 1 -- 172.21.15.76:0/390959931 >> v1:172.21.15.76:6789/0 conn(0x7fe94c071e10 legacy=0x7fe94c15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.115+0000 7fe94affd640 1 --2- 172.21.15.76:0/390959931 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe94c145240 0x7fe934003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.115+0000 7fe94affd640 1 --2- 172.21.15.76:0/390959931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe934003fa0 0x7fe934004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.115+0000 7fe94affd640 1 --2- 172.21.15.76:0/390959931 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe934004d40 0x7fe9340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.115+0000 7fe94affd640 1 -- 172.21.15.76:0/390959931 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c15c390 con 0x7fe934003fa0 2024-03-08T06:01:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.115+0000 7fe94affd640 1 -- 172.21.15.76:0/390959931 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93400b9c0 con 0x7fe934004d40 2024-03-08T06:01:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.115+0000 7fe94affd640 1 -- 172.21.15.76:0/390959931 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93400bb70 con 0x7fe94c145240 2024-03-08T06:01:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.115+0000 7fe94b7fe640 1 --2- 172.21.15.76:0/390959931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe934003fa0 0x7fe934004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.115+0000 7fe94bfff640 1 --2- 172.21.15.76:0/390959931 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe94c145240 0x7fe934003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.115+0000 7fe950a52640 1 --2- 172.21.15.76:0/390959931 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe934004d40 0x7fe9340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.115+0000 7fe94b7fe640 1 -- 172.21.15.76:0/390959931 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe94c145240 msgr2=0x7fe934003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.115+0000 7fe94b7fe640 1 --2- 172.21.15.76:0/390959931 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe94c145240 0x7fe934003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.115+0000 7fe94b7fe640 1 -- 172.21.15.76:0/390959931 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe934004d40 msgr2=0x7fe9340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.115+0000 7fe94b7fe640 1 --2- 172.21.15.76:0/390959931 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe934004d40 0x7fe9340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.115+0000 7fe94b7fe640 1 -- 172.21.15.76:0/390959931 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe934003520 con 0x7fe934003fa0 2024-03-08T06:01:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.115+0000 7fe950a52640 1 --2- 172.21.15.76:0/390959931 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe934004d40 0x7fe9340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.116+0000 7fe94b7fe640 1 --2- 172.21.15.76:0/390959931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe934003fa0 0x7fe934004800 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe94000c210 tx=0x7fe934005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce417affa0e1b069 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.116+0000 7fe94affd640 1 -- 172.21.15.76:0/390959931 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940007d70 con 0x7fe934003fa0 2024-03-08T06:01:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.116+0000 7fe94affd640 1 -- 172.21.15.76:0/390959931 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe940030070 con 0x7fe934003fa0 2024-03-08T06:01:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.116+0000 7fe94affd640 1 -- 172.21.15.76:0/390959931 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94002cda0 con 0x7fe934003fa0 2024-03-08T06:01:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.116+0000 7fe9524b5640 1 -- 172.21.15.76:0/390959931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe934003fa0 msgr2=0x7fe934004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.116+0000 7fe9524b5640 1 --2- 172.21.15.76:0/390959931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe934003fa0 0x7fe934004800 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe94000c210 tx=0x7fe934005830 comp rx=0 tx=0).stop 2024-03-08T06:01:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.116+0000 7fe950a52640 1 -- 172.21.15.76:0/390959931 reap_dead start 2024-03-08T06:01:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.116+0000 7fe9524b5640 1 -- 172.21.15.76:0/390959931 shutdown_connections 2024-03-08T06:01:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.116+0000 7fe9524b5640 1 -- 172.21.15.76:0/390959931 >> 172.21.15.76:0/390959931 conn(0x7fe94c06ac80 msgr2=0x7fe94c06b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.116+0000 7fe9524b5640 1 -- 172.21.15.76:0/390959931 shutdown_connections 2024-03-08T06:01:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.116+0000 7fe9524b5640 1 -- 172.21.15.76:0/390959931 wait complete. 2024-03-08T06:01:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.117+0000 7fe9524b5640 1 Processor -- start 2024-03-08T06:01:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.118+0000 7fe9524b5640 1 -- start start 2024-03-08T06:01:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.118+0000 7fe9524b5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe934003fa0 0x7fe94c1f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.118+0000 7fe9524b5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe94c13ec80 0x7fe94c13f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.118+0000 7fe9524b5640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe94c1f1900 0x7fe94c1f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.118+0000 7fe9524b5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c0ccd40 con 0x7fe94c13ec80 2024-03-08T06:01:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.118+0000 7fe9524b5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c0cca60 con 0x7fe934003fa0 2024-03-08T06:01:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.118+0000 7fe9524b5640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c0af0e0 con 0x7fe94c1f1900 2024-03-08T06:01:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.118+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe94c13ec80 0x7fe94c13f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.119+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe94c13ec80 0x7fe94c13f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53244/0 (socket says 172.21.15.76:53244) 2024-03-08T06:01:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.119+0000 7fe94b7fe640 1 -- 172.21.15.76:0/2303283036 learned_addr learned my addr 172.21.15.76:0/2303283036 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.119+0000 7fe94bfff640 1 --2- 172.21.15.76:0/2303283036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe934003fa0 0x7fe94c1f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.119+0000 7fe950a52640 1 --2- 172.21.15.76:0/2303283036 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe94c1f1900 0x7fe94c1f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.119+0000 7fe94b7fe640 1 -- 172.21.15.76:0/2303283036 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe94c1f1900 msgr2=0x7fe94c1f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.119+0000 7fe94b7fe640 1 --2- 172.21.15.76:0/2303283036 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe94c1f1900 0x7fe94c1f1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.119+0000 7fe94b7fe640 1 -- 172.21.15.76:0/2303283036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe934003fa0 msgr2=0x7fe94c1f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.119+0000 7fe94b7fe640 1 --2- 172.21.15.76:0/2303283036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe934003fa0 0x7fe94c1f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.119+0000 7fe94b7fe640 1 -- 172.21.15.76:0/2303283036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c15da90 con 0x7fe94c13ec80 2024-03-08T06:01:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.119+0000 7fe94bfff640 1 --2- 172.21.15.76:0/2303283036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe934003fa0 0x7fe94c1f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.119+0000 7fe94b7fe640 1 --2- 172.21.15.76:0/2303283036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe94c13ec80 0x7fe94c13f100 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe94000c330 tx=0x7fe940001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.119+0000 7fe948ff9640 1 -- 172.21.15.76:0/2303283036 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940001cb0 con 0x7fe94c13ec80 2024-03-08T06:01:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.119+0000 7fe948ff9640 1 -- 172.21.15.76:0/2303283036 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe940004c10 con 0x7fe94c13ec80 2024-03-08T06:01:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.119+0000 7fe948ff9640 1 -- 172.21.15.76:0/2303283036 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940007d70 con 0x7fe94c13ec80 2024-03-08T06:01:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.119+0000 7fe9524b5640 1 -- 172.21.15.76:0/2303283036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94c15dd70 con 0x7fe94c13ec80 2024-03-08T06:01:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.119+0000 7fe9524b5640 1 -- 172.21.15.76:0/2303283036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94c15e330 con 0x7fe94c13ec80 2024-03-08T06:01:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.120+0000 7fe9524b5640 1 -- 172.21.15.76:0/2303283036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94c04f030 con 0x7fe94c13ec80 2024-03-08T06:01:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.120+0000 7fe948ff9640 1 -- 172.21.15.76:0/2303283036 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe940048020 con 0x7fe94c13ec80 2024-03-08T06:01:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.121+0000 7fe948ff9640 1 --2- 172.21.15.76:0/2303283036 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe93806b4d0 0x7fe93806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.121+0000 7fe94bfff640 1 --2- 172.21.15.76:0/2303283036 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe93806b4d0 0x7fe93806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.121+0000 7fe948ff9640 1 -- 172.21.15.76:0/2303283036 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe9400ab770 con 0x7fe94c13ec80 2024-03-08T06:01:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.121+0000 7fe94bfff640 1 --2- 172.21.15.76:0/2303283036 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe93806b4d0 0x7fe93806d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009ae0 tx=0x7fe93c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:30.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.125+0000 7fe948ff9640 1 -- 172.21.15.76:0/2303283036 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe94007a1f0 con 0x7fe94c13ec80 2024-03-08T06:01:30.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.295+0000 7fe9524b5640 1 -- 172.21.15.76:0/2303283036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe94c06b270 con 0x7fe94c13ec80 2024-03-08T06:01:30.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.296+0000 7fe948ff9640 1 -- 172.21.15.76:0/2303283036 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe940049e80 con 0x7fe94c13ec80 2024-03-08T06:01:30.296 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:01:30.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.297+0000 7fe9524b5640 1 -- 172.21.15.76:0/2303283036 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe93806b4d0 msgr2=0x7fe93806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.297+0000 7fe9524b5640 1 --2- 172.21.15.76:0/2303283036 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe93806b4d0 0x7fe93806d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009ae0 tx=0x7fe93c008040 comp rx=0 tx=0).stop 2024-03-08T06:01:30.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.297+0000 7fe9524b5640 1 -- 172.21.15.76:0/2303283036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe94c13ec80 msgr2=0x7fe94c13f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.297+0000 7fe9524b5640 1 --2- 172.21.15.76:0/2303283036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe94c13ec80 0x7fe94c13f100 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe94000c330 tx=0x7fe940001a60 comp rx=0 tx=0).stop 2024-03-08T06:01:30.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.297+0000 7fe9524b5640 1 -- 172.21.15.76:0/2303283036 shutdown_connections 2024-03-08T06:01:30.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.297+0000 7fe9524b5640 1 --2- 172.21.15.76:0/2303283036 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe93806b4d0 0x7fe93806d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.297+0000 7fe9524b5640 1 --2- 172.21.15.76:0/2303283036 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe94c1f1900 0x7fe94c1f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.297+0000 7fe9524b5640 1 --2- 172.21.15.76:0/2303283036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe94c13ec80 0x7fe94c13f100 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.297+0000 7fe9524b5640 1 --2- 172.21.15.76:0/2303283036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe934003fa0 0x7fe94c1f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.297+0000 7fe9524b5640 1 -- 172.21.15.76:0/2303283036 >> 172.21.15.76:0/2303283036 conn(0x7fe94c06ac80 msgr2=0x7fe94c15f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:30.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.297+0000 7fe9524b5640 1 -- 172.21.15.76:0/2303283036 shutdown_connections 2024-03-08T06:01:30.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.297+0000 7fe9524b5640 1 -- 172.21.15.76:0/2303283036 wait complete. 2024-03-08T06:01:30.309 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T06:01:30.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.413+0000 7f824084e640 1 Processor -- start 2024-03-08T06:01:30.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.413+0000 7f824084e640 1 -- start start 2024-03-08T06:01:30.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.414+0000 7f824084e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f823c15dbc0 0x7f823c0d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.414+0000 7f824084e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f823c0b5cf0 0x7f823c0b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.414+0000 7f824084e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823c0d27d0 con 0x7f823c0b9840 2024-03-08T06:01:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.414+0000 7f824084e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823c0d2ab0 con 0x7f823c15dbc0 2024-03-08T06:01:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.414+0000 7f824084e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823c0d2d90 con 0x7f823c0b5cf0 2024-03-08T06:01:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.415+0000 7f823ad9d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f823c0b5cf0 0x7f823c0b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.415+0000 7f823a59c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f823c15dbc0 0x7f823c0d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.415+0000 7f8239d9b640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f823c0b9840 0x7f823c0b55e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:34468/0 (socket says 172.21.15.76:34468) 2024-03-08T06:01:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.415+0000 7f8239d9b640 1 -- 172.21.15.76:0/3834456607 learned_addr learned my addr 172.21.15.76:0/3834456607 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.415+0000 7f823ad9d640 1 -- 172.21.15.76:0/3834456607 >> v2:172.21.15.81:3300/0 conn(0x7f823c15dbc0 msgr2=0x7f823c0d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.415+0000 7f823ad9d640 1 --2- 172.21.15.76:0/3834456607 >> v2:172.21.15.81:3300/0 conn(0x7f823c15dbc0 0x7f823c0d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.415+0000 7f823ad9d640 1 -- 172.21.15.76:0/3834456607 >> v1:172.21.15.81:6789/0 conn(0x7f823c0b9840 legacy=0x7f823c0b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.415+0000 7f823ad9d640 1 -- 172.21.15.76:0/3834456607 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c0d3070 con 0x7f823c0b5cf0 2024-03-08T06:01:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.415+0000 7f823a59c640 1 --2- 172.21.15.76:0/3834456607 >> v2:172.21.15.81:3300/0 conn(0x7f823c15dbc0 0x7f823c0d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:30.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.416+0000 7f823ad9d640 1 --2- 172.21.15.76:0/3834456607 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f823c0b5cf0 0x7f823c0b6c90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f822c0097a0 tx=0x7f822c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f87adf8a33ec4e8d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:30.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.416+0000 7f823959a640 1 -- 172.21.15.76:0/3834456607 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c035020 con 0x7f823c0b5cf0 2024-03-08T06:01:30.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.416+0000 7f823959a640 1 -- 172.21.15.76:0/3834456607 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c031830 con 0x7f823c0b5cf0 2024-03-08T06:01:30.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.416+0000 7f823959a640 1 -- 172.21.15.76:0/3834456607 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c031c00 con 0x7f823c0b5cf0 2024-03-08T06:01:30.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.416+0000 7f824084e640 1 -- 172.21.15.76:0/3834456607 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f823c0b5cf0 msgr2=0x7f823c0b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.416+0000 7f824084e640 1 --2- 172.21.15.76:0/3834456607 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f823c0b5cf0 0x7f823c0b6c90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f822c0097a0 tx=0x7f822c02b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.417+0000 7f824084e640 1 -- 172.21.15.76:0/3834456607 shutdown_connections 2024-03-08T06:01:30.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.417+0000 7f824084e640 1 --2- 172.21.15.76:0/3834456607 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f823c0b5cf0 0x7f823c0b6c90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.417+0000 7f824084e640 1 --2- 172.21.15.76:0/3834456607 >> v2:172.21.15.81:3300/0 conn(0x7f823c15dbc0 0x7f823c0d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.417+0000 7f824084e640 1 -- 172.21.15.76:0/3834456607 >> 172.21.15.76:0/3834456607 conn(0x7f823c0670d0 msgr2=0x7f823c0d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:30.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.417+0000 7f824084e640 1 -- 172.21.15.76:0/3834456607 shutdown_connections 2024-03-08T06:01:30.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.417+0000 7f824084e640 1 -- 172.21.15.76:0/3834456607 wait complete. 2024-03-08T06:01:30.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.418+0000 7f824084e640 1 Processor -- start 2024-03-08T06:01:30.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.418+0000 7f824084e640 1 -- start start 2024-03-08T06:01:30.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f824084e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c0b5cf0 0x7f823c1f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f824084e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f823c0b9840 0x7f823c1e3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f824084e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f823c15dbc0 0x7f823c0cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f824084e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c0d25b0 con 0x7f823c0b5cf0 2024-03-08T06:01:30.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f824084e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c0d2430 con 0x7f823c15dbc0 2024-03-08T06:01:30.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f824084e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823c15dfa0 con 0x7f823c0b9840 2024-03-08T06:01:30.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f823a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c0b5cf0 0x7f823c1f78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f823a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c0b5cf0 0x7f823c1f78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53258/0 (socket says 172.21.15.76:53258) 2024-03-08T06:01:30.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f823a59c640 1 -- 172.21.15.76:0/4097066027 learned_addr learned my addr 172.21.15.76:0/4097066027 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:30.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f823ad9d640 1 --2- 172.21.15.76:0/4097066027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f823c15dbc0 0x7f823c0cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f8239d9b640 1 --2- 172.21.15.76:0/4097066027 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f823c0b9840 0x7f823c1e3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f823a59c640 1 -- 172.21.15.76:0/4097066027 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f823c0b9840 msgr2=0x7f823c1e3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f823a59c640 1 --2- 172.21.15.76:0/4097066027 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f823c0b9840 0x7f823c1e3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f823a59c640 1 -- 172.21.15.76:0/4097066027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f823c15dbc0 msgr2=0x7f823c0cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f823a59c640 1 --2- 172.21.15.76:0/4097066027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f823c15dbc0 0x7f823c0cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f823a59c640 1 -- 172.21.15.76:0/4097066027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c1f4710 con 0x7f823c0b5cf0 2024-03-08T06:01:30.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f823a59c640 1 --2- 172.21.15.76:0/4097066027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c0b5cf0 0x7f823c1f78b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f823400e960 tx=0x7f823400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:30.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f822affd640 1 -- 172.21.15.76:0/4097066027 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234017070 con 0x7f823c0b5cf0 2024-03-08T06:01:30.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f822affd640 1 -- 172.21.15.76:0/4097066027 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823400ca00 con 0x7f823c0b5cf0 2024-03-08T06:01:30.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f824084e640 1 -- 172.21.15.76:0/4097066027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823c1f3950 con 0x7f823c0b5cf0 2024-03-08T06:01:30.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.419+0000 7f822affd640 1 -- 172.21.15.76:0/4097066027 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823400cdd0 con 0x7f823c0b5cf0 2024-03-08T06:01:30.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.420+0000 7f824084e640 1 -- 172.21.15.76:0/4097066027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823c1f3f10 con 0x7f823c0b5cf0 2024-03-08T06:01:30.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.420+0000 7f824084e640 1 -- 172.21.15.76:0/4097066027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823c04f030 con 0x7f823c0b5cf0 2024-03-08T06:01:30.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.421+0000 7f822affd640 1 -- 172.21.15.76:0/4097066027 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f823401a070 con 0x7f823c0b5cf0 2024-03-08T06:01:30.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.421+0000 7f822affd640 1 --2- 172.21.15.76:0/4097066027 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f821c06b4d0 0x7f821c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.421+0000 7f822affd640 1 -- 172.21.15.76:0/4097066027 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f823408a730 con 0x7f823c0b5cf0 2024-03-08T06:01:30.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.421+0000 7f8239d9b640 1 --2- 172.21.15.76:0/4097066027 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f821c06b4d0 0x7f821c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.422+0000 7f8239d9b640 1 --2- 172.21.15.76:0/4097066027 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f821c06b4d0 0x7f821c06d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f823c157620 tx=0x7f8230001a30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:30.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.425+0000 7f822affd640 1 -- 172.21.15.76:0/4097066027 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8234058bf0 con 0x7f823c0b5cf0 2024-03-08T06:01:30.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.594+0000 7f824084e640 1 -- 172.21.15.76:0/4097066027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f823c0ca000 con 0x7f823c0b5cf0 2024-03-08T06:01:30.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.594+0000 7f822affd640 1 -- 172.21.15.76:0/4097066027 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f823405cc90 con 0x7f823c0b5cf0 2024-03-08T06:01:30.595 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:30.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.596+0000 7f824084e640 1 -- 172.21.15.76:0/4097066027 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f821c06b4d0 msgr2=0x7f821c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.596+0000 7f824084e640 1 --2- 172.21.15.76:0/4097066027 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f821c06b4d0 0x7f821c06d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f823c157620 tx=0x7f8230001a30 comp rx=0 tx=0).stop 2024-03-08T06:01:30.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.596+0000 7f824084e640 1 -- 172.21.15.76:0/4097066027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c0b5cf0 msgr2=0x7f823c1f78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.596+0000 7f824084e640 1 --2- 172.21.15.76:0/4097066027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c0b5cf0 0x7f823c1f78b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f823400e960 tx=0x7f823400ee20 comp rx=0 tx=0).stop 2024-03-08T06:01:30.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.596+0000 7f824084e640 1 -- 172.21.15.76:0/4097066027 shutdown_connections 2024-03-08T06:01:30.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.596+0000 7f824084e640 1 --2- 172.21.15.76:0/4097066027 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f821c06b4d0 0x7f821c06d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.596+0000 7f824084e640 1 --2- 172.21.15.76:0/4097066027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f823c15dbc0 0x7f823c0cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.596+0000 7f824084e640 1 --2- 172.21.15.76:0/4097066027 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f823c0b9840 0x7f823c1e3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.596+0000 7f824084e640 1 --2- 172.21.15.76:0/4097066027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c0b5cf0 0x7f823c1f78b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.596+0000 7f824084e640 1 -- 172.21.15.76:0/4097066027 >> 172.21.15.76:0/4097066027 conn(0x7f823c0670d0 msgr2=0x7f823c0c7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:30.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.596+0000 7f824084e640 1 -- 172.21.15.76:0/4097066027 shutdown_connections 2024-03-08T06:01:30.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.596+0000 7f824084e640 1 -- 172.21.15.76:0/4097066027 wait complete. 2024-03-08T06:01:30.608 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T06:01:30.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.708+0000 7f6643b1e640 1 Processor -- start 2024-03-08T06:01:30.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.708+0000 7f6643b1e640 1 -- start start 2024-03-08T06:01:30.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.709+0000 7f6643b1e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f663c154df0 0x7f663c0b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.709+0000 7f6643b1e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f663c0cc7e0 0x7f663c15af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.709+0000 7f6643b1e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663c15f450 con 0x7f663c0b3640 2024-03-08T06:01:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.709+0000 7f6643b1e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663c0af1e0 con 0x7f663c0cc7e0 2024-03-08T06:01:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.709+0000 7f6643b1e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663c0b3c80 con 0x7f663c154df0 2024-03-08T06:01:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.709+0000 7f66410b9640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f663c0b3640 0x7f663c0cc0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46734/0 (socket says 172.21.15.76:46734) 2024-03-08T06:01:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.709+0000 7f66420bb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f663c0cc7e0 0x7f663c15af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.709+0000 7f66410b9640 1 -- 172.21.15.76:0/710681528 learned_addr learned my addr 172.21.15.76:0/710681528 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.709+0000 7f66418ba640 1 --2- 172.21.15.76:0/710681528 >> v2:172.21.15.81:3300/0 conn(0x7f663c154df0 0x7f663c0b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.709+0000 7f66420bb640 1 -- 172.21.15.76:0/710681528 >> v2:172.21.15.81:3300/0 conn(0x7f663c154df0 msgr2=0x7f663c0b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.709+0000 7f66420bb640 1 --2- 172.21.15.76:0/710681528 >> v2:172.21.15.81:3300/0 conn(0x7f663c154df0 0x7f663c0b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.709+0000 7f66420bb640 1 -- 172.21.15.76:0/710681528 >> v1:172.21.15.76:6789/0 conn(0x7f663c0b3640 legacy=0x7f663c0cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.709+0000 7f66408b8640 1 -- 172.21.15.76:0/710681528 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1255663999 0 0) 0x7f663c15f450 con 0x7f663c0b3640 2024-03-08T06:01:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.709+0000 7f66420bb640 1 -- 172.21.15.76:0/710681528 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c15e690 con 0x7f663c0cc7e0 2024-03-08T06:01:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.710+0000 7f66418ba640 1 --2- 172.21.15.76:0/710681528 >> v2:172.21.15.81:3300/0 conn(0x7f663c154df0 0x7f663c0b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.710+0000 7f66420bb640 1 --2- 172.21.15.76:0/710681528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f663c0cc7e0 0x7f663c15af70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f663c06ba20 tx=0x7f663402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76978827d2a668c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.710+0000 7f66408b8640 1 -- 172.21.15.76:0/710681528 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66340315a0 con 0x7f663c0cc7e0 2024-03-08T06:01:30.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.710+0000 7f66408b8640 1 -- 172.21.15.76:0/710681528 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6634037070 con 0x7f663c0cc7e0 2024-03-08T06:01:30.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.710+0000 7f66408b8640 1 -- 172.21.15.76:0/710681528 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6634031960 con 0x7f663c0cc7e0 2024-03-08T06:01:30.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.711+0000 7f6643b1e640 1 -- 172.21.15.76:0/710681528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f663c0cc7e0 msgr2=0x7f663c15af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.711+0000 7f6643b1e640 1 --2- 172.21.15.76:0/710681528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f663c0cc7e0 0x7f663c15af70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f663c06ba20 tx=0x7f663402b840 comp rx=0 tx=0).stop 2024-03-08T06:01:30.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.711+0000 7f6643b1e640 1 -- 172.21.15.76:0/710681528 shutdown_connections 2024-03-08T06:01:30.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.711+0000 7f6643b1e640 1 --2- 172.21.15.76:0/710681528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f663c0cc7e0 0x7f663c15af70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.711+0000 7f6643b1e640 1 --2- 172.21.15.76:0/710681528 >> v2:172.21.15.81:3300/0 conn(0x7f663c154df0 0x7f663c0b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.711+0000 7f6643b1e640 1 -- 172.21.15.76:0/710681528 >> 172.21.15.76:0/710681528 conn(0x7f663c06ac80 msgr2=0x7f663c06b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:30.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.711+0000 7f6643b1e640 1 -- 172.21.15.76:0/710681528 shutdown_connections 2024-03-08T06:01:30.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.711+0000 7f6643b1e640 1 -- 172.21.15.76:0/710681528 wait complete. 2024-03-08T06:01:30.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.712+0000 7f6643b1e640 1 Processor -- start 2024-03-08T06:01:30.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.712+0000 7f6643b1e640 1 -- start start 2024-03-08T06:01:30.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f6643b1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f663c0b3640 0x7f663c13df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f6643b1e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f663c0cc7e0 0x7f663c14bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f6643b1e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f663c154df0 0x7f663c0af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f6643b1e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c15ef30 con 0x7f663c0b3640 2024-03-08T06:01:30.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f6643b1e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c15ec50 con 0x7f663c0cc7e0 2024-03-08T06:01:30.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f6643b1e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c1551d0 con 0x7f663c154df0 2024-03-08T06:01:30.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f66418ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f663c0b3640 0x7f663c13df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f66418ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f663c0b3640 0x7f663c13df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53272/0 (socket says 172.21.15.76:53272) 2024-03-08T06:01:30.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f66418ba640 1 -- 172.21.15.76:0/878034336 learned_addr learned my addr 172.21.15.76:0/878034336 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:30.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f66410b9640 1 --2- 172.21.15.76:0/878034336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f663c0cc7e0 0x7f663c14bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f66420bb640 1 --2- 172.21.15.76:0/878034336 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f663c154df0 0x7f663c0af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f66418ba640 1 -- 172.21.15.76:0/878034336 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f663c154df0 msgr2=0x7f663c0af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f66418ba640 1 --2- 172.21.15.76:0/878034336 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f663c154df0 0x7f663c0af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f66418ba640 1 -- 172.21.15.76:0/878034336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f663c0cc7e0 msgr2=0x7f663c14bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f66418ba640 1 --2- 172.21.15.76:0/878034336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f663c0cc7e0 0x7f663c14bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f66418ba640 1 -- 172.21.15.76:0/878034336 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c14b1e0 con 0x7f663c0b3640 2024-03-08T06:01:30.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f66410b9640 1 --2- 172.21.15.76:0/878034336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f663c0cc7e0 0x7f663c14bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:30.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.713+0000 7f66418ba640 1 --2- 172.21.15.76:0/878034336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f663c0b3640 0x7f663c13df80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6638009fd0 tx=0x7f663800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:30.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.714+0000 7f66327fc640 1 -- 172.21.15.76:0/878034336 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663801a070 con 0x7f663c0b3640 2024-03-08T06:01:30.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.714+0000 7f66327fc640 1 -- 172.21.15.76:0/878034336 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6638003bb0 con 0x7f663c0b3640 2024-03-08T06:01:30.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.714+0000 7f66327fc640 1 -- 172.21.15.76:0/878034336 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6638004030 con 0x7f663c0b3640 2024-03-08T06:01:30.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.714+0000 7f6643b1e640 1 -- 172.21.15.76:0/878034336 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663c14b4c0 con 0x7f663c0b3640 2024-03-08T06:01:30.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.714+0000 7f6643b1e640 1 -- 172.21.15.76:0/878034336 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c14ba80 con 0x7f663c0b3640 2024-03-08T06:01:30.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.715+0000 7f6643b1e640 1 -- 172.21.15.76:0/878034336 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663c04f030 con 0x7f663c0b3640 2024-03-08T06:01:30.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.715+0000 7f66327fc640 1 -- 172.21.15.76:0/878034336 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f663801d070 con 0x7f663c0b3640 2024-03-08T06:01:30.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.715+0000 7f66327fc640 1 --2- 172.21.15.76:0/878034336 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f662006b4d0 0x7f662006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:30.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.715+0000 7f66327fc640 1 -- 172.21.15.76:0/878034336 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f663808a740 con 0x7f663c0b3640 2024-03-08T06:01:30.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.715+0000 7f66410b9640 1 --2- 172.21.15.76:0/878034336 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f662006b4d0 0x7f662006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:30.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.716+0000 7f66410b9640 1 --2- 172.21.15.76:0/878034336 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f662006b4d0 0x7f662006d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f663c14cb80 tx=0x7f662c007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:30.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.720+0000 7f66327fc640 1 -- 172.21.15.76:0/878034336 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6638010070 con 0x7f663c0b3640 2024-03-08T06:01:30.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.902+0000 7f6643b1e640 1 -- 172.21.15.76:0/878034336 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f663c0aed90 con 0x7f663c0b3640 2024-03-08T06:01:30.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.902+0000 7f66327fc640 1 -- 172.21.15.76:0/878034336 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6638058c00 con 0x7f663c0b3640 2024-03-08T06:01:30.903 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-03-08T06:01:30.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.904+0000 7f6643b1e640 1 -- 172.21.15.76:0/878034336 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f662006b4d0 msgr2=0x7f662006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.904+0000 7f6643b1e640 1 --2- 172.21.15.76:0/878034336 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f662006b4d0 0x7f662006d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f663c14cb80 tx=0x7f662c007c10 comp rx=0 tx=0).stop 2024-03-08T06:01:30.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.904+0000 7f6643b1e640 1 -- 172.21.15.76:0/878034336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f663c0b3640 msgr2=0x7f663c13df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:30.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.904+0000 7f6643b1e640 1 --2- 172.21.15.76:0/878034336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f663c0b3640 0x7f663c13df80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6638009fd0 tx=0x7f663800d560 comp rx=0 tx=0).stop 2024-03-08T06:01:30.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.904+0000 7f6643b1e640 1 -- 172.21.15.76:0/878034336 shutdown_connections 2024-03-08T06:01:30.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.904+0000 7f6643b1e640 1 --2- 172.21.15.76:0/878034336 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f662006b4d0 0x7f662006d990 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.904+0000 7f6643b1e640 1 --2- 172.21.15.76:0/878034336 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f663c154df0 0x7f663c0af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.904+0000 7f6643b1e640 1 --2- 172.21.15.76:0/878034336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f663c0cc7e0 0x7f663c14bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.904+0000 7f6643b1e640 1 --2- 172.21.15.76:0/878034336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f663c0b3640 0x7f663c13df80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:30.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.904+0000 7f6643b1e640 1 -- 172.21.15.76:0/878034336 >> 172.21.15.76:0/878034336 conn(0x7f663c06ac80 msgr2=0x7f663c15a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:30.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.904+0000 7f6643b1e640 1 -- 172.21.15.76:0/878034336 shutdown_connections 2024-03-08T06:01:30.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:30.904+0000 7f6643b1e640 1 -- 172.21.15.76:0/878034336 wait complete. 2024-03-08T06:01:30.915 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-03-08T06:01:30.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:30.915 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-03-08T06:01:30.943 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-03-08T06:01:30.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:30.943 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-08T06:01:30.998 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:01:30.998 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:31.017 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:01:31.017 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:31.076 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:01:31.076 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:31.133 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:31.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.233+0000 7f667c548640 1 Processor -- start 2024-03-08T06:01:31.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.233+0000 7f667c548640 1 -- start start 2024-03-08T06:01:31.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.234+0000 7f667c548640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f66740ceee0 0x7f667415aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.234+0000 7f667c548640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66740cdc50 0x7f66740cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.234+0000 7f667c548640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66740cf800 con 0x7f66740ce4f0 2024-03-08T06:01:31.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.234+0000 7f667c548640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f667415bea0 con 0x7f66740ceee0 2024-03-08T06:01:31.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.234+0000 7f667c548640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667415b570 con 0x7f66740cdc50 2024-03-08T06:01:31.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.234+0000 7f667a2e4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f66740ceee0 0x7f667415aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.234+0000 7f667aae5640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f66740ce4f0 0x7f667415e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46736/0 (socket says 172.21.15.76:46736) 2024-03-08T06:01:31.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.234+0000 7f667aae5640 1 -- 172.21.15.76:0/1226228902 learned_addr learned my addr 172.21.15.76:0/1226228902 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:31.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.235+0000 7f6679ae3640 1 --2- 172.21.15.76:0/1226228902 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66740cdc50 0x7f66740cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.235+0000 7f66792e2640 1 -- 172.21.15.76:0/1226228902 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679633411 0 0) 0x7f66740cf800 con 0x7f66740ce4f0 2024-03-08T06:01:31.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.235+0000 7f66792e2640 1 -- 172.21.15.76:0/1226228902 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6660003520 con 0x7f66740ce4f0 2024-03-08T06:01:31.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.235+0000 7f667a2e4640 1 -- 172.21.15.76:0/1226228902 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66740cdc50 msgr2=0x7f66740cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:31.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.235+0000 7f667a2e4640 1 --2- 172.21.15.76:0/1226228902 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66740cdc50 0x7f66740cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.235+0000 7f667a2e4640 1 -- 172.21.15.76:0/1226228902 >> v1:172.21.15.76:6789/0 conn(0x7f66740ce4f0 legacy=0x7f667415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.235+0000 7f667a2e4640 1 -- 172.21.15.76:0/1226228902 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667415b850 con 0x7f66740ceee0 2024-03-08T06:01:31.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.235+0000 7f6679ae3640 1 --2- 172.21.15.76:0/1226228902 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66740cdc50 0x7f66740cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:31.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.235+0000 7f667a2e4640 1 --2- 172.21.15.76:0/1226228902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66740ceee0 0x7f667415aed0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6670009d90 tx=0x7f667002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bb2b418f407128 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:31.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.236+0000 7f66792e2640 1 -- 172.21.15.76:0/1226228902 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670039070 con 0x7f66740ceee0 2024-03-08T06:01:31.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.236+0000 7f66792e2640 1 -- 172.21.15.76:0/1226228902 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667002cc50 con 0x7f66740ceee0 2024-03-08T06:01:31.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.236+0000 7f66792e2640 1 -- 172.21.15.76:0/1226228902 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670034440 con 0x7f66740ceee0 2024-03-08T06:01:31.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.236+0000 7f667c548640 1 -- 172.21.15.76:0/1226228902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66740ceee0 msgr2=0x7f667415aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.236+0000 7f667c548640 1 --2- 172.21.15.76:0/1226228902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66740ceee0 0x7f667415aed0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6670009d90 tx=0x7f667002bea0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.236+0000 7f667c548640 1 -- 172.21.15.76:0/1226228902 shutdown_connections 2024-03-08T06:01:31.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.236+0000 7f667c548640 1 --2- 172.21.15.76:0/1226228902 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66740cdc50 0x7f66740cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.236+0000 7f667c548640 1 --2- 172.21.15.76:0/1226228902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66740ceee0 0x7f667415aed0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.236+0000 7f667c548640 1 -- 172.21.15.76:0/1226228902 >> 172.21.15.76:0/1226228902 conn(0x7f6674099a60 msgr2=0x7f66740b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:31.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.236+0000 7f667c548640 1 -- 172.21.15.76:0/1226228902 shutdown_connections 2024-03-08T06:01:31.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.236+0000 7f667c548640 1 -- 172.21.15.76:0/1226228902 wait complete. 2024-03-08T06:01:31.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.237+0000 7f667c548640 1 Processor -- start 2024-03-08T06:01:31.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.237+0000 7f667c548640 1 -- start start 2024-03-08T06:01:31.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.238+0000 7f667c548640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66740cdc50 0x7f66740b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.238+0000 7f667c548640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66740ce4f0 0x7f66740c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.238+0000 7f667c548640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66740ceee0 0x7f6674155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.238+0000 7f667c548640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66740ceb00 con 0x7f66740cdc50 2024-03-08T06:01:31.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.238+0000 7f667c548640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66740ce980 con 0x7f66740ce4f0 2024-03-08T06:01:31.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.238+0000 7f667c548640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66740cc400 con 0x7f66740ceee0 2024-03-08T06:01:31.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.238+0000 7f667a2e4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66740cdc50 0x7f66740b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.238+0000 7f667a2e4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66740cdc50 0x7f66740b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53280/0 (socket says 172.21.15.76:53280) 2024-03-08T06:01:31.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.238+0000 7f667a2e4640 1 -- 172.21.15.76:0/2350971171 learned_addr learned my addr 172.21.15.76:0/2350971171 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:31.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.238+0000 7f667a2e4640 1 -- 172.21.15.76:0/2350971171 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66740ceee0 msgr2=0x7f6674155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.238+0000 7f667aae5640 1 --2- 172.21.15.76:0/2350971171 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66740ceee0 0x7f6674155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.238+0000 7f6679ae3640 1 --2- 172.21.15.76:0/2350971171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66740ce4f0 0x7f66740c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.238+0000 7f667a2e4640 1 --2- 172.21.15.76:0/2350971171 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66740ceee0 0x7f6674155b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.238+0000 7f667a2e4640 1 -- 172.21.15.76:0/2350971171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66740ce4f0 msgr2=0x7f66740c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.238+0000 7f667a2e4640 1 --2- 172.21.15.76:0/2350971171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66740ce4f0 0x7f66740c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.238+0000 7f667a2e4640 1 -- 172.21.15.76:0/2350971171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66740b4cc0 con 0x7f66740cdc50 2024-03-08T06:01:31.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.239+0000 7f667a2e4640 1 --2- 172.21.15.76:0/2350971171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66740cdc50 0x7f66740b45e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6670009d90 tx=0x7f6670004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:31.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.239+0000 7f666affd640 1 -- 172.21.15.76:0/2350971171 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670044020 con 0x7f66740cdc50 2024-03-08T06:01:31.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.239+0000 7f666affd640 1 -- 172.21.15.76:0/2350971171 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667002ca50 con 0x7f66740cdc50 2024-03-08T06:01:31.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.239+0000 7f667c548640 1 -- 172.21.15.76:0/2350971171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66740b4fa0 con 0x7f66740cdc50 2024-03-08T06:01:31.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.239+0000 7f666affd640 1 -- 172.21.15.76:0/2350971171 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670039040 con 0x7f66740cdc50 2024-03-08T06:01:31.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.239+0000 7f667c548640 1 -- 172.21.15.76:0/2350971171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66740c43c0 con 0x7f66740cdc50 2024-03-08T06:01:31.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.240+0000 7f667c548640 1 -- 172.21.15.76:0/2350971171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667404f030 con 0x7f66740cdc50 2024-03-08T06:01:31.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.240+0000 7f666affd640 1 -- 172.21.15.76:0/2350971171 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6670047020 con 0x7f66740cdc50 2024-03-08T06:01:31.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.240+0000 7f666affd640 1 --2- 172.21.15.76:0/2350971171 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f665806b4d0 0x7f665806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.241+0000 7f666affd640 1 -- 172.21.15.76:0/2350971171 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f66700acfc0 con 0x7f66740cdc50 2024-03-08T06:01:31.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.241+0000 7f6679ae3640 1 --2- 172.21.15.76:0/2350971171 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f665806b4d0 0x7f665806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.241+0000 7f6679ae3640 1 --2- 172.21.15.76:0/2350971171 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f665806b4d0 0x7f665806d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f66740cd6d0 tx=0x7f6664008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:31.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.245+0000 7f666affd640 1 -- 172.21.15.76:0/2350971171 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f667004e030 con 0x7f66740cdc50 2024-03-08T06:01:31.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.416+0000 7f667c548640 1 -- 172.21.15.76:0/2350971171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66740c7760 con 0x7f66740cdc50 2024-03-08T06:01:31.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.417+0000 7f666affd640 1 -- 172.21.15.76:0/2350971171 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18757 (secure 0 0 0) 0x7f667002cc20 con 0x7f66740cdc50 2024-03-08T06:01:31.419 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:31.419 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":28,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:01:17.313272+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:31.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.419+0000 7f667c548640 1 -- 172.21.15.76:0/2350971171 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f665806b4d0 msgr2=0x7f665806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.419+0000 7f667c548640 1 --2- 172.21.15.76:0/2350971171 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f665806b4d0 0x7f665806d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f66740cd6d0 tx=0x7f6664008040 comp rx=0 tx=0).stop 2024-03-08T06:01:31.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.419+0000 7f667c548640 1 -- 172.21.15.76:0/2350971171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66740cdc50 msgr2=0x7f66740b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.419+0000 7f667c548640 1 --2- 172.21.15.76:0/2350971171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66740cdc50 0x7f66740b45e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6670009d90 tx=0x7f6670004a40 comp rx=0 tx=0).stop 2024-03-08T06:01:31.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.420+0000 7f667c548640 1 -- 172.21.15.76:0/2350971171 shutdown_connections 2024-03-08T06:01:31.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.420+0000 7f667c548640 1 --2- 172.21.15.76:0/2350971171 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f665806b4d0 0x7f665806d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.420+0000 7f667c548640 1 --2- 172.21.15.76:0/2350971171 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66740ceee0 0x7f6674155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.420+0000 7f667c548640 1 --2- 172.21.15.76:0/2350971171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66740ce4f0 0x7f66740c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.420+0000 7f667c548640 1 --2- 172.21.15.76:0/2350971171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66740cdc50 0x7f66740b45e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.420+0000 7f667c548640 1 -- 172.21.15.76:0/2350971171 >> 172.21.15.76:0/2350971171 conn(0x7f6674099a60 msgr2=0x7f66740cb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:31.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.420+0000 7f667c548640 1 -- 172.21.15.76:0/2350971171 shutdown_connections 2024-03-08T06:01:31.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.420+0000 7f667c548640 1 -- 172.21.15.76:0/2350971171 wait complete. 2024-03-08T06:01:31.432 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:00:33.926651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:00:34.474030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T06:00:38.207853+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T06:01:31.433 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:01:31.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.535+0000 7f0a61f0d640 1 Processor -- start 2024-03-08T06:01:31.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.535+0000 7f0a61f0d640 1 -- start start 2024-03-08T06:01:31.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.536+0000 7f0a61f0d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0a5c15dfb0 0x7f0a5c15e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.536+0000 7f0a61f0d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5c161840 con 0x7f0a5c06d8a0 2024-03-08T06:01:31.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.536+0000 7f0a61f0d640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5c15eb30 con 0x7f0a5c0ce5e0 2024-03-08T06:01:31.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.536+0000 7f0a61f0d640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5c161b90 con 0x7f0a5c15dfb0 2024-03-08T06:01:31.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.536+0000 7f0a5affd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0a5c06d8a0 0x7f0a5c162020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46738/0 (socket says 172.21.15.76:46738) 2024-03-08T06:01:31.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.536+0000 7f0a5affd640 1 -- 172.21.15.76:0/3312553161 learned_addr learned my addr 172.21.15.76:0/3312553161 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:31.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.536+0000 7f0a5bfff640 1 --2- 172.21.15.76:0/3312553161 >> v2:172.21.15.81:3300/0 conn(0x7f0a5c15dfb0 0x7f0a5c15e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.536+0000 7f0a5a7fc640 1 -- 172.21.15.76:0/3312553161 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425331211 0 0) 0x7f0a5c161840 con 0x7f0a5c06d8a0 2024-03-08T06:01:31.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.536+0000 7f0a5a7fc640 1 -- 172.21.15.76:0/3312553161 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a44003520 con 0x7f0a5c06d8a0 2024-03-08T06:01:31.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5a7fc640 1 -- 172.21.15.76:0/3312553161 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4160127377 0 0) 0x7f0a44003520 con 0x7f0a5c06d8a0 2024-03-08T06:01:31.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5a7fc640 1 -- 172.21.15.76:0/3312553161 >> v2:172.21.15.81:3300/0 conn(0x7f0a5c15dfb0 msgr2=0x7f0a5c15e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:31.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5a7fc640 1 --2- 172.21.15.76:0/3312553161 >> v2:172.21.15.81:3300/0 conn(0x7f0a5c15dfb0 0x7f0a5c15e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5a7fc640 1 -- 172.21.15.76:0/3312553161 >> v1:172.21.15.81:6789/0 conn(0x7f0a5c0ce5e0 legacy=0x7f0a5c0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5a7fc640 1 -- 172.21.15.76:0/3312553161 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c15ee10 con 0x7f0a5c06d8a0 2024-03-08T06:01:31.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5a7fc640 1 -- 172.21.15.76:0/3312553161 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f0a50001670 con 0x7f0a5c06d8a0 2024-03-08T06:01:31.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5a7fc640 1 -- 172.21.15.76:0/3312553161 >> v1:172.21.15.76:6789/0 conn(0x7f0a5c06d8a0 legacy=0x7f0a5c162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5a7fc640 1 --2- 172.21.15.76:0/3312553161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a44003a20 0x7f0a44003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5a7fc640 1 --2- 172.21.15.76:0/3312553161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a44004800 0x7f0a44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5a7fc640 1 --2- 172.21.15.76:0/3312553161 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a440051c0 0x7f0a4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5a7fc640 1 -- 172.21.15.76:0/3312553161 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c161b90 con 0x7f0a44003a20 2024-03-08T06:01:31.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5a7fc640 1 -- 172.21.15.76:0/3312553161 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4400bfb0 con 0x7f0a44004800 2024-03-08T06:01:31.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5a7fc640 1 -- 172.21.15.76:0/3312553161 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4400c1b0 con 0x7f0a440051c0 2024-03-08T06:01:31.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5b7fe640 1 --2- 172.21.15.76:0/3312553161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a44003a20 0x7f0a44003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5affd640 1 --2- 172.21.15.76:0/3312553161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a44004800 0x7f0a44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5bfff640 1 --2- 172.21.15.76:0/3312553161 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a440051c0 0x7f0a4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5b7fe640 1 -- 172.21.15.76:0/3312553161 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a440051c0 msgr2=0x7f0a4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:31.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5b7fe640 1 --2- 172.21.15.76:0/3312553161 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a440051c0 0x7f0a4400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5b7fe640 1 -- 172.21.15.76:0/3312553161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a44004800 msgr2=0x7f0a44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:31.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5b7fe640 1 --2- 172.21.15.76:0/3312553161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a44004800 0x7f0a44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5b7fe640 1 -- 172.21.15.76:0/3312553161 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a44003520 con 0x7f0a44003a20 2024-03-08T06:01:31.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.537+0000 7f0a5bfff640 1 -- 172.21.15.76:0/3312553161 reap_dead start 2024-03-08T06:01:31.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.538+0000 7f0a5b7fe640 1 --2- 172.21.15.76:0/3312553161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a44003a20 0x7f0a44003e00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c001830 tx=0x7f0a44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2e79500b8406e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:31.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.538+0000 7f0a5a7fc640 1 -- 172.21.15.76:0/3312553161 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c001430 con 0x7f0a44003a20 2024-03-08T06:01:31.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.538+0000 7f0a5a7fc640 1 -- 172.21.15.76:0/3312553161 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c032620 con 0x7f0a44003a20 2024-03-08T06:01:31.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.538+0000 7f0a5a7fc640 1 -- 172.21.15.76:0/3312553161 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c032a10 con 0x7f0a44003a20 2024-03-08T06:01:31.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.538+0000 7f0a61f0d640 1 -- 172.21.15.76:0/3312553161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a44003a20 msgr2=0x7f0a44003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.538+0000 7f0a61f0d640 1 --2- 172.21.15.76:0/3312553161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a44003a20 0x7f0a44003e00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c001830 tx=0x7f0a44005830 comp rx=0 tx=0).stop 2024-03-08T06:01:31.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.539+0000 7f0a61f0d640 1 -- 172.21.15.76:0/3312553161 shutdown_connections 2024-03-08T06:01:31.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.539+0000 7f0a61f0d640 1 --2- 172.21.15.76:0/3312553161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a44003a20 0x7f0a44003e00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.539+0000 7f0a61f0d640 1 -- 172.21.15.76:0/3312553161 >> 172.21.15.76:0/3312553161 conn(0x7f0a5c06ae60 msgr2=0x7f0a5c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:31.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.539+0000 7f0a61f0d640 1 -- 172.21.15.76:0/3312553161 shutdown_connections 2024-03-08T06:01:31.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.539+0000 7f0a61f0d640 1 -- 172.21.15.76:0/3312553161 wait complete. 2024-03-08T06:01:31.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.540+0000 7f0a61f0d640 1 Processor -- start 2024-03-08T06:01:31.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.540+0000 7f0a61f0d640 1 -- start start 2024-03-08T06:01:31.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a61f0d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a44003a20 0x7f0a5c0c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a61f0d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0cb280 0x7f0a5c1f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a61f0d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a5c1f7a50 0x7f0a5c159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a61f0d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c15e5c0 con 0x7f0a5c0cb280 2024-03-08T06:01:31.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a61f0d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c15e440 con 0x7f0a5c1f7a50 2024-03-08T06:01:31.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a61f0d640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c15e740 con 0x7f0a44003a20 2024-03-08T06:01:31.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a5affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0cb280 0x7f0a5c1f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a5affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0cb280 0x7f0a5c1f7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53294/0 (socket says 172.21.15.76:53294) 2024-03-08T06:01:31.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a5affd640 1 -- 172.21.15.76:0/2563928478 learned_addr learned my addr 172.21.15.76:0/2563928478 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:31.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a5b7fe640 1 --2- 172.21.15.76:0/2563928478 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a44003a20 0x7f0a5c0c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a5bfff640 1 --2- 172.21.15.76:0/2563928478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a5c1f7a50 0x7f0a5c159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a5affd640 1 -- 172.21.15.76:0/2563928478 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a44003a20 msgr2=0x7f0a5c0c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a5affd640 1 --2- 172.21.15.76:0/2563928478 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a44003a20 0x7f0a5c0c8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a5affd640 1 -- 172.21.15.76:0/2563928478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a5c1f7a50 msgr2=0x7f0a5c159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a5affd640 1 --2- 172.21.15.76:0/2563928478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a5c1f7a50 0x7f0a5c159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a5affd640 1 -- 172.21.15.76:0/2563928478 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c0ca130 con 0x7f0a5c0cb280 2024-03-08T06:01:31.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a5affd640 1 --2- 172.21.15.76:0/2563928478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0cb280 0x7f0a5c1f7510 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0a50004f70 tx=0x7f0a500074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:31.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a43fff640 1 -- 172.21.15.76:0/2563928478 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5000f750 con 0x7f0a5c0cb280 2024-03-08T06:01:31.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a43fff640 1 -- 172.21.15.76:0/2563928478 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a50018070 con 0x7f0a5c0cb280 2024-03-08T06:01:31.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a61f0d640 1 -- 172.21.15.76:0/2563928478 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5c0ca410 con 0x7f0a5c0cb280 2024-03-08T06:01:31.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a43fff640 1 -- 172.21.15.76:0/2563928478 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5000fb50 con 0x7f0a5c0cb280 2024-03-08T06:01:31.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.541+0000 7f0a61f0d640 1 -- 172.21.15.76:0/2563928478 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5c0cb700 con 0x7f0a5c0cb280 2024-03-08T06:01:31.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.542+0000 7f0a61f0d640 1 -- 172.21.15.76:0/2563928478 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5c1f7e60 con 0x7f0a5c0cb280 2024-03-08T06:01:31.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.543+0000 7f0a43fff640 1 -- 172.21.15.76:0/2563928478 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a5000fcf0 con 0x7f0a5c0cb280 2024-03-08T06:01:31.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.543+0000 7f0a43fff640 1 --2- 172.21.15.76:0/2563928478 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0a4806b500 0x7f0a4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.543+0000 7f0a5b7fe640 1 --2- 172.21.15.76:0/2563928478 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0a4806b500 0x7f0a4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.543+0000 7f0a43fff640 1 -- 172.21.15.76:0/2563928478 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0a5008a330 con 0x7f0a5c0cb280 2024-03-08T06:01:31.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.544+0000 7f0a5b7fe640 1 --2- 172.21.15.76:0/2563928478 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0a4806b500 0x7f0a4806d9c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c001830 tx=0x7f0a4c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:31.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.548+0000 7f0a43fff640 1 -- 172.21.15.76:0/2563928478 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a5c1f7e60 con 0x7f0a5c0cb280 2024-03-08T06:01:31.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.718+0000 7f0a61f0d640 1 -- 172.21.15.76:0/2563928478 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a5c04f030 con 0x7f0a5c0cb280 2024-03-08T06:01:31.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.718+0000 7f0a43fff640 1 -- 172.21.15.76:0/2563928478 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a500587f0 con 0x7f0a5c0cb280 2024-03-08T06:01:31.719 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:01:31.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.720+0000 7f0a61f0d640 1 -- 172.21.15.76:0/2563928478 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0a4806b500 msgr2=0x7f0a4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.720+0000 7f0a61f0d640 1 --2- 172.21.15.76:0/2563928478 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0a4806b500 0x7f0a4806d9c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c001830 tx=0x7f0a4c030000 comp rx=0 tx=0).stop 2024-03-08T06:01:31.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.720+0000 7f0a61f0d640 1 -- 172.21.15.76:0/2563928478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0cb280 msgr2=0x7f0a5c1f7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.720+0000 7f0a61f0d640 1 --2- 172.21.15.76:0/2563928478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0cb280 0x7f0a5c1f7510 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0a50004f70 tx=0x7f0a500074f0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.720+0000 7f0a61f0d640 1 -- 172.21.15.76:0/2563928478 shutdown_connections 2024-03-08T06:01:31.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.720+0000 7f0a61f0d640 1 --2- 172.21.15.76:0/2563928478 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0a4806b500 0x7f0a4806d9c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.720+0000 7f0a61f0d640 1 --2- 172.21.15.76:0/2563928478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a5c1f7a50 0x7f0a5c159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.720+0000 7f0a61f0d640 1 --2- 172.21.15.76:0/2563928478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0cb280 0x7f0a5c1f7510 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.720+0000 7f0a61f0d640 1 --2- 172.21.15.76:0/2563928478 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a44003a20 0x7f0a5c0c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.720+0000 7f0a61f0d640 1 -- 172.21.15.76:0/2563928478 >> 172.21.15.76:0/2563928478 conn(0x7f0a5c06ae60 msgr2=0x7f0a5c06c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:31.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.720+0000 7f0a61f0d640 1 -- 172.21.15.76:0/2563928478 shutdown_connections 2024-03-08T06:01:31.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.720+0000 7f0a61f0d640 1 -- 172.21.15.76:0/2563928478 wait complete. 2024-03-08T06:01:31.732 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:01:31.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.834+0000 7fd4deb3e640 1 Processor -- start 2024-03-08T06:01:31.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.834+0000 7fd4deb3e640 1 -- start start 2024-03-08T06:01:31.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.835+0000 7fd4deb3e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd4d815f5c0 0x7fd4d8071890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.835+0000 7fd4deb3e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd4d815c600 0x7fd4d815ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.835+0000 7fd4deb3e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d815fc10 con 0x7fd4d8159770 2024-03-08T06:01:31.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.835+0000 7fd4deb3e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d8071dd0 con 0x7fd4d815c600 2024-03-08T06:01:31.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.835+0000 7fd4deb3e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d815c170 con 0x7fd4d815f5c0 2024-03-08T06:01:31.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.835+0000 7fd4dd0db640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd4d815c600 0x7fd4d815ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.835+0000 7fd4dc8da640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd4d8159770 0x7fd4d80cf4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46748/0 (socket says 172.21.15.76:46748) 2024-03-08T06:01:31.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.835+0000 7fd4dc8da640 1 -- 172.21.15.76:0/3888616304 learned_addr learned my addr 172.21.15.76:0/3888616304 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:31.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.836+0000 7fd4d7fff640 1 --2- 172.21.15.76:0/3888616304 >> v2:172.21.15.81:3300/0 conn(0x7fd4d815f5c0 0x7fd4d8071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.836+0000 7fd4dd0db640 1 -- 172.21.15.76:0/3888616304 >> v2:172.21.15.81:3300/0 conn(0x7fd4d815f5c0 msgr2=0x7fd4d8071890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:31.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.836+0000 7fd4dd0db640 1 --2- 172.21.15.76:0/3888616304 >> v2:172.21.15.81:3300/0 conn(0x7fd4d815f5c0 0x7fd4d8071890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.836+0000 7fd4dd0db640 1 -- 172.21.15.76:0/3888616304 >> v1:172.21.15.76:6789/0 conn(0x7fd4d8159770 legacy=0x7fd4d80cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.836+0000 7fd4dd0db640 1 -- 172.21.15.76:0/3888616304 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d815fef0 con 0x7fd4d815c600 2024-03-08T06:01:31.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.836+0000 7fd4d77fe640 1 -- 172.21.15.76:0/3888616304 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728990460 0 0) 0x7fd4d815fc10 con 0x7fd4d8159770 2024-03-08T06:01:31.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.836+0000 7fd4dd0db640 1 --2- 172.21.15.76:0/3888616304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d815c600 0x7fd4d815ca70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80b79c0 tx=0x7fd4d002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a5d2e128d8bfe7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:31.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.837+0000 7fd4d77fe640 1 -- 172.21.15.76:0/3888616304 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0039070 con 0x7fd4d815c600 2024-03-08T06:01:31.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.837+0000 7fd4d77fe640 1 -- 172.21.15.76:0/3888616304 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d002cc80 con 0x7fd4d815c600 2024-03-08T06:01:31.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.837+0000 7fd4d77fe640 1 -- 172.21.15.76:0/3888616304 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0034660 con 0x7fd4d815c600 2024-03-08T06:01:31.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.837+0000 7fd4deb3e640 1 -- 172.21.15.76:0/3888616304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d815c600 msgr2=0x7fd4d815ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.837+0000 7fd4deb3e640 1 --2- 172.21.15.76:0/3888616304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d815c600 0x7fd4d815ca70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80b79c0 tx=0x7fd4d002bed0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.837+0000 7fd4deb3e640 1 -- 172.21.15.76:0/3888616304 shutdown_connections 2024-03-08T06:01:31.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.837+0000 7fd4deb3e640 1 --2- 172.21.15.76:0/3888616304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d815c600 0x7fd4d815ca70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.837+0000 7fd4deb3e640 1 --2- 172.21.15.76:0/3888616304 >> v2:172.21.15.81:3300/0 conn(0x7fd4d815f5c0 0x7fd4d8071890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.837+0000 7fd4deb3e640 1 -- 172.21.15.76:0/3888616304 >> 172.21.15.76:0/3888616304 conn(0x7fd4d806f530 msgr2=0x7fd4d806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:31.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.837+0000 7fd4deb3e640 1 -- 172.21.15.76:0/3888616304 shutdown_connections 2024-03-08T06:01:31.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.837+0000 7fd4deb3e640 1 -- 172.21.15.76:0/3888616304 wait complete. 2024-03-08T06:01:31.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.838+0000 7fd4deb3e640 1 Processor -- start 2024-03-08T06:01:31.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.838+0000 7fd4deb3e640 1 -- start start 2024-03-08T06:01:31.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.839+0000 7fd4deb3e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d8159770 0x7fd4d81f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.839+0000 7fd4deb3e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4d815c600 0x7fd4d81ee870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.839+0000 7fd4deb3e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4d815f5c0 0x7fd4d80c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.839+0000 7fd4deb3e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d80cfbb0 con 0x7fd4d8159770 2024-03-08T06:01:31.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.839+0000 7fd4deb3e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d81604b0 con 0x7fd4d815f5c0 2024-03-08T06:01:31.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.839+0000 7fd4deb3e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d80ce250 con 0x7fd4d815c600 2024-03-08T06:01:31.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.839+0000 7fd4dc8da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d8159770 0x7fd4d81f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.839+0000 7fd4dc8da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d8159770 0x7fd4d81f1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53314/0 (socket says 172.21.15.76:53314) 2024-03-08T06:01:31.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.839+0000 7fd4dc8da640 1 -- 172.21.15.76:0/449930920 learned_addr learned my addr 172.21.15.76:0/449930920 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:31.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.839+0000 7fd4dd0db640 1 --2- 172.21.15.76:0/449930920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4d815f5c0 0x7fd4d80c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.839+0000 7fd4d7fff640 1 --2- 172.21.15.76:0/449930920 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4d815c600 0x7fd4d81ee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.840+0000 7fd4dc8da640 1 -- 172.21.15.76:0/449930920 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4d815c600 msgr2=0x7fd4d81ee870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.840+0000 7fd4dc8da640 1 --2- 172.21.15.76:0/449930920 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4d815c600 0x7fd4d81ee870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.840+0000 7fd4dc8da640 1 -- 172.21.15.76:0/449930920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4d815f5c0 msgr2=0x7fd4d80c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:31.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.840+0000 7fd4dc8da640 1 --2- 172.21.15.76:0/449930920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4d815f5c0 0x7fd4d80c6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:31.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.840+0000 7fd4dc8da640 1 -- 172.21.15.76:0/449930920 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d81f4ce0 con 0x7fd4d8159770 2024-03-08T06:01:31.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.840+0000 7fd4d7fff640 1 --2- 172.21.15.76:0/449930920 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4d815c600 0x7fd4d81ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:31.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.840+0000 7fd4dc8da640 1 --2- 172.21.15.76:0/449930920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d8159770 0x7fd4d81f1e40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800c500 tx=0x7fd4c800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:31.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.840+0000 7fd4d57fa640 1 -- 172.21.15.76:0/449930920 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8018070 con 0x7fd4d8159770 2024-03-08T06:01:31.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.840+0000 7fd4d57fa640 1 -- 172.21.15.76:0/449930920 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c8007580 con 0x7fd4d8159770 2024-03-08T06:01:31.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.840+0000 7fd4deb3e640 1 -- 172.21.15.76:0/449930920 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d81f4fc0 con 0x7fd4d8159770 2024-03-08T06:01:31.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.840+0000 7fd4d57fa640 1 -- 172.21.15.76:0/449930920 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8004a20 con 0x7fd4d8159770 2024-03-08T06:01:31.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.840+0000 7fd4deb3e640 1 -- 172.21.15.76:0/449930920 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d81f5580 con 0x7fd4d8159770 2024-03-08T06:01:31.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.841+0000 7fd4deb3e640 1 -- 172.21.15.76:0/449930920 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d804f030 con 0x7fd4d8159770 2024-03-08T06:01:31.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.841+0000 7fd4d57fa640 1 -- 172.21.15.76:0/449930920 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4c801b070 con 0x7fd4d8159770 2024-03-08T06:01:31.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.842+0000 7fd4d57fa640 1 --2- 172.21.15.76:0/449930920 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd4c406b500 0x7fd4c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:31.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.842+0000 7fd4d57fa640 1 -- 172.21.15.76:0/449930920 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd4c80897b0 con 0x7fd4d8159770 2024-03-08T06:01:31.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.842+0000 7fd4d7fff640 1 --2- 172.21.15.76:0/449930920 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd4c406b500 0x7fd4c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:31.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.842+0000 7fd4d7fff640 1 --2- 172.21.15.76:0/449930920 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd4c406b500 0x7fd4c406d9c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80c8420 tx=0x7fd4cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:31.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:31.846+0000 7fd4d57fa640 1 -- 172.21.15.76:0/449930920 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4c8057c70 con 0x7fd4d8159770 2024-03-08T06:01:32.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.017+0000 7fd4deb3e640 1 -- 172.21.15.76:0/449930920 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd4d80703a0 con 0x7fd4d8159770 2024-03-08T06:01:32.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.018+0000 7fd4d57fa640 1 -- 172.21.15.76:0/449930920 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd4c805bd10 con 0x7fd4d8159770 2024-03-08T06:01:32.019 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:32.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.020+0000 7fd4deb3e640 1 -- 172.21.15.76:0/449930920 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd4c406b500 msgr2=0x7fd4c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.020+0000 7fd4deb3e640 1 --2- 172.21.15.76:0/449930920 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd4c406b500 0x7fd4c406d9c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80c8420 tx=0x7fd4cc008040 comp rx=0 tx=0).stop 2024-03-08T06:01:32.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.020+0000 7fd4deb3e640 1 -- 172.21.15.76:0/449930920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d8159770 msgr2=0x7fd4d81f1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.020+0000 7fd4deb3e640 1 --2- 172.21.15.76:0/449930920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d8159770 0x7fd4d81f1e40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800c500 tx=0x7fd4c800c9c0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.020+0000 7fd4deb3e640 1 -- 172.21.15.76:0/449930920 shutdown_connections 2024-03-08T06:01:32.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.020+0000 7fd4deb3e640 1 --2- 172.21.15.76:0/449930920 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd4c406b500 0x7fd4c406d9c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.020+0000 7fd4deb3e640 1 --2- 172.21.15.76:0/449930920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4d815f5c0 0x7fd4d80c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.020+0000 7fd4deb3e640 1 --2- 172.21.15.76:0/449930920 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4d815c600 0x7fd4d81ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.020+0000 7fd4deb3e640 1 --2- 172.21.15.76:0/449930920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d8159770 0x7fd4d81f1e40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.020+0000 7fd4deb3e640 1 -- 172.21.15.76:0/449930920 >> 172.21.15.76:0/449930920 conn(0x7fd4d806f530 msgr2=0x7fd4d8159bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:32.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.020+0000 7fd4deb3e640 1 -- 172.21.15.76:0/449930920 shutdown_connections 2024-03-08T06:01:32.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.020+0000 7fd4deb3e640 1 -- 172.21.15.76:0/449930920 wait complete. 2024-03-08T06:01:32.032 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:01:32.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.134+0000 7f007edc7640 1 Processor -- start 2024-03-08T06:01:32.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.134+0000 7f007edc7640 1 -- start start 2024-03-08T06:01:32.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.135+0000 7f007edc7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0078161210 0x7f00780cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.135+0000 7f007edc7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f007806d040 0x7f0078070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.135+0000 7f007edc7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f007806d4d0 0x7f00780714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.135+0000 7f007edc7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0078070470 con 0x7f007806d040 2024-03-08T06:01:32.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.135+0000 7f007edc7640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007806bfa0 con 0x7f0078161210 2024-03-08T06:01:32.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.135+0000 7f007edc7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007806c280 con 0x7f007806d4d0 2024-03-08T06:01:32.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.135+0000 7f0077fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f007806d040 0x7f0078070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.135+0000 7f0077fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f007806d040 0x7f0078070f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53330/0 (socket says 172.21.15.76:53330) 2024-03-08T06:01:32.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.135+0000 7f0077fff640 1 -- 172.21.15.76:0/614875355 learned_addr learned my addr 172.21.15.76:0/614875355 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:32.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.136+0000 7f007d364640 1 --2- 172.21.15.76:0/614875355 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f007806d4d0 0x7f00780714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.136+0000 7f007cb63640 1 --2- 172.21.15.76:0/614875355 >> v2:172.21.15.81:3300/0 conn(0x7f0078161210 0x7f00780cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.136+0000 7f0077fff640 1 -- 172.21.15.76:0/614875355 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f007806d4d0 msgr2=0x7f00780714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:32.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.136+0000 7f0077fff640 1 --2- 172.21.15.76:0/614875355 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f007806d4d0 0x7f00780714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.136+0000 7f0077fff640 1 -- 172.21.15.76:0/614875355 >> v2:172.21.15.81:3300/0 conn(0x7f0078161210 msgr2=0x7f00780cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:32.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.136+0000 7f0077fff640 1 --2- 172.21.15.76:0/614875355 >> v2:172.21.15.81:3300/0 conn(0x7f0078161210 0x7f00780cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.136+0000 7f0077fff640 1 -- 172.21.15.76:0/614875355 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007806c560 con 0x7f007806d040 2024-03-08T06:01:32.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.136+0000 7f007cb63640 1 --2- 172.21.15.76:0/614875355 >> v2:172.21.15.81:3300/0 conn(0x7f0078161210 0x7f00780cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:32.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.136+0000 7f0077fff640 1 --2- 172.21.15.76:0/614875355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f007806d040 0x7f0078070f60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f00780d18a0 tx=0x7f006c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b0913a735ba6d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:32.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.137+0000 7f00777fe640 1 -- 172.21.15.76:0/614875355 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c035020 con 0x7f007806d040 2024-03-08T06:01:32.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.137+0000 7f00777fe640 1 -- 172.21.15.76:0/614875355 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c031870 con 0x7f007806d040 2024-03-08T06:01:32.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.137+0000 7f00777fe640 1 -- 172.21.15.76:0/614875355 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c031c40 con 0x7f007806d040 2024-03-08T06:01:32.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.137+0000 7f007edc7640 1 -- 172.21.15.76:0/614875355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f007806d040 msgr2=0x7f0078070f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.137+0000 7f007edc7640 1 --2- 172.21.15.76:0/614875355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f007806d040 0x7f0078070f60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f00780d18a0 tx=0x7f006c02b910 comp rx=0 tx=0).stop 2024-03-08T06:01:32.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.137+0000 7f007edc7640 1 -- 172.21.15.76:0/614875355 shutdown_connections 2024-03-08T06:01:32.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.137+0000 7f007edc7640 1 --2- 172.21.15.76:0/614875355 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f007806d4d0 0x7f00780714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.137+0000 7f007edc7640 1 --2- 172.21.15.76:0/614875355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f007806d040 0x7f0078070f60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.137+0000 7f007edc7640 1 --2- 172.21.15.76:0/614875355 >> v2:172.21.15.81:3300/0 conn(0x7f0078161210 0x7f00780cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.137+0000 7f007edc7640 1 -- 172.21.15.76:0/614875355 >> 172.21.15.76:0/614875355 conn(0x7f007806f530 msgr2=0x7f007806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:32.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.137+0000 7f007edc7640 1 -- 172.21.15.76:0/614875355 shutdown_connections 2024-03-08T06:01:32.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.137+0000 7f007edc7640 1 -- 172.21.15.76:0/614875355 wait complete. 2024-03-08T06:01:32.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.138+0000 7f007edc7640 1 Processor -- start 2024-03-08T06:01:32.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.138+0000 7f007edc7640 1 -- start start 2024-03-08T06:01:32.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.139+0000 7f007edc7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f007806d040 0x7f00780b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.139+0000 7f007edc7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f007806d4d0 0x7f00781f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.139+0000 7f007edc7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0078161210 0x7f00781e8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.139+0000 7f007edc7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00781617d0 con 0x7f007806d040 2024-03-08T06:01:32.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.139+0000 7f007edc7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007806cb20 con 0x7f0078161210 2024-03-08T06:01:32.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.139+0000 7f007edc7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0078161950 con 0x7f007806d4d0 2024-03-08T06:01:32.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.139+0000 7f007cb63640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f007806d040 0x7f00780b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.139+0000 7f007cb63640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f007806d040 0x7f00780b6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53342/0 (socket says 172.21.15.76:53342) 2024-03-08T06:01:32.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.139+0000 7f007cb63640 1 -- 172.21.15.76:0/3894361140 learned_addr learned my addr 172.21.15.76:0/3894361140 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:32.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.140+0000 7f0077fff640 1 --2- 172.21.15.76:0/3894361140 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f007806d4d0 0x7f00781f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.140+0000 7f007d364640 1 --2- 172.21.15.76:0/3894361140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0078161210 0x7f00781e8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.140+0000 7f007cb63640 1 -- 172.21.15.76:0/3894361140 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f007806d4d0 msgr2=0x7f00781f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.140+0000 7f007cb63640 1 --2- 172.21.15.76:0/3894361140 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f007806d4d0 0x7f00781f4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.140+0000 7f007cb63640 1 -- 172.21.15.76:0/3894361140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0078161210 msgr2=0x7f00781e8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.140+0000 7f007cb63640 1 --2- 172.21.15.76:0/3894361140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0078161210 0x7f00781e8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.140+0000 7f007cb63640 1 -- 172.21.15.76:0/3894361140 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00780b5f10 con 0x7f007806d040 2024-03-08T06:01:32.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.140+0000 7f0077fff640 1 --2- 172.21.15.76:0/3894361140 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f007806d4d0 0x7f00781f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:32.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.140+0000 7f007d364640 1 --2- 172.21.15.76:0/3894361140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0078161210 0x7f00781e8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:32.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.140+0000 7f007cb63640 1 --2- 172.21.15.76:0/3894361140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f007806d040 0x7f00780b6cd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f006800c7f0 tx=0x7f006800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:32.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.140+0000 7f00757fa640 1 -- 172.21.15.76:0/3894361140 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068015070 con 0x7f007806d040 2024-03-08T06:01:32.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.140+0000 7f00757fa640 1 -- 172.21.15.76:0/3894361140 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0068007820 con 0x7f007806d040 2024-03-08T06:01:32.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.140+0000 7f007edc7640 1 -- 172.21.15.76:0/3894361140 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00780b61f0 con 0x7f007806d040 2024-03-08T06:01:32.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.140+0000 7f00757fa640 1 -- 172.21.15.76:0/3894361140 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068007bd0 con 0x7f007806d040 2024-03-08T06:01:32.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.140+0000 7f007edc7640 1 -- 172.21.15.76:0/3894361140 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00780b67b0 con 0x7f007806d040 2024-03-08T06:01:32.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.141+0000 7f007edc7640 1 -- 172.21.15.76:0/3894361140 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007804f030 con 0x7f007806d040 2024-03-08T06:01:32.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.141+0000 7f00757fa640 1 -- 172.21.15.76:0/3894361140 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0068018070 con 0x7f007806d040 2024-03-08T06:01:32.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.142+0000 7f00757fa640 1 --2- 172.21.15.76:0/3894361140 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f006406b4d0 0x7f006406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.142+0000 7f00757fa640 1 -- 172.21.15.76:0/3894361140 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f006808a6d0 con 0x7f007806d040 2024-03-08T06:01:32.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.142+0000 7f0077fff640 1 --2- 172.21.15.76:0/3894361140 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f006406b4d0 0x7f006406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.142+0000 7f0077fff640 1 --2- 172.21.15.76:0/3894361140 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f006406b4d0 0x7f006406d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f006c036000 tx=0x7f006c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:32.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.146+0000 7f00757fa640 1 -- 172.21.15.76:0/3894361140 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0068059150 con 0x7f007806d040 2024-03-08T06:01:32.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.321+0000 7f007edc7640 1 -- 172.21.15.76:0/3894361140 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f00780b7850 con 0x7f007806d040 2024-03-08T06:01:32.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.321+0000 7f00757fa640 1 -- 172.21.15.76:0/3894361140 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0068058a50 con 0x7f007806d040 2024-03-08T06:01:32.322 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:01:32.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.323+0000 7f007edc7640 1 -- 172.21.15.76:0/3894361140 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f006406b4d0 msgr2=0x7f006406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.323+0000 7f007edc7640 1 --2- 172.21.15.76:0/3894361140 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f006406b4d0 0x7f006406d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f006c036000 tx=0x7f006c008030 comp rx=0 tx=0).stop 2024-03-08T06:01:32.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.323+0000 7f007edc7640 1 -- 172.21.15.76:0/3894361140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f007806d040 msgr2=0x7f00780b6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.323+0000 7f007edc7640 1 --2- 172.21.15.76:0/3894361140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f007806d040 0x7f00780b6cd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f006800c7f0 tx=0x7f006800ccb0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.323+0000 7f007edc7640 1 -- 172.21.15.76:0/3894361140 shutdown_connections 2024-03-08T06:01:32.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.323+0000 7f007edc7640 1 --2- 172.21.15.76:0/3894361140 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f006406b4d0 0x7f006406d990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.323+0000 7f007edc7640 1 --2- 172.21.15.76:0/3894361140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0078161210 0x7f00781e8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.323+0000 7f007edc7640 1 --2- 172.21.15.76:0/3894361140 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f007806d4d0 0x7f00781f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.323+0000 7f007edc7640 1 --2- 172.21.15.76:0/3894361140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f007806d040 0x7f00780b6cd0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.323+0000 7f007edc7640 1 -- 172.21.15.76:0/3894361140 >> 172.21.15.76:0/3894361140 conn(0x7f007806f530 msgr2=0x7f00781e9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:32.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.323+0000 7f007edc7640 1 -- 172.21.15.76:0/3894361140 shutdown_connections 2024-03-08T06:01:32.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.323+0000 7f007edc7640 1 -- 172.21.15.76:0/3894361140 wait complete. 2024-03-08T06:01:32.335 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T06:01:32.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.438+0000 7f8777446640 1 Processor -- start 2024-03-08T06:01:32.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.439+0000 7f8777446640 1 -- start start 2024-03-08T06:01:32.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.439+0000 7f8777446640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f87700afde0 0x7f877006b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.439+0000 7f8777446640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87700cd580 con 0x7f877015dcb0 2024-03-08T06:01:32.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.439+0000 7f8777446640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87700cd860 con 0x7f87700af950 2024-03-08T06:01:32.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.439+0000 7f8777446640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87700cdb40 con 0x7f87700afde0 2024-03-08T06:01:32.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f87751e2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f877015dcb0 0x7f87700ad600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46750/0 (socket says 172.21.15.76:46750) 2024-03-08T06:01:32.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f87751e2640 1 -- 172.21.15.76:0/488328250 learned_addr learned my addr 172.21.15.76:0/488328250 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:32.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f8763fff640 1 -- 172.21.15.76:0/488328250 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078390887 0 0) 0x7f87700cd580 con 0x7f877015dcb0 2024-03-08T06:01:32.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f8763fff640 1 -- 172.21.15.76:0/488328250 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f875c003520 con 0x7f877015dcb0 2024-03-08T06:01:32.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f87759e3640 1 --2- 172.21.15.76:0/488328250 >> v2:172.21.15.81:3300/0 conn(0x7f87700afde0 0x7f877006b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f8763fff640 1 -- 172.21.15.76:0/488328250 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3424384862 0 0) 0x7f875c003520 con 0x7f877015dcb0 2024-03-08T06:01:32.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f8763fff640 1 -- 172.21.15.76:0/488328250 >> v2:172.21.15.81:3300/0 conn(0x7f87700afde0 msgr2=0x7f877006b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:32.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f8763fff640 1 --2- 172.21.15.76:0/488328250 >> v2:172.21.15.81:3300/0 conn(0x7f87700afde0 0x7f877006b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f8763fff640 1 -- 172.21.15.76:0/488328250 >> v1:172.21.15.81:6789/0 conn(0x7f87700af950 legacy=0x7f877006fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f8763fff640 1 -- 172.21.15.76:0/488328250 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87700cde20 con 0x7f877015dcb0 2024-03-08T06:01:32.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f8763fff640 1 -- 172.21.15.76:0/488328250 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f8764001670 con 0x7f877015dcb0 2024-03-08T06:01:32.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f8763fff640 1 -- 172.21.15.76:0/488328250 >> v1:172.21.15.76:6789/0 conn(0x7f877015dcb0 legacy=0x7f87700ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f8763fff640 1 --2- 172.21.15.76:0/488328250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f875c003a20 0x7f875c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f8763fff640 1 --2- 172.21.15.76:0/488328250 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f875c004800 0x7f875c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f8763fff640 1 --2- 172.21.15.76:0/488328250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875c0051c0 0x7f875c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f8763fff640 1 -- 172.21.15.76:0/488328250 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87700cdb40 con 0x7f875c0051c0 2024-03-08T06:01:32.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f8763fff640 1 -- 172.21.15.76:0/488328250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c00c0f0 con 0x7f875c003a20 2024-03-08T06:01:32.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f8763fff640 1 -- 172.21.15.76:0/488328250 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875c00c2f0 con 0x7f875c004800 2024-03-08T06:01:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f87759e3640 1 --2- 172.21.15.76:0/488328250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875c0051c0 0x7f875c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f87751e2640 1 --2- 172.21.15.76:0/488328250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f875c003a20 0x7f875c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.440+0000 7f87749e1640 1 --2- 172.21.15.76:0/488328250 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f875c004800 0x7f875c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.441+0000 7f87759e3640 1 -- 172.21.15.76:0/488328250 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f875c004800 msgr2=0x7f875c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.441+0000 7f87759e3640 1 --2- 172.21.15.76:0/488328250 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f875c004800 0x7f875c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.441+0000 7f87759e3640 1 -- 172.21.15.76:0/488328250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f875c003a20 msgr2=0x7f875c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.441+0000 7f87759e3640 1 --2- 172.21.15.76:0/488328250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f875c003a20 0x7f875c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.441+0000 7f87759e3640 1 -- 172.21.15.76:0/488328250 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c003520 con 0x7f875c0051c0 2024-03-08T06:01:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.441+0000 7f87759e3640 1 -- 172.21.15.76:0/488328250 reap_dead start 2024-03-08T06:01:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.441+0000 7f87759e3640 1 --2- 172.21.15.76:0/488328250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875c0051c0 0x7f875c00b9c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8768009fc0 tx=0x7f875c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9582ce3c275ab63f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.441+0000 7f8763fff640 1 -- 172.21.15.76:0/488328250 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768009c10 con 0x7f875c0051c0 2024-03-08T06:01:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.441+0000 7f8763fff640 1 -- 172.21.15.76:0/488328250 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876802cca0 con 0x7f875c0051c0 2024-03-08T06:01:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.441+0000 7f8763fff640 1 -- 172.21.15.76:0/488328250 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768003bd0 con 0x7f875c0051c0 2024-03-08T06:01:32.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.442+0000 7f8777446640 1 -- 172.21.15.76:0/488328250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875c0051c0 msgr2=0x7f875c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.442+0000 7f8777446640 1 --2- 172.21.15.76:0/488328250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875c0051c0 0x7f875c00b9c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8768009fc0 tx=0x7f875c005830 comp rx=0 tx=0).stop 2024-03-08T06:01:32.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.442+0000 7f8777446640 1 -- 172.21.15.76:0/488328250 shutdown_connections 2024-03-08T06:01:32.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.442+0000 7f8777446640 1 --2- 172.21.15.76:0/488328250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875c0051c0 0x7f875c00b9c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.442+0000 7f8777446640 1 -- 172.21.15.76:0/488328250 >> 172.21.15.76:0/488328250 conn(0x7f87700b2320 msgr2=0x7f87700b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:32.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.442+0000 7f8777446640 1 -- 172.21.15.76:0/488328250 shutdown_connections 2024-03-08T06:01:32.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.442+0000 7f8777446640 1 -- 172.21.15.76:0/488328250 wait complete. 2024-03-08T06:01:32.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.443+0000 7f8777446640 1 Processor -- start 2024-03-08T06:01:32.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.443+0000 7f8777446640 1 -- start start 2024-03-08T06:01:32.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f8777446640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875c0051c0 0x7f877009d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f8777446640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f877009a710 0x7f87700b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f8777446640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770150760 0x7f8770150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f8777446640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877015e210 con 0x7f875c0051c0 2024-03-08T06:01:32.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f8777446640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877015e090 con 0x7f8770150760 2024-03-08T06:01:32.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f8777446640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87700ad600 con 0x7f877009a710 2024-03-08T06:01:32.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f87751e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875c0051c0 0x7f877009d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f87751e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875c0051c0 0x7f877009d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53352/0 (socket says 172.21.15.76:53352) 2024-03-08T06:01:32.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f87751e2640 1 -- 172.21.15.76:0/2457912726 learned_addr learned my addr 172.21.15.76:0/2457912726 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:32.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f87749e1640 1 --2- 172.21.15.76:0/2457912726 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f877009a710 0x7f87700b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f87759e3640 1 --2- 172.21.15.76:0/2457912726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770150760 0x7f8770150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f87751e2640 1 -- 172.21.15.76:0/2457912726 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f877009a710 msgr2=0x7f87700b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f87751e2640 1 --2- 172.21.15.76:0/2457912726 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f877009a710 0x7f87700b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f87751e2640 1 -- 172.21.15.76:0/2457912726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770150760 msgr2=0x7f8770150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f87751e2640 1 --2- 172.21.15.76:0/2457912726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770150760 0x7f8770150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f87751e2640 1 -- 172.21.15.76:0/2457912726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8770151120 con 0x7f875c0051c0 2024-03-08T06:01:32.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.444+0000 7f87759e3640 1 --2- 172.21.15.76:0/2457912726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770150760 0x7f8770150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:32.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.445+0000 7f87751e2640 1 --2- 172.21.15.76:0/2457912726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875c0051c0 0x7f877009d490 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8764007290 tx=0x7f8764007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:32.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.445+0000 7f8761ffb640 1 -- 172.21.15.76:0/2457912726 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764001640 con 0x7f875c0051c0 2024-03-08T06:01:32.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.445+0000 7f8761ffb640 1 -- 172.21.15.76:0/2457912726 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8764018070 con 0x7f875c0051c0 2024-03-08T06:01:32.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.445+0000 7f8777446640 1 -- 172.21.15.76:0/2457912726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877009db70 con 0x7f875c0051c0 2024-03-08T06:01:32.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.445+0000 7f8761ffb640 1 -- 172.21.15.76:0/2457912726 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876400fba0 con 0x7f875c0051c0 2024-03-08T06:01:32.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.445+0000 7f8777446640 1 -- 172.21.15.76:0/2457912726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877009e130 con 0x7f875c0051c0 2024-03-08T06:01:32.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.446+0000 7f8777446640 1 -- 172.21.15.76:0/2457912726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f877004f030 con 0x7f875c0051c0 2024-03-08T06:01:32.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.446+0000 7f8761ffb640 1 -- 172.21.15.76:0/2457912726 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f876400fd40 con 0x7f875c0051c0 2024-03-08T06:01:32.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.446+0000 7f8761ffb640 1 --2- 172.21.15.76:0/2457912726 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f875406b500 0x7f875406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.447+0000 7f8761ffb640 1 -- 172.21.15.76:0/2457912726 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f876408a560 con 0x7f875c0051c0 2024-03-08T06:01:32.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.447+0000 7f87749e1640 1 --2- 172.21.15.76:0/2457912726 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f875406b500 0x7f875406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.447+0000 7f87749e1640 1 --2- 172.21.15.76:0/2457912726 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f875406b500 0x7f875406d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f876c000c00 tx=0x7f876c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:32.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.451+0000 7f8761ffb640 1 -- 172.21.15.76:0/2457912726 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8764058f50 con 0x7f875c0051c0 2024-03-08T06:01:32.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.619+0000 7f8777446640 1 -- 172.21.15.76:0/2457912726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f87700ae360 con 0x7f875c0051c0 2024-03-08T06:01:32.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.619+0000 7f8761ffb640 1 -- 172.21.15.76:0/2457912726 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8764058850 con 0x7f875c0051c0 2024-03-08T06:01:32.620 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:32.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.621+0000 7f8777446640 1 -- 172.21.15.76:0/2457912726 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f875406b500 msgr2=0x7f875406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.621+0000 7f8777446640 1 --2- 172.21.15.76:0/2457912726 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f875406b500 0x7f875406d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f876c000c00 tx=0x7f876c009040 comp rx=0 tx=0).stop 2024-03-08T06:01:32.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.621+0000 7f8777446640 1 -- 172.21.15.76:0/2457912726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875c0051c0 msgr2=0x7f877009d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.621+0000 7f8777446640 1 --2- 172.21.15.76:0/2457912726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875c0051c0 0x7f877009d490 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8764007290 tx=0x7f8764007750 comp rx=0 tx=0).stop 2024-03-08T06:01:32.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.621+0000 7f8777446640 1 -- 172.21.15.76:0/2457912726 shutdown_connections 2024-03-08T06:01:32.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.621+0000 7f8777446640 1 --2- 172.21.15.76:0/2457912726 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f875406b500 0x7f875406d9c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.621+0000 7f8777446640 1 --2- 172.21.15.76:0/2457912726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770150760 0x7f8770150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.621+0000 7f8777446640 1 --2- 172.21.15.76:0/2457912726 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f877009a710 0x7f87700b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.621+0000 7f8777446640 1 --2- 172.21.15.76:0/2457912726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875c0051c0 0x7f877009d490 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.621+0000 7f8777446640 1 -- 172.21.15.76:0/2457912726 >> 172.21.15.76:0/2457912726 conn(0x7f87700b2320 msgr2=0x7f877006ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:32.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.622+0000 7f8777446640 1 -- 172.21.15.76:0/2457912726 shutdown_connections 2024-03-08T06:01:32.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.622+0000 7f8777446640 1 -- 172.21.15.76:0/2457912726 wait complete. 2024-03-08T06:01:32.633 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T06:01:32.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.733+0000 7fca799f4640 1 Processor -- start 2024-03-08T06:01:32.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.733+0000 7fca799f4640 1 -- start start 2024-03-08T06:01:32.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca799f4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fca7415dbc0 0x7fca740d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca799f4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fca740b9840 0x7fca740b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca799f4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fca740b5b20 0x7fca7406c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca799f4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca740b5f90 con 0x7fca740b9840 2024-03-08T06:01:32.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca799f4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca740d27d0 con 0x7fca7415dbc0 2024-03-08T06:01:32.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca799f4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca740d2ab0 con 0x7fca740b5b20 2024-03-08T06:01:32.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca727fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fca740b9840 0x7fca740b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca727fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fca740b9840 0x7fca740b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53364/0 (socket says 172.21.15.76:53364) 2024-03-08T06:01:32.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca727fc640 1 -- 172.21.15.76:0/916881374 learned_addr learned my addr 172.21.15.76:0/916881374 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:32.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca737fe640 1 --2- 172.21.15.76:0/916881374 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fca740b5b20 0x7fca7406c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca72ffd640 1 --2- 172.21.15.76:0/916881374 >> v2:172.21.15.81:3300/0 conn(0x7fca7415dbc0 0x7fca740d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca727fc640 1 -- 172.21.15.76:0/916881374 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fca740b5b20 msgr2=0x7fca7406c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:32.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca727fc640 1 --2- 172.21.15.76:0/916881374 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fca740b5b20 0x7fca7406c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca727fc640 1 -- 172.21.15.76:0/916881374 >> v2:172.21.15.81:3300/0 conn(0x7fca7415dbc0 msgr2=0x7fca740d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:32.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca727fc640 1 --2- 172.21.15.76:0/916881374 >> v2:172.21.15.81:3300/0 conn(0x7fca7415dbc0 0x7fca740d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca727fc640 1 -- 172.21.15.76:0/916881374 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca740d2d90 con 0x7fca740b9840 2024-03-08T06:01:32.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca737fe640 1 --2- 172.21.15.76:0/916881374 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fca740b5b20 0x7fca7406c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:32.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.734+0000 7fca72ffd640 1 --2- 172.21.15.76:0/916881374 >> v2:172.21.15.81:3300/0 conn(0x7fca7415dbc0 0x7fca740d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:32.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.735+0000 7fca727fc640 1 --2- 172.21.15.76:0/916881374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca740b9840 0x7fca740b55e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fca7406d280 tx=0x7fca5c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9aa1afb96fd3a6c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:32.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.735+0000 7fca71ffb640 1 -- 172.21.15.76:0/916881374 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c035020 con 0x7fca740b9840 2024-03-08T06:01:32.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.735+0000 7fca71ffb640 1 -- 172.21.15.76:0/916881374 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5c031870 con 0x7fca740b9840 2024-03-08T06:01:32.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.735+0000 7fca71ffb640 1 -- 172.21.15.76:0/916881374 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c031c40 con 0x7fca740b9840 2024-03-08T06:01:32.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.736+0000 7fca799f4640 1 -- 172.21.15.76:0/916881374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca740b9840 msgr2=0x7fca740b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.736+0000 7fca799f4640 1 --2- 172.21.15.76:0/916881374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca740b9840 0x7fca740b55e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fca7406d280 tx=0x7fca5c02b910 comp rx=0 tx=0).stop 2024-03-08T06:01:32.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.736+0000 7fca799f4640 1 -- 172.21.15.76:0/916881374 shutdown_connections 2024-03-08T06:01:32.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.736+0000 7fca799f4640 1 --2- 172.21.15.76:0/916881374 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fca740b5b20 0x7fca7406c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.736+0000 7fca799f4640 1 --2- 172.21.15.76:0/916881374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca740b9840 0x7fca740b55e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.736+0000 7fca799f4640 1 --2- 172.21.15.76:0/916881374 >> v2:172.21.15.81:3300/0 conn(0x7fca7415dbc0 0x7fca740d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.736+0000 7fca799f4640 1 -- 172.21.15.76:0/916881374 >> 172.21.15.76:0/916881374 conn(0x7fca740670d0 msgr2=0x7fca740d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:32.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.736+0000 7fca799f4640 1 -- 172.21.15.76:0/916881374 shutdown_connections 2024-03-08T06:01:32.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.736+0000 7fca799f4640 1 -- 172.21.15.76:0/916881374 wait complete. 2024-03-08T06:01:32.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.737+0000 7fca799f4640 1 Processor -- start 2024-03-08T06:01:32.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.737+0000 7fca799f4640 1 -- start start 2024-03-08T06:01:32.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.738+0000 7fca799f4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca740b5b20 0x7fca740cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.738+0000 7fca799f4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca740b9840 0x7fca741f7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.738+0000 7fca799f4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fca7415dbc0 0x7fca741e59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.738+0000 7fca799f4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca740d2430 con 0x7fca740b5b20 2024-03-08T06:01:32.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.738+0000 7fca799f4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca740d3870 con 0x7fca740b9840 2024-03-08T06:01:32.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.738+0000 7fca799f4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca740d25b0 con 0x7fca7415dbc0 2024-03-08T06:01:32.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.738+0000 7fca72ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca740b5b20 0x7fca740cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.738+0000 7fca72ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca740b5b20 0x7fca740cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53376/0 (socket says 172.21.15.76:53376) 2024-03-08T06:01:32.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.738+0000 7fca72ffd640 1 -- 172.21.15.76:0/510817972 learned_addr learned my addr 172.21.15.76:0/510817972 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:32.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.738+0000 7fca727fc640 1 --2- 172.21.15.76:0/510817972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca740b9840 0x7fca741f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.738+0000 7fca737fe640 1 --2- 172.21.15.76:0/510817972 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fca7415dbc0 0x7fca741e59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.738+0000 7fca72ffd640 1 -- 172.21.15.76:0/510817972 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fca7415dbc0 msgr2=0x7fca741e59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.738+0000 7fca72ffd640 1 --2- 172.21.15.76:0/510817972 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fca7415dbc0 0x7fca741e59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.738+0000 7fca72ffd640 1 -- 172.21.15.76:0/510817972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca740b9840 msgr2=0x7fca741f7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.738+0000 7fca72ffd640 1 --2- 172.21.15.76:0/510817972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca740b9840 0x7fca741f7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.738+0000 7fca72ffd640 1 -- 172.21.15.76:0/510817972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca741f7bd0 con 0x7fca740b5b20 2024-03-08T06:01:32.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.739+0000 7fca72ffd640 1 --2- 172.21.15.76:0/510817972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca740b5b20 0x7fca740cc530 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00ce00 tx=0x7fca6c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:32.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.739+0000 7fca6bfff640 1 -- 172.21.15.76:0/510817972 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c00f890 con 0x7fca740b5b20 2024-03-08T06:01:32.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.739+0000 7fca6bfff640 1 -- 172.21.15.76:0/510817972 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6c00fa30 con 0x7fca740b5b20 2024-03-08T06:01:32.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.739+0000 7fca6bfff640 1 -- 172.21.15.76:0/510817972 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c00fe40 con 0x7fca740b5b20 2024-03-08T06:01:32.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.739+0000 7fca799f4640 1 -- 172.21.15.76:0/510817972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca740cb490 con 0x7fca740b5b20 2024-03-08T06:01:32.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.739+0000 7fca799f4640 1 -- 172.21.15.76:0/510817972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca740cba50 con 0x7fca740b5b20 2024-03-08T06:01:32.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.740+0000 7fca799f4640 1 -- 172.21.15.76:0/510817972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7404f030 con 0x7fca740b5b20 2024-03-08T06:01:32.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.740+0000 7fca6bfff640 1 -- 172.21.15.76:0/510817972 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca6c018400 con 0x7fca740b5b20 2024-03-08T06:01:32.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.740+0000 7fca6bfff640 1 --2- 172.21.15.76:0/510817972 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fca5406b4d0 0x7fca5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:32.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.741+0000 7fca6bfff640 1 -- 172.21.15.76:0/510817972 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fca6c08b310 con 0x7fca740b5b20 2024-03-08T06:01:32.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.741+0000 7fca727fc640 1 --2- 172.21.15.76:0/510817972 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fca5406b4d0 0x7fca5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:32.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.741+0000 7fca727fc640 1 --2- 172.21.15.76:0/510817972 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fca5406b4d0 0x7fca5406d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca5c036000 tx=0x7fca5c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:32.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.745+0000 7fca6bfff640 1 -- 172.21.15.76:0/510817972 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca741f7bd0 con 0x7fca740b5b20 2024-03-08T06:01:32.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.925+0000 7fca799f4640 1 -- 172.21.15.76:0/510817972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fca740c9fc0 con 0x7fca740b5b20 2024-03-08T06:01:32.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.926+0000 7fca6bfff640 1 -- 172.21.15.76:0/510817972 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fca6c014030 con 0x7fca740b5b20 2024-03-08T06:01:32.926 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-03-08T06:01:32.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.928+0000 7fca799f4640 1 -- 172.21.15.76:0/510817972 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fca5406b4d0 msgr2=0x7fca5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.928+0000 7fca799f4640 1 --2- 172.21.15.76:0/510817972 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fca5406b4d0 0x7fca5406d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca5c036000 tx=0x7fca5c008030 comp rx=0 tx=0).stop 2024-03-08T06:01:32.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.928+0000 7fca799f4640 1 -- 172.21.15.76:0/510817972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca740b5b20 msgr2=0x7fca740cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:32.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.928+0000 7fca799f4640 1 --2- 172.21.15.76:0/510817972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca740b5b20 0x7fca740cc530 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00ce00 tx=0x7fca6c00a650 comp rx=0 tx=0).stop 2024-03-08T06:01:32.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.928+0000 7fca799f4640 1 -- 172.21.15.76:0/510817972 shutdown_connections 2024-03-08T06:01:32.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.928+0000 7fca799f4640 1 --2- 172.21.15.76:0/510817972 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fca5406b4d0 0x7fca5406d990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.928+0000 7fca799f4640 1 --2- 172.21.15.76:0/510817972 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fca7415dbc0 0x7fca741e59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.928+0000 7fca799f4640 1 --2- 172.21.15.76:0/510817972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca740b9840 0x7fca741f7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.928+0000 7fca799f4640 1 --2- 172.21.15.76:0/510817972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca740b5b20 0x7fca740cc530 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:32.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.928+0000 7fca799f4640 1 -- 172.21.15.76:0/510817972 >> 172.21.15.76:0/510817972 conn(0x7fca740670d0 msgr2=0x7fca7415c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:32.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.928+0000 7fca799f4640 1 -- 172.21.15.76:0/510817972 shutdown_connections 2024-03-08T06:01:32.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:32.928+0000 7fca799f4640 1 -- 172.21.15.76:0/510817972 wait complete. 2024-03-08T06:01:32.939 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-03-08T06:01:32.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:32.939 DEBUG:teuthology.orchestra.run.smithi081:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-03-08T06:01:32.973 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-03-08T06:01:32.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:32.974 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-08T06:01:33.031 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-08T06:01:33.031 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:01:33.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.131+0000 7fd47a963640 1 Processor -- start 2024-03-08T06:01:33.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.131+0000 7fd47a963640 1 -- start start 2024-03-08T06:01:33.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.132+0000 7fd47a963640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd47415dc60 0x7fd4740d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.132+0000 7fd47a963640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd47409a250 0x7fd4740cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.132+0000 7fd47a963640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4740cfcc0 con 0x7fd47415e040 2024-03-08T06:01:33.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.132+0000 7fd47a963640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4740b4360 con 0x7fd47409a250 2024-03-08T06:01:33.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.132+0000 7fd47a963640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4740cff60 con 0x7fd47415dc60 2024-03-08T06:01:33.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.132+0000 7fd478f00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd47409a250 0x7fd4740cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.132+0000 7fd4737fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd47415e040 0x7fd474099b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46766/0 (socket says 172.21.15.76:46766) 2024-03-08T06:01:33.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.132+0000 7fd4737fe640 1 -- 172.21.15.76:0/382772208 learned_addr learned my addr 172.21.15.76:0/382772208 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:33.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.132+0000 7fd478f00640 1 -- 172.21.15.76:0/382772208 >> v2:172.21.15.81:3300/0 conn(0x7fd47415dc60 msgr2=0x7fd4740d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:33.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.132+0000 7fd478f00640 1 --2- 172.21.15.76:0/382772208 >> v2:172.21.15.81:3300/0 conn(0x7fd47415dc60 0x7fd4740d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.132+0000 7fd478f00640 1 -- 172.21.15.76:0/382772208 >> v1:172.21.15.76:6789/0 conn(0x7fd47415e040 legacy=0x7fd474099b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.132+0000 7fd478f00640 1 -- 172.21.15.76:0/382772208 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4740d0240 con 0x7fd47409a250 2024-03-08T06:01:33.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.132+0000 7fd472ffd640 1 -- 172.21.15.76:0/382772208 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261851916 0 0) 0x7fd4740cfcc0 con 0x7fd47415e040 2024-03-08T06:01:33.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.133+0000 7fd478f00640 1 --2- 172.21.15.76:0/382772208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd47409a250 0x7fd4740cbfa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd474072ac0 tx=0x7fd46c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f005a3aa44e590 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:33.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.133+0000 7fd472ffd640 1 -- 172.21.15.76:0/382772208 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c039070 con 0x7fd47409a250 2024-03-08T06:01:33.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.133+0000 7fd472ffd640 1 -- 172.21.15.76:0/382772208 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c02cd00 con 0x7fd47409a250 2024-03-08T06:01:33.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.133+0000 7fd472ffd640 1 -- 172.21.15.76:0/382772208 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c034660 con 0x7fd47409a250 2024-03-08T06:01:33.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.133+0000 7fd47a963640 1 -- 172.21.15.76:0/382772208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd47409a250 msgr2=0x7fd4740cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.133+0000 7fd47a963640 1 --2- 172.21.15.76:0/382772208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd47409a250 0x7fd4740cbfa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd474072ac0 tx=0x7fd46c02bf50 comp rx=0 tx=0).stop 2024-03-08T06:01:33.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.134+0000 7fd47a963640 1 -- 172.21.15.76:0/382772208 shutdown_connections 2024-03-08T06:01:33.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.134+0000 7fd47a963640 1 --2- 172.21.15.76:0/382772208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd47409a250 0x7fd4740cbfa0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.134+0000 7fd47a963640 1 --2- 172.21.15.76:0/382772208 >> v2:172.21.15.81:3300/0 conn(0x7fd47415dc60 0x7fd4740d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.134+0000 7fd47a963640 1 -- 172.21.15.76:0/382772208 >> 172.21.15.76:0/382772208 conn(0x7fd4740b3440 msgr2=0x7fd4740b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:33.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.134+0000 7fd47a963640 1 -- 172.21.15.76:0/382772208 shutdown_connections 2024-03-08T06:01:33.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.134+0000 7fd47a963640 1 -- 172.21.15.76:0/382772208 wait complete. 2024-03-08T06:01:33.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.135+0000 7fd47a963640 1 Processor -- start 2024-03-08T06:01:33.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.135+0000 7fd47a963640 1 -- start start 2024-03-08T06:01:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd47a963640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd47409a250 0x7fd47415eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd47a963640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd47415dc60 0x7fd4740c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd47a963640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd47415e040 0x7fd474156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd47a963640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4740d0ae0 con 0x7fd47415dc60 2024-03-08T06:01:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd47a963640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4740d0800 con 0x7fd47409a250 2024-03-08T06:01:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd47a963640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4740d1540 con 0x7fd47415e040 2024-03-08T06:01:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd4737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd47415dc60 0x7fd4740c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd4737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd47415dc60 0x7fd4740c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53394/0 (socket says 172.21.15.76:53394) 2024-03-08T06:01:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd4737fe640 1 -- 172.21.15.76:0/4288720385 learned_addr learned my addr 172.21.15.76:0/4288720385 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd473fff640 1 --2- 172.21.15.76:0/4288720385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd47409a250 0x7fd47415eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd478f00640 1 --2- 172.21.15.76:0/4288720385 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd47415e040 0x7fd474156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd4737fe640 1 -- 172.21.15.76:0/4288720385 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd47415e040 msgr2=0x7fd474156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd4737fe640 1 --2- 172.21.15.76:0/4288720385 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd47415e040 0x7fd474156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd4737fe640 1 -- 172.21.15.76:0/4288720385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd47409a250 msgr2=0x7fd47415eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd4737fe640 1 --2- 172.21.15.76:0/4288720385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd47409a250 0x7fd47415eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd4737fe640 1 -- 172.21.15.76:0/4288720385 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4740c9390 con 0x7fd47415dc60 2024-03-08T06:01:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd473fff640 1 --2- 172.21.15.76:0/4288720385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd47409a250 0x7fd47415eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd478f00640 1 --2- 172.21.15.76:0/4288720385 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd47415e040 0x7fd474156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd4737fe640 1 --2- 172.21.15.76:0/4288720385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd47415dc60 0x7fd4740c8a20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd46800c430 tx=0x7fd46800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd470ff9640 1 -- 172.21.15.76:0/4288720385 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4680074e0 con 0x7fd47415dc60 2024-03-08T06:01:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd470ff9640 1 -- 172.21.15.76:0/4288720385 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4680047e0 con 0x7fd47415dc60 2024-03-08T06:01:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd470ff9640 1 -- 172.21.15.76:0/4288720385 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468004bd0 con 0x7fd47415dc60 2024-03-08T06:01:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd47a963640 1 -- 172.21.15.76:0/4288720385 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4740c9670 con 0x7fd47415dc60 2024-03-08T06:01:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.136+0000 7fd47a963640 1 -- 172.21.15.76:0/4288720385 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47415f400 con 0x7fd47415dc60 2024-03-08T06:01:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.137+0000 7fd47a963640 1 -- 172.21.15.76:0/4288720385 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd47404f030 con 0x7fd47415dc60 2024-03-08T06:01:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.138+0000 7fd470ff9640 1 -- 172.21.15.76:0/4288720385 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd468011070 con 0x7fd47415dc60 2024-03-08T06:01:33.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.138+0000 7fd470ff9640 1 --2- 172.21.15.76:0/4288720385 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd46006b4d0 0x7fd46006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.138+0000 7fd470ff9640 1 -- 172.21.15.76:0/4288720385 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd46808a230 con 0x7fd47415dc60 2024-03-08T06:01:33.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.138+0000 7fd473fff640 1 --2- 172.21.15.76:0/4288720385 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd46006b4d0 0x7fd46006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.138+0000 7fd473fff640 1 --2- 172.21.15.76:0/4288720385 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd46006b4d0 0x7fd46006d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd464006fd0 tx=0x7fd464008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:33.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.142+0000 7fd470ff9640 1 -- 172.21.15.76:0/4288720385 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd468058d40 con 0x7fd47415dc60 2024-03-08T06:01:33.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.347+0000 7fd47a963640 1 -- 172.21.15.76:0/4288720385 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd47409a670 con 0x7fd47415dc60 2024-03-08T06:01:33.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.348+0000 7fd470ff9640 1 -- 172.21.15.76:0/4288720385 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7fd468058640 con 0x7fd47415dc60 2024-03-08T06:01:33.348 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:33.349 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4249563566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4249563566},{"type":"v1","addr":"172.21.15.76:6835","nonce":4249563566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1514641617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1514641617},{"type":"v1","addr":"172.21.15.76:6837","nonce":1514641617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6834/1332442802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1332442802},{"type":"v1","addr":"172.21.15.81:6834","nonce":1332442802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:00:35.504231+0000","modified":"2024-03-08T06:00:57.594023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6835/2357927814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":2357927814},{"type":"v1","addr":"172.21.15.81:6835","nonce":2357927814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T06:01:33.349 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-03-08T06:01:33.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.350+0000 7fd47a963640 1 -- 172.21.15.76:0/4288720385 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd46006b4d0 msgr2=0x7fd46006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.350+0000 7fd47a963640 1 --2- 172.21.15.76:0/4288720385 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd46006b4d0 0x7fd46006d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd464006fd0 tx=0x7fd464008040 comp rx=0 tx=0).stop 2024-03-08T06:01:33.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.350+0000 7fd47a963640 1 -- 172.21.15.76:0/4288720385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd47415dc60 msgr2=0x7fd4740c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.350+0000 7fd47a963640 1 --2- 172.21.15.76:0/4288720385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd47415dc60 0x7fd4740c8a20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd46800c430 tx=0x7fd46800c8f0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.350+0000 7fd47a963640 1 -- 172.21.15.76:0/4288720385 shutdown_connections 2024-03-08T06:01:33.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.350+0000 7fd47a963640 1 --2- 172.21.15.76:0/4288720385 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd46006b4d0 0x7fd46006d990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.350+0000 7fd47a963640 1 --2- 172.21.15.76:0/4288720385 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd47415e040 0x7fd474156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.350+0000 7fd47a963640 1 --2- 172.21.15.76:0/4288720385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd47415dc60 0x7fd4740c8a20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.350+0000 7fd47a963640 1 --2- 172.21.15.76:0/4288720385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd47409a250 0x7fd47415eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.350+0000 7fd47a963640 1 -- 172.21.15.76:0/4288720385 >> 172.21.15.76:0/4288720385 conn(0x7fd4740b3440 msgr2=0x7fd47415cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:33.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.350+0000 7fd47a963640 1 -- 172.21.15.76:0/4288720385 shutdown_connections 2024-03-08T06:01:33.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.350+0000 7fd47a963640 1 -- 172.21.15.76:0/4288720385 wait complete. 2024-03-08T06:01:33.362 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:33.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.461+0000 7f3dafbb5640 1 Processor -- start 2024-03-08T06:01:33.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.462+0000 7f3dafbb5640 1 -- start start 2024-03-08T06:01:33.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.462+0000 7f3dafbb5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3da8146b20 0x7f3da80aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.462+0000 7f3dafbb5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3da8147760 0x7f3da806f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.462+0000 7f3dafbb5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da806e730 con 0x7f3da80af020 2024-03-08T06:01:33.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.462+0000 7f3dafbb5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da806e9b0 con 0x7f3da8146b20 2024-03-08T06:01:33.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.463+0000 7f3dafbb5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da806ec90 con 0x7f3da8147760 2024-03-08T06:01:33.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.463+0000 7f3dad951640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3da8146b20 0x7f3da80aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.463+0000 7f3dad150640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3da80af020 0x7f3da80b2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46768/0 (socket says 172.21.15.76:46768) 2024-03-08T06:01:33.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.463+0000 7f3dad150640 1 -- 172.21.15.76:0/55364834 learned_addr learned my addr 172.21.15.76:0/55364834 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:33.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.463+0000 7f3dae152640 1 --2- 172.21.15.76:0/55364834 >> v2:172.21.15.81:3300/0 conn(0x7f3da8147760 0x7f3da806f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.463+0000 7f3dad951640 1 -- 172.21.15.76:0/55364834 >> v2:172.21.15.81:3300/0 conn(0x7f3da8147760 msgr2=0x7f3da806f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:33.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.463+0000 7f3dad951640 1 --2- 172.21.15.76:0/55364834 >> v2:172.21.15.81:3300/0 conn(0x7f3da8147760 0x7f3da806f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.463+0000 7f3dad951640 1 -- 172.21.15.76:0/55364834 >> v1:172.21.15.76:6789/0 conn(0x7f3da80af020 legacy=0x7f3da80b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.463+0000 7f3dad951640 1 -- 172.21.15.76:0/55364834 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da80ae2e0 con 0x7f3da8146b20 2024-03-08T06:01:33.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.463+0000 7f3dac94f640 1 -- 172.21.15.76:0/55364834 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854362168 0 0) 0x7f3d9c001110 con 0x7f3da80af020 2024-03-08T06:01:33.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.464+0000 7f3dad951640 1 --2- 172.21.15.76:0/55364834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da8146b20 0x7f3da80aeae0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3da815e290 tx=0x7f3da402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d81d94b39bc3b4f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:33.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.464+0000 7f3dac94f640 1 -- 172.21.15.76:0/55364834 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4035020 con 0x7f3da8146b20 2024-03-08T06:01:33.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.464+0000 7f3dac94f640 1 -- 172.21.15.76:0/55364834 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da4031830 con 0x7f3da8146b20 2024-03-08T06:01:33.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.464+0000 7f3dac94f640 1 -- 172.21.15.76:0/55364834 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4031c00 con 0x7f3da8146b20 2024-03-08T06:01:33.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.464+0000 7f3dafbb5640 1 -- 172.21.15.76:0/55364834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da8146b20 msgr2=0x7f3da80aeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.464+0000 7f3dafbb5640 1 --2- 172.21.15.76:0/55364834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da8146b20 0x7f3da80aeae0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3da815e290 tx=0x7f3da402b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.464+0000 7f3dafbb5640 1 -- 172.21.15.76:0/55364834 shutdown_connections 2024-03-08T06:01:33.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.464+0000 7f3dafbb5640 1 --2- 172.21.15.76:0/55364834 >> v2:172.21.15.81:3300/0 conn(0x7f3da8147760 0x7f3da806f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.464+0000 7f3dafbb5640 1 --2- 172.21.15.76:0/55364834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da8146b20 0x7f3da80aeae0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.464+0000 7f3dafbb5640 1 -- 172.21.15.76:0/55364834 >> 172.21.15.76:0/55364834 conn(0x7f3da80b1aa0 msgr2=0x7f3da80b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:33.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.464+0000 7f3dafbb5640 1 -- 172.21.15.76:0/55364834 shutdown_connections 2024-03-08T06:01:33.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.464+0000 7f3dafbb5640 1 -- 172.21.15.76:0/55364834 wait complete. 2024-03-08T06:01:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.466+0000 7f3dafbb5640 1 Processor -- start 2024-03-08T06:01:33.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.466+0000 7f3dafbb5640 1 -- start start 2024-03-08T06:01:33.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.466+0000 7f3dafbb5640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3da80af020 0x7f3da815d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.466+0000 7f3dafbb5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da8146b20 0x7f3da8130d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.466+0000 7f3dafbb5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3da8147760 0x7f3da80c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.466+0000 7f3dafbb5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80adf00 con 0x7f3da8146b20 2024-03-08T06:01:33.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.466+0000 7f3dafbb5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da8146f00 con 0x7f3da8147760 2024-03-08T06:01:33.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.466+0000 7f3dafbb5640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80ae080 con 0x7f3da80af020 2024-03-08T06:01:33.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.466+0000 7f3dad150640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da8146b20 0x7f3da8130d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.466+0000 7f3dad150640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da8146b20 0x7f3da8130d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53410/0 (socket says 172.21.15.76:53410) 2024-03-08T06:01:33.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.466+0000 7f3dad150640 1 -- 172.21.15.76:0/3552732578 learned_addr learned my addr 172.21.15.76:0/3552732578 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:33.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.467+0000 7f3dad951640 1 --2- 172.21.15.76:0/3552732578 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3da80af020 0x7f3da815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.467+0000 7f3dae152640 1 --2- 172.21.15.76:0/3552732578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3da8147760 0x7f3da80c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.467+0000 7f3dad150640 1 -- 172.21.15.76:0/3552732578 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3da80af020 msgr2=0x7f3da815d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.467+0000 7f3dad150640 1 --2- 172.21.15.76:0/3552732578 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3da80af020 0x7f3da815d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.467+0000 7f3dad150640 1 -- 172.21.15.76:0/3552732578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3da8147760 msgr2=0x7f3da80c4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.467+0000 7f3dad150640 1 --2- 172.21.15.76:0/3552732578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3da8147760 0x7f3da80c4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.467+0000 7f3dad150640 1 -- 172.21.15.76:0/3552732578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da8131550 con 0x7f3da8146b20 2024-03-08T06:01:33.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.467+0000 7f3dad951640 1 --2- 172.21.15.76:0/3552732578 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3da80af020 0x7f3da815d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:33.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.467+0000 7f3dad150640 1 --2- 172.21.15.76:0/3552732578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da8146b20 0x7f3da8130d00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00c780 tx=0x7f3d9c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:33.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.467+0000 7f3d9a7fc640 1 -- 172.21.15.76:0/3552732578 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c018070 con 0x7f3da8146b20 2024-03-08T06:01:33.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.467+0000 7f3d9a7fc640 1 -- 172.21.15.76:0/3552732578 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9c0072d0 con 0x7f3da8146b20 2024-03-08T06:01:33.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.467+0000 7f3dafbb5640 1 -- 172.21.15.76:0/3552732578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da8131830 con 0x7f3da8146b20 2024-03-08T06:01:33.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.467+0000 7f3d9a7fc640 1 -- 172.21.15.76:0/3552732578 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c0047e0 con 0x7f3da8146b20 2024-03-08T06:01:33.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.467+0000 7f3dafbb5640 1 -- 172.21.15.76:0/3552732578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da8131df0 con 0x7f3da8146b20 2024-03-08T06:01:33.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.468+0000 7f3dafbb5640 1 -- 172.21.15.76:0/3552732578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da804f030 con 0x7f3da8146b20 2024-03-08T06:01:33.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.468+0000 7f3d9a7fc640 1 -- 172.21.15.76:0/3552732578 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d9c01b070 con 0x7f3da8146b20 2024-03-08T06:01:33.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.469+0000 7f3d9a7fc640 1 --2- 172.21.15.76:0/3552732578 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3d8c06b4d0 0x7f3d8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.469+0000 7f3d9a7fc640 1 -- 172.21.15.76:0/3552732578 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3d9c0898d0 con 0x7f3da8146b20 2024-03-08T06:01:33.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.469+0000 7f3dad951640 1 --2- 172.21.15.76:0/3552732578 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3d8c06b4d0 0x7f3d8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.469+0000 7f3dad951640 1 --2- 172.21.15.76:0/3552732578 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3d8c06b4d0 0x7f3d8c06d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3da4036000 tx=0x7f3da4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:33.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.473+0000 7f3d9a7fc640 1 -- 172.21.15.76:0/3552732578 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3da806e730 con 0x7f3da8146b20 2024-03-08T06:01:33.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.644+0000 7f3dafbb5640 1 -- 172.21.15.76:0/3552732578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3da8131240 con 0x7f3da8146b20 2024-03-08T06:01:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.646+0000 7f3d9a7fc640 1 -- 172.21.15.76:0/3552732578 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18757 (secure 0 0 0) 0x7f3d9c087e20 con 0x7f3da8146b20 2024-03-08T06:01:33.646 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:33.647 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":28,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:01:17.313272+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:33.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.647+0000 7f3dafbb5640 1 -- 172.21.15.76:0/3552732578 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3d8c06b4d0 msgr2=0x7f3d8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.647+0000 7f3dafbb5640 1 --2- 172.21.15.76:0/3552732578 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3d8c06b4d0 0x7f3d8c06d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3da4036000 tx=0x7f3da4008030 comp rx=0 tx=0).stop 2024-03-08T06:01:33.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.648+0000 7f3dafbb5640 1 -- 172.21.15.76:0/3552732578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da8146b20 msgr2=0x7f3da8130d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.648+0000 7f3dafbb5640 1 --2- 172.21.15.76:0/3552732578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da8146b20 0x7f3da8130d00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00c780 tx=0x7f3d9c00cc40 comp rx=0 tx=0).stop 2024-03-08T06:01:33.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.648+0000 7f3dafbb5640 1 -- 172.21.15.76:0/3552732578 shutdown_connections 2024-03-08T06:01:33.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.648+0000 7f3dafbb5640 1 --2- 172.21.15.76:0/3552732578 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3d8c06b4d0 0x7f3d8c06d990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.648+0000 7f3dafbb5640 1 --2- 172.21.15.76:0/3552732578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3da8147760 0x7f3da80c4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.648+0000 7f3dafbb5640 1 --2- 172.21.15.76:0/3552732578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da8146b20 0x7f3da8130d00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.648+0000 7f3dafbb5640 1 --2- 172.21.15.76:0/3552732578 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3da80af020 0x7f3da815d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.648+0000 7f3dafbb5640 1 -- 172.21.15.76:0/3552732578 >> 172.21.15.76:0/3552732578 conn(0x7f3da80b1aa0 msgr2=0x7f3da81452b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:33.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.648+0000 7f3dafbb5640 1 -- 172.21.15.76:0/3552732578 shutdown_connections 2024-03-08T06:01:33.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.648+0000 7f3dafbb5640 1 -- 172.21.15.76:0/3552732578 wait complete. 2024-03-08T06:01:33.659 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:00:33.926651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:00:34.474030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T06:00:38.207853+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T06:01:33.660 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:01:33.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.764+0000 7f79eb33e640 1 Processor -- start 2024-03-08T06:01:33.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.764+0000 7f79eb33e640 1 -- start start 2024-03-08T06:01:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79eb33e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79e40afde0 0x7f79e406b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79eb33e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e40cd580 con 0x7f79e40af950 2024-03-08T06:01:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79eb33e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e40cd860 con 0x7f79e415dcb0 2024-03-08T06:01:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79eb33e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e40cdb40 con 0x7f79e40afde0 2024-03-08T06:01:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79e88d9640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f79e40af950 0x7f79e406fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46778/0 (socket says 172.21.15.76:46778) 2024-03-08T06:01:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79e88d9640 1 -- 172.21.15.76:0/667776575 learned_addr learned my addr 172.21.15.76:0/667776575 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79dbfff640 1 -- 172.21.15.76:0/667776575 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292192029 0 0) 0x7f79e40cd580 con 0x7f79e40af950 2024-03-08T06:01:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79e98db640 1 --2- 172.21.15.76:0/667776575 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79e40afde0 0x7f79e406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79dbfff640 1 -- 172.21.15.76:0/667776575 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79d0003520 con 0x7f79e40af950 2024-03-08T06:01:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79dbfff640 1 -- 172.21.15.76:0/667776575 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2762796620 0 0) 0x7f79d0003520 con 0x7f79e40af950 2024-03-08T06:01:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79dbfff640 1 -- 172.21.15.76:0/667776575 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79e40afde0 msgr2=0x7f79e406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79dbfff640 1 --2- 172.21.15.76:0/667776575 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79e40afde0 0x7f79e406b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79dbfff640 1 -- 172.21.15.76:0/667776575 >> v1:172.21.15.81:6789/0 conn(0x7f79e415dcb0 legacy=0x7f79e40ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79dbfff640 1 -- 172.21.15.76:0/667776575 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e40cde20 con 0x7f79e40af950 2024-03-08T06:01:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79dbfff640 1 -- 172.21.15.76:0/667776575 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f79d4001690 con 0x7f79e40af950 2024-03-08T06:01:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79dbfff640 1 -- 172.21.15.76:0/667776575 >> v1:172.21.15.76:6789/0 conn(0x7f79e40af950 legacy=0x7f79e406fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79dbfff640 1 --2- 172.21.15.76:0/667776575 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79e40afde0 0x7f79d0003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79dbfff640 1 --2- 172.21.15.76:0/667776575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79d0003fd0 0x7f79d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79dbfff640 1 --2- 172.21.15.76:0/667776575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79d0004d40 0x7f79d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79dbfff640 1 -- 172.21.15.76:0/667776575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e40cdb40 con 0x7f79d0004d40 2024-03-08T06:01:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79dbfff640 1 -- 172.21.15.76:0/667776575 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d000bb20 con 0x7f79d0003fd0 2024-03-08T06:01:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.765+0000 7f79dbfff640 1 -- 172.21.15.76:0/667776575 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d000bd20 con 0x7f79e40afde0 2024-03-08T06:01:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.766+0000 7f79e98db640 1 --2- 172.21.15.76:0/667776575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79d0004d40 0x7f79d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.766+0000 7f79e90da640 1 --2- 172.21.15.76:0/667776575 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79e40afde0 0x7f79d0003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.766+0000 7f79e88d9640 1 --2- 172.21.15.76:0/667776575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79d0003fd0 0x7f79d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.766+0000 7f79e98db640 1 -- 172.21.15.76:0/667776575 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79e40afde0 msgr2=0x7f79d0003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.766+0000 7f79e98db640 1 --2- 172.21.15.76:0/667776575 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79e40afde0 0x7f79d0003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.766+0000 7f79e98db640 1 -- 172.21.15.76:0/667776575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79d0003fd0 msgr2=0x7f79d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.766+0000 7f79e98db640 1 --2- 172.21.15.76:0/667776575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79d0003fd0 0x7f79d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.766+0000 7f79e98db640 1 -- 172.21.15.76:0/667776575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d0003520 con 0x7f79d0004d40 2024-03-08T06:01:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.767+0000 7f79e98db640 1 --2- 172.21.15.76:0/667776575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79d0004d40 0x7f79d00051c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009fc0 tx=0x7f79d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9557552cdbd70abd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.767+0000 7f79dbfff640 1 -- 172.21.15.76:0/667776575 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc039070 con 0x7f79d0004d40 2024-03-08T06:01:33.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.767+0000 7f79dbfff640 1 -- 172.21.15.76:0/667776575 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79dc02cc20 con 0x7f79d0004d40 2024-03-08T06:01:33.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.767+0000 7f79dbfff640 1 -- 172.21.15.76:0/667776575 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc009c10 con 0x7f79d0004d40 2024-03-08T06:01:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.767+0000 7f79eb33e640 1 -- 172.21.15.76:0/667776575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79d0004d40 msgr2=0x7f79d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.767+0000 7f79eb33e640 1 --2- 172.21.15.76:0/667776575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79d0004d40 0x7f79d00051c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009fc0 tx=0x7f79d0005830 comp rx=0 tx=0).stop 2024-03-08T06:01:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.767+0000 7f79e98db640 1 -- 172.21.15.76:0/667776575 reap_dead start 2024-03-08T06:01:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.767+0000 7f79eb33e640 1 -- 172.21.15.76:0/667776575 shutdown_connections 2024-03-08T06:01:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.767+0000 7f79eb33e640 1 -- 172.21.15.76:0/667776575 >> 172.21.15.76:0/667776575 conn(0x7f79e40b2320 msgr2=0x7f79e40b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.767+0000 7f79eb33e640 1 -- 172.21.15.76:0/667776575 shutdown_connections 2024-03-08T06:01:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.767+0000 7f79eb33e640 1 -- 172.21.15.76:0/667776575 wait complete. 2024-03-08T06:01:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.769+0000 7f79eb33e640 1 Processor -- start 2024-03-08T06:01:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.769+0000 7f79eb33e640 1 -- start start 2024-03-08T06:01:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.769+0000 7f79eb33e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79d0004d40 0x7f79e414cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.769+0000 7f79eb33e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79e409a710 0x7f79e4157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.769+0000 7f79eb33e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79e409ab90 0x7f79e4157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.769+0000 7f79eb33e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e415e210 con 0x7f79d0004d40 2024-03-08T06:01:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.769+0000 7f79eb33e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e415e090 con 0x7f79e409ab90 2024-03-08T06:01:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.769+0000 7f79eb33e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e4070210 con 0x7f79e409a710 2024-03-08T06:01:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.769+0000 7f79e90da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79d0004d40 0x7f79e414cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.769+0000 7f79e90da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79d0004d40 0x7f79e414cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53434/0 (socket says 172.21.15.76:53434) 2024-03-08T06:01:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.769+0000 7f79e90da640 1 -- 172.21.15.76:0/3303995909 learned_addr learned my addr 172.21.15.76:0/3303995909 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.770+0000 7f79e90da640 1 -- 172.21.15.76:0/3303995909 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79e409a710 msgr2=0x7f79e4157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.770+0000 7f79e90da640 1 --2- 172.21.15.76:0/3303995909 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79e409a710 0x7f79e4157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.770+0000 7f79e90da640 1 -- 172.21.15.76:0/3303995909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79e409ab90 msgr2=0x7f79e4157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.770+0000 7f79e90da640 1 --2- 172.21.15.76:0/3303995909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79e409ab90 0x7f79e4157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.770+0000 7f79e90da640 1 -- 172.21.15.76:0/3303995909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e409cf20 con 0x7f79d0004d40 2024-03-08T06:01:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.770+0000 7f79e90da640 1 --2- 172.21.15.76:0/3303995909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79d0004d40 0x7f79e414cb00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f79e000f820 tx=0x7f79e000fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.770+0000 7f79d9ffb640 1 -- 172.21.15.76:0/3303995909 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e00050d0 con 0x7f79d0004d40 2024-03-08T06:01:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.770+0000 7f79d9ffb640 1 -- 172.21.15.76:0/3303995909 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e0018070 con 0x7f79d0004d40 2024-03-08T06:01:33.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.770+0000 7f79eb33e640 1 -- 172.21.15.76:0/3303995909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e409d200 con 0x7f79d0004d40 2024-03-08T06:01:33.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.770+0000 7f79d9ffb640 1 -- 172.21.15.76:0/3303995909 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0005470 con 0x7f79d0004d40 2024-03-08T06:01:33.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.770+0000 7f79eb33e640 1 -- 172.21.15.76:0/3303995909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e409d7c0 con 0x7f79d0004d40 2024-03-08T06:01:33.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.771+0000 7f79eb33e640 1 -- 172.21.15.76:0/3303995909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e406cd20 con 0x7f79d0004d40 2024-03-08T06:01:33.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.771+0000 7f79d9ffb640 1 -- 172.21.15.76:0/3303995909 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79e000d6b0 con 0x7f79d0004d40 2024-03-08T06:01:33.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.772+0000 7f79d9ffb640 1 --2- 172.21.15.76:0/3303995909 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f79c806b500 0x7f79c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:33.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.772+0000 7f79d9ffb640 1 -- 172.21.15.76:0/3303995909 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f79e008a480 con 0x7f79d0004d40 2024-03-08T06:01:33.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.772+0000 7f79e88d9640 1 --2- 172.21.15.76:0/3303995909 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f79c806b500 0x7f79c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:33.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.772+0000 7f79e88d9640 1 --2- 172.21.15.76:0/3303995909 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f79c806b500 0x7f79c806d9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f79d4001ba0 tx=0x7f79d4001b10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:33.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.776+0000 7f79d9ffb640 1 -- 172.21.15.76:0/3303995909 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79e0058f00 con 0x7f79d0004d40 2024-03-08T06:01:33.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.947+0000 7f79eb33e640 1 -- 172.21.15.76:0/3303995909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79e40af4c0 con 0x7f79d0004d40 2024-03-08T06:01:33.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.947+0000 7f79d9ffb640 1 -- 172.21.15.76:0/3303995909 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79e0058800 con 0x7f79d0004d40 2024-03-08T06:01:33.948 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:01:33.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.949+0000 7f79eb33e640 1 -- 172.21.15.76:0/3303995909 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f79c806b500 msgr2=0x7f79c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.949+0000 7f79eb33e640 1 --2- 172.21.15.76:0/3303995909 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f79c806b500 0x7f79c806d9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f79d4001ba0 tx=0x7f79d4001b10 comp rx=0 tx=0).stop 2024-03-08T06:01:33.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.949+0000 7f79eb33e640 1 -- 172.21.15.76:0/3303995909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79d0004d40 msgr2=0x7f79e414cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:33.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.949+0000 7f79eb33e640 1 --2- 172.21.15.76:0/3303995909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79d0004d40 0x7f79e414cb00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f79e000f820 tx=0x7f79e000fce0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.949+0000 7f79eb33e640 1 -- 172.21.15.76:0/3303995909 shutdown_connections 2024-03-08T06:01:33.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.949+0000 7f79eb33e640 1 --2- 172.21.15.76:0/3303995909 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f79c806b500 0x7f79c806d9c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.949+0000 7f79eb33e640 1 --2- 172.21.15.76:0/3303995909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79e409ab90 0x7f79e4157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.949+0000 7f79eb33e640 1 --2- 172.21.15.76:0/3303995909 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79e409a710 0x7f79e4157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.949+0000 7f79eb33e640 1 --2- 172.21.15.76:0/3303995909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79d0004d40 0x7f79e414cb00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:33.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.949+0000 7f79eb33e640 1 -- 172.21.15.76:0/3303995909 >> 172.21.15.76:0/3303995909 conn(0x7f79e40b2320 msgr2=0x7f79e40ae610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:33.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.949+0000 7f79eb33e640 1 -- 172.21.15.76:0/3303995909 shutdown_connections 2024-03-08T06:01:33.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:33.949+0000 7f79eb33e640 1 -- 172.21.15.76:0/3303995909 wait complete. 2024-03-08T06:01:33.961 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:01:34.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.063+0000 7fe428c9d640 1 Processor -- start 2024-03-08T06:01:34.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.063+0000 7fe428c9d640 1 -- start start 2024-03-08T06:01:34.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.064+0000 7fe428c9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe4240ce690 0x7fe4240b0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.064+0000 7fe428c9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe4240cea70 0x7fe4240d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.064+0000 7fe428c9d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4240d0d40 0x7fe4240b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.064+0000 7fe428c9d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4240b0d80 con 0x7fe4240cea70 2024-03-08T06:01:34.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.064+0000 7fe428c9d640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4240cfa80 con 0x7fe4240ce690 2024-03-08T06:01:34.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.064+0000 7fe428c9d640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4240b1000 con 0x7fe4240d0d40 2024-03-08T06:01:34.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.065+0000 7fe421d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe4240cea70 0x7fe4240d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.065+0000 7fe421d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe4240cea70 0x7fe4240d0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53438/0 (socket says 172.21.15.76:53438) 2024-03-08T06:01:34.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.065+0000 7fe421d9b640 1 -- 172.21.15.76:0/3344575797 learned_addr learned my addr 172.21.15.76:0/3344575797 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:34.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.065+0000 7fe422d9d640 1 --2- 172.21.15.76:0/3344575797 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4240d0d40 0x7fe4240b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.065+0000 7fe42259c640 1 --2- 172.21.15.76:0/3344575797 >> v2:172.21.15.81:3300/0 conn(0x7fe4240ce690 0x7fe4240b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.065+0000 7fe421d9b640 1 -- 172.21.15.76:0/3344575797 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4240d0d40 msgr2=0x7fe4240b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:34.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.065+0000 7fe421d9b640 1 --2- 172.21.15.76:0/3344575797 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4240d0d40 0x7fe4240b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.065+0000 7fe421d9b640 1 -- 172.21.15.76:0/3344575797 >> v2:172.21.15.81:3300/0 conn(0x7fe4240ce690 msgr2=0x7fe4240b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:34.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.065+0000 7fe421d9b640 1 --2- 172.21.15.76:0/3344575797 >> v2:172.21.15.81:3300/0 conn(0x7fe4240ce690 0x7fe4240b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.065+0000 7fe421d9b640 1 -- 172.21.15.76:0/3344575797 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4240cfd20 con 0x7fe4240cea70 2024-03-08T06:01:34.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.066+0000 7fe421d9b640 1 --2- 172.21.15.76:0/3344575797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4240cea70 0x7fe4240d0800 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe4240b39a0 tx=0x7fe41802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ada116eb9f5e8d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:34.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.066+0000 7fe42159a640 1 -- 172.21.15.76:0/3344575797 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe418035020 con 0x7fe4240cea70 2024-03-08T06:01:34.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.066+0000 7fe42159a640 1 -- 172.21.15.76:0/3344575797 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe418031870 con 0x7fe4240cea70 2024-03-08T06:01:34.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.066+0000 7fe42159a640 1 -- 172.21.15.76:0/3344575797 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe418031c40 con 0x7fe4240cea70 2024-03-08T06:01:34.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.066+0000 7fe428c9d640 1 -- 172.21.15.76:0/3344575797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4240cea70 msgr2=0x7fe4240d0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.066+0000 7fe428c9d640 1 --2- 172.21.15.76:0/3344575797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4240cea70 0x7fe4240d0800 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe4240b39a0 tx=0x7fe41802b910 comp rx=0 tx=0).stop 2024-03-08T06:01:34.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.066+0000 7fe428c9d640 1 -- 172.21.15.76:0/3344575797 shutdown_connections 2024-03-08T06:01:34.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.066+0000 7fe428c9d640 1 --2- 172.21.15.76:0/3344575797 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4240d0d40 0x7fe4240b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.066+0000 7fe428c9d640 1 --2- 172.21.15.76:0/3344575797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4240cea70 0x7fe4240d0800 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.066+0000 7fe428c9d640 1 --2- 172.21.15.76:0/3344575797 >> v2:172.21.15.81:3300/0 conn(0x7fe4240ce690 0x7fe4240b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.066+0000 7fe428c9d640 1 -- 172.21.15.76:0/3344575797 >> 172.21.15.76:0/3344575797 conn(0x7fe42406ae60 msgr2=0x7fe42406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:34.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.066+0000 7fe428c9d640 1 -- 172.21.15.76:0/3344575797 shutdown_connections 2024-03-08T06:01:34.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.066+0000 7fe428c9d640 1 -- 172.21.15.76:0/3344575797 wait complete. 2024-03-08T06:01:34.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.068+0000 7fe428c9d640 1 Processor -- start 2024-03-08T06:01:34.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.068+0000 7fe428c9d640 1 -- start start 2024-03-08T06:01:34.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.068+0000 7fe428c9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4240ce690 0x7fe42415b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.068+0000 7fe428c9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4240cea70 0x7fe4241539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.068+0000 7fe428c9d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4240d0d40 0x7fe4241f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.068+0000 7fe428c9d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4240ceed0 con 0x7fe4240cea70 2024-03-08T06:01:34.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.068+0000 7fe428c9d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4240d02e0 con 0x7fe4240ce690 2024-03-08T06:01:34.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.068+0000 7fe428c9d640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42415be20 con 0x7fe4240d0d40 2024-03-08T06:01:34.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.069+0000 7fe421d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4240cea70 0x7fe4241539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.069+0000 7fe421d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4240cea70 0x7fe4241539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53442/0 (socket says 172.21.15.76:53442) 2024-03-08T06:01:34.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.069+0000 7fe421d9b640 1 -- 172.21.15.76:0/352752348 learned_addr learned my addr 172.21.15.76:0/352752348 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:34.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.069+0000 7fe422d9d640 1 --2- 172.21.15.76:0/352752348 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4240d0d40 0x7fe4241f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.069+0000 7fe42259c640 1 --2- 172.21.15.76:0/352752348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4240ce690 0x7fe42415b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.069+0000 7fe421d9b640 1 -- 172.21.15.76:0/352752348 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4240d0d40 msgr2=0x7fe4241f3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.069+0000 7fe421d9b640 1 --2- 172.21.15.76:0/352752348 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4240d0d40 0x7fe4241f3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.069+0000 7fe421d9b640 1 -- 172.21.15.76:0/352752348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4240ce690 msgr2=0x7fe42415b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.069+0000 7fe421d9b640 1 --2- 172.21.15.76:0/352752348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4240ce690 0x7fe42415b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.069+0000 7fe421d9b640 1 -- 172.21.15.76:0/352752348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4240c68b0 con 0x7fe4240cea70 2024-03-08T06:01:34.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.069+0000 7fe421d9b640 1 --2- 172.21.15.76:0/352752348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4240cea70 0x7fe4241539c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe418000c00 tx=0x7fe4180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:34.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.069+0000 7fe412ffd640 1 -- 172.21.15.76:0/352752348 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe418035040 con 0x7fe4240cea70 2024-03-08T06:01:34.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.069+0000 7fe412ffd640 1 -- 172.21.15.76:0/352752348 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe418030d00 con 0x7fe4240cea70 2024-03-08T06:01:34.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.069+0000 7fe412ffd640 1 -- 172.21.15.76:0/352752348 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4180456a0 con 0x7fe4240cea70 2024-03-08T06:01:34.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.069+0000 7fe428c9d640 1 -- 172.21.15.76:0/352752348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4240c6b90 con 0x7fe4240cea70 2024-03-08T06:01:34.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.069+0000 7fe428c9d640 1 -- 172.21.15.76:0/352752348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4240c7150 con 0x7fe4240cea70 2024-03-08T06:01:34.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.071+0000 7fe428c9d640 1 -- 172.21.15.76:0/352752348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42404f030 con 0x7fe4240cea70 2024-03-08T06:01:34.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.071+0000 7fe412ffd640 1 -- 172.21.15.76:0/352752348 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe418038020 con 0x7fe4240cea70 2024-03-08T06:01:34.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.071+0000 7fe412ffd640 1 --2- 172.21.15.76:0/352752348 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe40406b4d0 0x7fe40406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.071+0000 7fe412ffd640 1 -- 172.21.15.76:0/352752348 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe4180ac930 con 0x7fe4240cea70 2024-03-08T06:01:34.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.071+0000 7fe42259c640 1 --2- 172.21.15.76:0/352752348 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe40406b4d0 0x7fe40406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.072+0000 7fe42259c640 1 --2- 172.21.15.76:0/352752348 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe40406b4d0 0x7fe40406d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe414004450 tx=0x7fe414003a30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:34.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.076+0000 7fe412ffd640 1 -- 172.21.15.76:0/352752348 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe41802f320 con 0x7fe4240cea70 2024-03-08T06:01:34.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.245+0000 7fe428c9d640 1 -- 172.21.15.76:0/352752348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe4240cba50 con 0x7fe4240cea70 2024-03-08T06:01:34.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.246+0000 7fe412ffd640 1 -- 172.21.15.76:0/352752348 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe41807b440 con 0x7fe4240cea70 2024-03-08T06:01:34.246 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:34.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.248+0000 7fe428c9d640 1 -- 172.21.15.76:0/352752348 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe40406b4d0 msgr2=0x7fe40406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.248+0000 7fe428c9d640 1 --2- 172.21.15.76:0/352752348 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe40406b4d0 0x7fe40406d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe414004450 tx=0x7fe414003a30 comp rx=0 tx=0).stop 2024-03-08T06:01:34.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.248+0000 7fe428c9d640 1 -- 172.21.15.76:0/352752348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4240cea70 msgr2=0x7fe4241539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.248+0000 7fe428c9d640 1 --2- 172.21.15.76:0/352752348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4240cea70 0x7fe4241539c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe418000c00 tx=0x7fe4180046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.248+0000 7fe428c9d640 1 -- 172.21.15.76:0/352752348 shutdown_connections 2024-03-08T06:01:34.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.248+0000 7fe428c9d640 1 --2- 172.21.15.76:0/352752348 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe40406b4d0 0x7fe40406d990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.248+0000 7fe428c9d640 1 --2- 172.21.15.76:0/352752348 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4240d0d40 0x7fe4241f3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.248+0000 7fe428c9d640 1 --2- 172.21.15.76:0/352752348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4240cea70 0x7fe4241539c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.248+0000 7fe428c9d640 1 --2- 172.21.15.76:0/352752348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4240ce690 0x7fe42415b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.248+0000 7fe428c9d640 1 -- 172.21.15.76:0/352752348 >> 172.21.15.76:0/352752348 conn(0x7fe42406ae60 msgr2=0x7fe42406d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:34.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.248+0000 7fe428c9d640 1 -- 172.21.15.76:0/352752348 shutdown_connections 2024-03-08T06:01:34.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.248+0000 7fe428c9d640 1 -- 172.21.15.76:0/352752348 wait complete. 2024-03-08T06:01:34.259 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:01:34.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.362+0000 7f84618c9640 1 Processor -- start 2024-03-08T06:01:34.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.362+0000 7f84618c9640 1 -- start start 2024-03-08T06:01:34.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.362+0000 7f84618c9640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f845c0b9840 0x7f845c0b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.362+0000 7f84618c9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f845c0b5b20 0x7f845c0b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.362+0000 7f84618c9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845c0d27d0 con 0x7f845c15dbc0 2024-03-08T06:01:34.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.362+0000 7f84618c9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845c0b5f90 con 0x7f845c0b5b20 2024-03-08T06:01:34.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.362+0000 7f84618c9640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845c0d2ab0 con 0x7f845c0b9840 2024-03-08T06:01:34.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.363+0000 7f845b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f845c0b5b20 0x7f845c0b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.363+0000 7f845affd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f845c15dbc0 0x7f845c0d1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46780/0 (socket says 172.21.15.76:46780) 2024-03-08T06:01:34.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.363+0000 7f845affd640 1 -- 172.21.15.76:0/2983515582 learned_addr learned my addr 172.21.15.76:0/2983515582 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:34.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.363+0000 7f845a7fc640 1 --2- 172.21.15.76:0/2983515582 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f845c0b9840 0x7f845c0b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.363+0000 7f845b7fe640 1 -- 172.21.15.76:0/2983515582 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f845c0b9840 msgr2=0x7f845c0b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:34.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.363+0000 7f845b7fe640 1 --2- 172.21.15.76:0/2983515582 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f845c0b9840 0x7f845c0b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.363+0000 7f845b7fe640 1 -- 172.21.15.76:0/2983515582 >> v1:172.21.15.76:6789/0 conn(0x7f845c15dbc0 legacy=0x7f845c0d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.363+0000 7f8459ffb640 1 -- 172.21.15.76:0/2983515582 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762884135 0 0) 0x7f845c0d27d0 con 0x7f845c15dbc0 2024-03-08T06:01:34.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.363+0000 7f845b7fe640 1 -- 172.21.15.76:0/2983515582 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c0d2d90 con 0x7f845c0b5b20 2024-03-08T06:01:34.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.363+0000 7f845a7fc640 1 --2- 172.21.15.76:0/2983515582 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f845c0b9840 0x7f845c0b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:34.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.364+0000 7f845b7fe640 1 --2- 172.21.15.76:0/2983515582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f845c0b5b20 0x7f845c0b6c90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f84540097a0 tx=0x7f845402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7babe098b5398e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:34.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.364+0000 7f8459ffb640 1 -- 172.21.15.76:0/2983515582 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8454035020 con 0x7f845c0b5b20 2024-03-08T06:01:34.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.364+0000 7f8459ffb640 1 -- 172.21.15.76:0/2983515582 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8454031830 con 0x7f845c0b5b20 2024-03-08T06:01:34.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.364+0000 7f8459ffb640 1 -- 172.21.15.76:0/2983515582 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8454031c00 con 0x7f845c0b5b20 2024-03-08T06:01:34.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.364+0000 7f84618c9640 1 -- 172.21.15.76:0/2983515582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f845c0b5b20 msgr2=0x7f845c0b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.364+0000 7f84618c9640 1 --2- 172.21.15.76:0/2983515582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f845c0b5b20 0x7f845c0b6c90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f84540097a0 tx=0x7f845402b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.364+0000 7f84618c9640 1 -- 172.21.15.76:0/2983515582 shutdown_connections 2024-03-08T06:01:34.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.364+0000 7f84618c9640 1 --2- 172.21.15.76:0/2983515582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f845c0b5b20 0x7f845c0b6c90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.364+0000 7f84618c9640 1 --2- 172.21.15.76:0/2983515582 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f845c0b9840 0x7f845c0b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.364+0000 7f84618c9640 1 -- 172.21.15.76:0/2983515582 >> 172.21.15.76:0/2983515582 conn(0x7f845c0670d0 msgr2=0x7f845c0d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:34.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.364+0000 7f84618c9640 1 -- 172.21.15.76:0/2983515582 shutdown_connections 2024-03-08T06:01:34.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.364+0000 7f84618c9640 1 -- 172.21.15.76:0/2983515582 wait complete. 2024-03-08T06:01:34.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.365+0000 7f84618c9640 1 Processor -- start 2024-03-08T06:01:34.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.366+0000 7f84618c9640 1 -- start start 2024-03-08T06:01:34.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.366+0000 7f84618c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f845c0b5b20 0x7f845c14b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.366+0000 7f84618c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f845c0b9840 0x7f845c0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.366+0000 7f84618c9640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f845c15dbc0 0x7f845c157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.366+0000 7f84618c9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c15dfa0 con 0x7f845c0b5b20 2024-03-08T06:01:34.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.366+0000 7f84618c9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c0d3870 con 0x7f845c0b9840 2024-03-08T06:01:34.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.366+0000 7f84618c9640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845c15e120 con 0x7f845c15dbc0 2024-03-08T06:01:34.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.366+0000 7f845affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f845c0b5b20 0x7f845c14b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.366+0000 7f845affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f845c0b5b20 0x7f845c14b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53466/0 (socket says 172.21.15.76:53466) 2024-03-08T06:01:34.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.366+0000 7f845affd640 1 -- 172.21.15.76:0/739155531 learned_addr learned my addr 172.21.15.76:0/739155531 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:34.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.367+0000 7f845a7fc640 1 --2- 172.21.15.76:0/739155531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f845c0b9840 0x7f845c0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.367+0000 7f845b7fe640 1 --2- 172.21.15.76:0/739155531 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f845c15dbc0 0x7f845c157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.367+0000 7f845affd640 1 -- 172.21.15.76:0/739155531 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f845c15dbc0 msgr2=0x7f845c157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.367+0000 7f845affd640 1 --2- 172.21.15.76:0/739155531 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f845c15dbc0 0x7f845c157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.367+0000 7f845affd640 1 -- 172.21.15.76:0/739155531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f845c0b9840 msgr2=0x7f845c0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.367+0000 7f845affd640 1 --2- 172.21.15.76:0/739155531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f845c0b9840 0x7f845c0c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.367+0000 7f845affd640 1 -- 172.21.15.76:0/739155531 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c0cc590 con 0x7f845c0b5b20 2024-03-08T06:01:34.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.367+0000 7f845a7fc640 1 --2- 172.21.15.76:0/739155531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f845c0b9840 0x7f845c0c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:34.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.367+0000 7f845b7fe640 1 --2- 172.21.15.76:0/739155531 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f845c15dbc0 0x7f845c157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:34.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.367+0000 7f845affd640 1 --2- 172.21.15.76:0/739155531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f845c0b5b20 0x7f845c14b9c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f844c00c7c0 tx=0x7f844c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:34.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.367+0000 7f8443fff640 1 -- 172.21.15.76:0/739155531 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c018070 con 0x7f845c0b5b20 2024-03-08T06:01:34.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.367+0000 7f8443fff640 1 -- 172.21.15.76:0/739155531 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c00a640 con 0x7f845c0b5b20 2024-03-08T06:01:34.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.367+0000 7f8443fff640 1 -- 172.21.15.76:0/739155531 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c00aa30 con 0x7f845c0b5b20 2024-03-08T06:01:34.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.367+0000 7f84618c9640 1 -- 172.21.15.76:0/739155531 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f845c0cc870 con 0x7f845c0b5b20 2024-03-08T06:01:34.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.367+0000 7f84618c9640 1 -- 172.21.15.76:0/739155531 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845c159ea0 con 0x7f845c0b5b20 2024-03-08T06:01:34.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.368+0000 7f84618c9640 1 -- 172.21.15.76:0/739155531 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f845c04f030 con 0x7f845c0b5b20 2024-03-08T06:01:34.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.369+0000 7f8443fff640 1 -- 172.21.15.76:0/739155531 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f844c01b070 con 0x7f845c0b5b20 2024-03-08T06:01:34.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.369+0000 7f8443fff640 1 --2- 172.21.15.76:0/739155531 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f844806b4d0 0x7f844806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.369+0000 7f845a7fc640 1 --2- 172.21.15.76:0/739155531 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f844806b4d0 0x7f844806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.369+0000 7f8443fff640 1 -- 172.21.15.76:0/739155531 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f844c089ba0 con 0x7f845c0b5b20 2024-03-08T06:01:34.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.369+0000 7f845a7fc640 1 --2- 172.21.15.76:0/739155531 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f844806b4d0 0x7f844806d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8450006fd0 tx=0x7f8450008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:34.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.373+0000 7f8443fff640 1 -- 172.21.15.76:0/739155531 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f844c0586b0 con 0x7f845c0b5b20 2024-03-08T06:01:34.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.543+0000 7f84618c9640 1 -- 172.21.15.76:0/739155531 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f845c0cca40 con 0x7f845c0b5b20 2024-03-08T06:01:34.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.543+0000 7f8443fff640 1 -- 172.21.15.76:0/739155531 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f844c057fb0 con 0x7f845c0b5b20 2024-03-08T06:01:34.544 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:01:34.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.545+0000 7f84618c9640 1 -- 172.21.15.76:0/739155531 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f844806b4d0 msgr2=0x7f844806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.545+0000 7f84618c9640 1 --2- 172.21.15.76:0/739155531 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f844806b4d0 0x7f844806d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8450006fd0 tx=0x7f8450008040 comp rx=0 tx=0).stop 2024-03-08T06:01:34.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.545+0000 7f84618c9640 1 -- 172.21.15.76:0/739155531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f845c0b5b20 msgr2=0x7f845c14b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.545+0000 7f84618c9640 1 --2- 172.21.15.76:0/739155531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f845c0b5b20 0x7f845c14b9c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f844c00c7c0 tx=0x7f844c00cc80 comp rx=0 tx=0).stop 2024-03-08T06:01:34.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.545+0000 7f84618c9640 1 -- 172.21.15.76:0/739155531 shutdown_connections 2024-03-08T06:01:34.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.545+0000 7f84618c9640 1 --2- 172.21.15.76:0/739155531 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f844806b4d0 0x7f844806d990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.545+0000 7f84618c9640 1 --2- 172.21.15.76:0/739155531 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f845c15dbc0 0x7f845c157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.545+0000 7f84618c9640 1 --2- 172.21.15.76:0/739155531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f845c0b9840 0x7f845c0c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.545+0000 7f84618c9640 1 --2- 172.21.15.76:0/739155531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f845c0b5b20 0x7f845c14b9c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.545+0000 7f84618c9640 1 -- 172.21.15.76:0/739155531 >> 172.21.15.76:0/739155531 conn(0x7f845c0670d0 msgr2=0x7f845c0ceb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:34.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.545+0000 7f84618c9640 1 -- 172.21.15.76:0/739155531 shutdown_connections 2024-03-08T06:01:34.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.545+0000 7f84618c9640 1 -- 172.21.15.76:0/739155531 wait complete. 2024-03-08T06:01:34.557 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T06:01:34.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.658+0000 7fb0f76c2640 1 Processor -- start 2024-03-08T06:01:34.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.658+0000 7fb0f76c2640 1 -- start start 2024-03-08T06:01:34.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.659+0000 7fb0f76c2640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb0f015dbc0 0x7fb0f00d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.659+0000 7fb0f76c2640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0f00b9840 0x7fb0f00b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.659+0000 7fb0f76c2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f00d27d0 con 0x7fb0f00b5b20 2024-03-08T06:01:34.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.659+0000 7fb0f76c2640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f00b5f90 con 0x7fb0f015dbc0 2024-03-08T06:01:34.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.659+0000 7fb0f76c2640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00d2ab0 con 0x7fb0f00b9840 2024-03-08T06:01:34.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.660+0000 7fb0f4c5d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0f00b9840 0x7fb0f00b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.660+0000 7fb0f545e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb0f015dbc0 0x7fb0f00d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.660+0000 7fb0f4c5d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0f00b9840 0x7fb0f00b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:56900/0 (socket says 172.21.15.76:56900) 2024-03-08T06:01:34.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.660+0000 7fb0f4c5d640 1 -- 172.21.15.76:0/3268857152 learned_addr learned my addr 172.21.15.76:0/3268857152 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:34.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.660+0000 7fb0f4c5d640 1 -- 172.21.15.76:0/3268857152 >> v2:172.21.15.81:3300/0 conn(0x7fb0f015dbc0 msgr2=0x7fb0f00d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:34.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.660+0000 7fb0f4c5d640 1 --2- 172.21.15.76:0/3268857152 >> v2:172.21.15.81:3300/0 conn(0x7fb0f015dbc0 0x7fb0f00d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.660+0000 7fb0f4c5d640 1 -- 172.21.15.76:0/3268857152 >> v1:172.21.15.81:6789/0 conn(0x7fb0f00b5b20 legacy=0x7fb0f006c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.660+0000 7fb0f4c5d640 1 -- 172.21.15.76:0/3268857152 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00d2d90 con 0x7fb0f00b9840 2024-03-08T06:01:34.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.661+0000 7fb0f4c5d640 1 --2- 172.21.15.76:0/3268857152 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0f00b9840 0x7fb0f00b55e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009870 tx=0x7fb0e002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da2734cf1f5ee2be server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:34.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.661+0000 7fb0e7fff640 1 -- 172.21.15.76:0/3268857152 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0034070 con 0x7fb0f00b9840 2024-03-08T06:01:34.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.661+0000 7fb0e7fff640 1 -- 172.21.15.76:0/3268857152 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e0030840 con 0x7fb0f00b9840 2024-03-08T06:01:34.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.661+0000 7fb0e7fff640 1 -- 172.21.15.76:0/3268857152 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0030c10 con 0x7fb0f00b9840 2024-03-08T06:01:34.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.661+0000 7fb0f76c2640 1 -- 172.21.15.76:0/3268857152 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0f00b9840 msgr2=0x7fb0f00b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.661+0000 7fb0f76c2640 1 --2- 172.21.15.76:0/3268857152 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0f00b9840 0x7fb0f00b55e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009870 tx=0x7fb0e002b980 comp rx=0 tx=0).stop 2024-03-08T06:01:34.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.662+0000 7fb0f76c2640 1 -- 172.21.15.76:0/3268857152 shutdown_connections 2024-03-08T06:01:34.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.662+0000 7fb0f76c2640 1 --2- 172.21.15.76:0/3268857152 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0f00b9840 0x7fb0f00b55e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.662+0000 7fb0f76c2640 1 --2- 172.21.15.76:0/3268857152 >> v2:172.21.15.81:3300/0 conn(0x7fb0f015dbc0 0x7fb0f00d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.662+0000 7fb0f76c2640 1 -- 172.21.15.76:0/3268857152 >> 172.21.15.76:0/3268857152 conn(0x7fb0f00670d0 msgr2=0x7fb0f00d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:34.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.662+0000 7fb0f76c2640 1 -- 172.21.15.76:0/3268857152 shutdown_connections 2024-03-08T06:01:34.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.662+0000 7fb0f76c2640 1 -- 172.21.15.76:0/3268857152 wait complete. 2024-03-08T06:01:34.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.663+0000 7fb0f76c2640 1 Processor -- start 2024-03-08T06:01:34.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.663+0000 7fb0f76c2640 1 -- start start 2024-03-08T06:01:34.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.663+0000 7fb0f76c2640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0f00b5b20 0x7fb0f00ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.663+0000 7fb0f76c2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f00b9840 0x7fb0f01f6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.663+0000 7fb0f76c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f015dbc0 0x7fb0f01f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.663+0000 7fb0f76c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00d2430 con 0x7fb0f015dbc0 2024-03-08T06:01:34.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.663+0000 7fb0f76c2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00d3870 con 0x7fb0f00b9840 2024-03-08T06:01:34.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.663+0000 7fb0f76c2640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00d25b0 con 0x7fb0f00b5b20 2024-03-08T06:01:34.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.664+0000 7fb0f5c5f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f015dbc0 0x7fb0f01f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.664+0000 7fb0f5c5f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f015dbc0 0x7fb0f01f6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53474/0 (socket says 172.21.15.76:53474) 2024-03-08T06:01:34.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.664+0000 7fb0f5c5f640 1 -- 172.21.15.76:0/3176632635 learned_addr learned my addr 172.21.15.76:0/3176632635 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:34.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.664+0000 7fb0f545e640 1 --2- 172.21.15.76:0/3176632635 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0f00b5b20 0x7fb0f00ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.664+0000 7fb0f4c5d640 1 --2- 172.21.15.76:0/3176632635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f00b9840 0x7fb0f01f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.664+0000 7fb0f5c5f640 1 -- 172.21.15.76:0/3176632635 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0f00b5b20 msgr2=0x7fb0f00ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.664+0000 7fb0f5c5f640 1 --2- 172.21.15.76:0/3176632635 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0f00b5b20 0x7fb0f00ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.664+0000 7fb0f5c5f640 1 -- 172.21.15.76:0/3176632635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f00b9840 msgr2=0x7fb0f01f6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.664+0000 7fb0f5c5f640 1 --2- 172.21.15.76:0/3176632635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f00b9840 0x7fb0f01f6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.664+0000 7fb0f5c5f640 1 -- 172.21.15.76:0/3176632635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f01e30d0 con 0x7fb0f015dbc0 2024-03-08T06:01:34.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.664+0000 7fb0f5c5f640 1 --2- 172.21.15.76:0/3176632635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f015dbc0 0x7fb0f01f6870 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800c7d0 tx=0x7fb0e800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:34.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.664+0000 7fb0e5ffb640 1 -- 172.21.15.76:0/3176632635 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8003b70 con 0x7fb0f015dbc0 2024-03-08T06:01:34.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.664+0000 7fb0e5ffb640 1 -- 172.21.15.76:0/3176632635 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e8003d30 con 0x7fb0f015dbc0 2024-03-08T06:01:34.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.664+0000 7fb0e5ffb640 1 -- 172.21.15.76:0/3176632635 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e800aa90 con 0x7fb0f015dbc0 2024-03-08T06:01:34.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.664+0000 7fb0f76c2640 1 -- 172.21.15.76:0/3176632635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f01e33b0 con 0x7fb0f015dbc0 2024-03-08T06:01:34.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.664+0000 7fb0f76c2640 1 -- 172.21.15.76:0/3176632635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f01f7190 con 0x7fb0f015dbc0 2024-03-08T06:01:34.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.665+0000 7fb0f76c2640 1 -- 172.21.15.76:0/3176632635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f004f030 con 0x7fb0f015dbc0 2024-03-08T06:01:34.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.666+0000 7fb0e5ffb640 1 -- 172.21.15.76:0/3176632635 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0e800ac30 con 0x7fb0f015dbc0 2024-03-08T06:01:34.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.666+0000 7fb0e5ffb640 1 --2- 172.21.15.76:0/3176632635 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb0d406b4d0 0x7fb0d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.666+0000 7fb0f545e640 1 --2- 172.21.15.76:0/3176632635 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb0d406b4d0 0x7fb0d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.666+0000 7fb0e5ffb640 1 -- 172.21.15.76:0/3176632635 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb0e808a380 con 0x7fb0f015dbc0 2024-03-08T06:01:34.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.667+0000 7fb0f545e640 1 --2- 172.21.15.76:0/3176632635 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb0d406b4d0 0x7fb0d406d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009aa0 tx=0x7fb0ec009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:34.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.671+0000 7fb0e5ffb640 1 -- 172.21.15.76:0/3176632635 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0e8058840 con 0x7fb0f015dbc0 2024-03-08T06:01:34.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.839+0000 7fb0f76c2640 1 -- 172.21.15.76:0/3176632635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb0f015b750 con 0x7fb0f015dbc0 2024-03-08T06:01:34.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.839+0000 7fb0e5ffb640 1 -- 172.21.15.76:0/3176632635 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb0e805c8e0 con 0x7fb0f015dbc0 2024-03-08T06:01:34.840 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:01:34.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.841+0000 7fb0f76c2640 1 -- 172.21.15.76:0/3176632635 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb0d406b4d0 msgr2=0x7fb0d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.841+0000 7fb0f76c2640 1 --2- 172.21.15.76:0/3176632635 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb0d406b4d0 0x7fb0d406d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009aa0 tx=0x7fb0ec009190 comp rx=0 tx=0).stop 2024-03-08T06:01:34.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.841+0000 7fb0f76c2640 1 -- 172.21.15.76:0/3176632635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f015dbc0 msgr2=0x7fb0f01f6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.841+0000 7fb0f76c2640 1 --2- 172.21.15.76:0/3176632635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f015dbc0 0x7fb0f01f6870 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800c7d0 tx=0x7fb0e800cc90 comp rx=0 tx=0).stop 2024-03-08T06:01:34.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.841+0000 7fb0f76c2640 1 -- 172.21.15.76:0/3176632635 shutdown_connections 2024-03-08T06:01:34.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.841+0000 7fb0f76c2640 1 --2- 172.21.15.76:0/3176632635 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb0d406b4d0 0x7fb0d406d990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.841+0000 7fb0f76c2640 1 --2- 172.21.15.76:0/3176632635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f015dbc0 0x7fb0f01f6870 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.841+0000 7fb0f76c2640 1 --2- 172.21.15.76:0/3176632635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f00b9840 0x7fb0f01f6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.841+0000 7fb0f76c2640 1 --2- 172.21.15.76:0/3176632635 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0f00b5b20 0x7fb0f00ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.841+0000 7fb0f76c2640 1 -- 172.21.15.76:0/3176632635 >> 172.21.15.76:0/3176632635 conn(0x7fb0f00670d0 msgr2=0x7fb0f00c7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:34.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.841+0000 7fb0f76c2640 1 -- 172.21.15.76:0/3176632635 shutdown_connections 2024-03-08T06:01:34.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.841+0000 7fb0f76c2640 1 -- 172.21.15.76:0/3176632635 wait complete. 2024-03-08T06:01:34.853 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:01:34.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.953+0000 7fe946581640 1 Processor -- start 2024-03-08T06:01:34.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.953+0000 7fe946581640 1 -- start start 2024-03-08T06:01:34.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.953+0000 7fe946581640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe940157c70 0x7fe94015a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.953+0000 7fe946581640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe94006eb70 0x7fe94006efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.953+0000 7fe946581640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe940158050 con 0x7fe94015abb0 2024-03-08T06:01:34.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.953+0000 7fe946581640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94015b160 con 0x7fe94006eb70 2024-03-08T06:01:34.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.953+0000 7fe946581640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94015b440 con 0x7fe940157c70 2024-03-08T06:01:34.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe94015abb0 0x7fe9400cd310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46794/0 (socket says 172.21.15.76:46794) 2024-03-08T06:01:34.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93f7fe640 1 -- 172.21.15.76:0/1320389279 learned_addr learned my addr 172.21.15.76:0/1320389279 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:34.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93effd640 1 -- 172.21.15.76:0/1320389279 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536824455 0 0) 0x7fe940158050 con 0x7fe94015abb0 2024-03-08T06:01:34.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93ffff640 1 --2- 172.21.15.76:0/1320389279 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe940157c70 0x7fe94015a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe944b1e640 1 --2- 172.21.15.76:0/1320389279 >> v2:172.21.15.81:3300/0 conn(0x7fe94006eb70 0x7fe94006efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93effd640 1 -- 172.21.15.76:0/1320389279 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe928003520 con 0x7fe94015abb0 2024-03-08T06:01:34.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93effd640 1 -- 172.21.15.76:0/1320389279 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2768882565 0 0) 0x7fe928003520 con 0x7fe94015abb0 2024-03-08T06:01:34.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93effd640 1 -- 172.21.15.76:0/1320389279 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe940157c70 msgr2=0x7fe94015a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:34.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93effd640 1 --2- 172.21.15.76:0/1320389279 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe940157c70 0x7fe94015a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93effd640 1 -- 172.21.15.76:0/1320389279 >> v2:172.21.15.81:3300/0 conn(0x7fe94006eb70 msgr2=0x7fe94006efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:34.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93effd640 1 --2- 172.21.15.76:0/1320389279 >> v2:172.21.15.81:3300/0 conn(0x7fe94006eb70 0x7fe94006efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93effd640 1 -- 172.21.15.76:0/1320389279 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94006e370 con 0x7fe94015abb0 2024-03-08T06:01:34.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93effd640 1 -- 172.21.15.76:0/1320389279 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fe934001670 con 0x7fe94015abb0 2024-03-08T06:01:34.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93effd640 1 -- 172.21.15.76:0/1320389279 >> v1:172.21.15.76:6789/0 conn(0x7fe94015abb0 legacy=0x7fe9400cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93effd640 1 --2- 172.21.15.76:0/1320389279 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe940157c70 0x7fe928003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93effd640 1 --2- 172.21.15.76:0/1320389279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe928003fa0 0x7fe928004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93effd640 1 --2- 172.21.15.76:0/1320389279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe928004d40 0x7fe9280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93effd640 1 -- 172.21.15.76:0/1320389279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94015b160 con 0x7fe928004d40 2024-03-08T06:01:34.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93effd640 1 -- 172.21.15.76:0/1320389279 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94015b440 con 0x7fe928003fa0 2024-03-08T06:01:34.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93effd640 1 -- 172.21.15.76:0/1320389279 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92800b9f0 con 0x7fe940157c70 2024-03-08T06:01:34.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe944b1e640 1 --2- 172.21.15.76:0/1320389279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe928004d40 0x7fe9280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93ffff640 1 --2- 172.21.15.76:0/1320389279 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe940157c70 0x7fe928003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.954+0000 7fe93f7fe640 1 --2- 172.21.15.76:0/1320389279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe928003fa0 0x7fe928004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.955+0000 7fe944b1e640 1 -- 172.21.15.76:0/1320389279 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe940157c70 msgr2=0x7fe928003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:34.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.955+0000 7fe944b1e640 1 --2- 172.21.15.76:0/1320389279 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe940157c70 0x7fe928003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.955+0000 7fe944b1e640 1 -- 172.21.15.76:0/1320389279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe928003fa0 msgr2=0x7fe928004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:34.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.955+0000 7fe944b1e640 1 --2- 172.21.15.76:0/1320389279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe928003fa0 0x7fe928004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.955+0000 7fe944b1e640 1 -- 172.21.15.76:0/1320389279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe928003520 con 0x7fe928004d40 2024-03-08T06:01:34.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.955+0000 7fe944b1e640 1 --2- 172.21.15.76:0/1320389279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe928004d40 0x7fe9280051c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe938009fc0 tx=0x7fe928005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5223e73a1de92ff server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:34.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.955+0000 7fe93effd640 1 -- 172.21.15.76:0/1320389279 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938039070 con 0x7fe928004d40 2024-03-08T06:01:34.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.955+0000 7fe93effd640 1 -- 172.21.15.76:0/1320389279 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93802cc60 con 0x7fe928004d40 2024-03-08T06:01:34.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.956+0000 7fe93effd640 1 -- 172.21.15.76:0/1320389279 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938009c10 con 0x7fe928004d40 2024-03-08T06:01:34.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.956+0000 7fe946581640 1 -- 172.21.15.76:0/1320389279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe928004d40 msgr2=0x7fe9280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.956+0000 7fe946581640 1 --2- 172.21.15.76:0/1320389279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe928004d40 0x7fe9280051c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe938009fc0 tx=0x7fe928005830 comp rx=0 tx=0).stop 2024-03-08T06:01:34.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.956+0000 7fe944b1e640 1 -- 172.21.15.76:0/1320389279 reap_dead start 2024-03-08T06:01:34.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.956+0000 7fe946581640 1 -- 172.21.15.76:0/1320389279 shutdown_connections 2024-03-08T06:01:34.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.956+0000 7fe946581640 1 -- 172.21.15.76:0/1320389279 >> 172.21.15.76:0/1320389279 conn(0x7fe9400cf740 msgr2=0x7fe9400cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:34.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.956+0000 7fe946581640 1 -- 172.21.15.76:0/1320389279 shutdown_connections 2024-03-08T06:01:34.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.956+0000 7fe946581640 1 -- 172.21.15.76:0/1320389279 wait complete. 2024-03-08T06:01:34.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.957+0000 7fe946581640 1 Processor -- start 2024-03-08T06:01:34.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.957+0000 7fe946581640 1 -- start start 2024-03-08T06:01:34.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.958+0000 7fe946581640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe940149200 0x7fe9400c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.958+0000 7fe946581640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9400b4540 0x7fe9400b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.958+0000 7fe946581640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe94015b8b0 0x7fe94015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.958+0000 7fe946581640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400cacf0 con 0x7fe9400b4540 2024-03-08T06:01:34.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.958+0000 7fe946581640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94006e020 con 0x7fe94015b8b0 2024-03-08T06:01:34.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.958+0000 7fe946581640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400cda20 con 0x7fe940149200 2024-03-08T06:01:34.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.958+0000 7fe93f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9400b4540 0x7fe9400b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.958+0000 7fe93f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9400b4540 0x7fe9400b49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53490/0 (socket says 172.21.15.76:53490) 2024-03-08T06:01:34.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.958+0000 7fe93f7fe640 1 -- 172.21.15.76:0/2727312987 learned_addr learned my addr 172.21.15.76:0/2727312987 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:34.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.958+0000 7fe944b1e640 1 --2- 172.21.15.76:0/2727312987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe94015b8b0 0x7fe94015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.958+0000 7fe93ffff640 1 --2- 172.21.15.76:0/2727312987 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe940149200 0x7fe9400c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.958+0000 7fe93f7fe640 1 -- 172.21.15.76:0/2727312987 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe940149200 msgr2=0x7fe9400c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.959+0000 7fe93f7fe640 1 --2- 172.21.15.76:0/2727312987 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe940149200 0x7fe9400c9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.959+0000 7fe93f7fe640 1 -- 172.21.15.76:0/2727312987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe94015b8b0 msgr2=0x7fe94015bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:34.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.959+0000 7fe93f7fe640 1 --2- 172.21.15.76:0/2727312987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe94015b8b0 0x7fe94015bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:34.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.959+0000 7fe93f7fe640 1 -- 172.21.15.76:0/2727312987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9400c83b0 con 0x7fe9400b4540 2024-03-08T06:01:34.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.959+0000 7fe93f7fe640 1 --2- 172.21.15.76:0/2727312987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9400b4540 0x7fe9400b49a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe9340072c0 tx=0x7fe9340076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:34.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.959+0000 7fe93cff9640 1 -- 172.21.15.76:0/2727312987 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93400f9c0 con 0x7fe9400b4540 2024-03-08T06:01:34.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.959+0000 7fe93cff9640 1 -- 172.21.15.76:0/2727312987 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe934018070 con 0x7fe9400b4540 2024-03-08T06:01:34.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.959+0000 7fe946581640 1 -- 172.21.15.76:0/2727312987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9400c8690 con 0x7fe9400b4540 2024-03-08T06:01:34.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.959+0000 7fe93cff9640 1 -- 172.21.15.76:0/2727312987 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93400fde0 con 0x7fe9400b4540 2024-03-08T06:01:34.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.959+0000 7fe946581640 1 -- 172.21.15.76:0/2727312987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9400c8c50 con 0x7fe9400b4540 2024-03-08T06:01:34.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.960+0000 7fe946581640 1 -- 172.21.15.76:0/2727312987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94004f030 con 0x7fe9400b4540 2024-03-08T06:01:34.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.960+0000 7fe93cff9640 1 -- 172.21.15.76:0/2727312987 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe93401d460 con 0x7fe9400b4540 2024-03-08T06:01:34.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.961+0000 7fe93cff9640 1 --2- 172.21.15.76:0/2727312987 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe92c06b4d0 0x7fe92c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:34.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.961+0000 7fe93cff9640 1 -- 172.21.15.76:0/2727312987 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe93408a6e0 con 0x7fe9400b4540 2024-03-08T06:01:34.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.961+0000 7fe93ffff640 1 --2- 172.21.15.76:0/2727312987 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe92c06b4d0 0x7fe92c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:34.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.961+0000 7fe93ffff640 1 --2- 172.21.15.76:0/2727312987 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe92c06b4d0 0x7fe92c06d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe930009bc0 tx=0x7fe930013040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:34.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:34.965+0000 7fe93cff9640 1 -- 172.21.15.76:0/2727312987 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9340590d0 con 0x7fe9400b4540 2024-03-08T06:01:35.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.173+0000 7fe946581640 1 -- 172.21.15.76:0/2727312987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9400ca1e0 con 0x7fe9400b4540 2024-03-08T06:01:35.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.174+0000 7fe93cff9640 1 -- 172.21.15.76:0/2727312987 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7fe9340589d0 con 0x7fe9400b4540 2024-03-08T06:01:35.175 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:35.175 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4249563566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4249563566},{"type":"v1","addr":"172.21.15.76:6835","nonce":4249563566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1514641617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1514641617},{"type":"v1","addr":"172.21.15.76:6837","nonce":1514641617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6834/1332442802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1332442802},{"type":"v1","addr":"172.21.15.81:6834","nonce":1332442802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:00:35.504231+0000","modified":"2024-03-08T06:00:57.594023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6835/2357927814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":2357927814},{"type":"v1","addr":"172.21.15.81:6835","nonce":2357927814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T06:01:35.175 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-03-08T06:01:35.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.176+0000 7fe946581640 1 -- 172.21.15.76:0/2727312987 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe92c06b4d0 msgr2=0x7fe92c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.176+0000 7fe946581640 1 --2- 172.21.15.76:0/2727312987 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe92c06b4d0 0x7fe92c06d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe930009bc0 tx=0x7fe930013040 comp rx=0 tx=0).stop 2024-03-08T06:01:35.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.176+0000 7fe946581640 1 -- 172.21.15.76:0/2727312987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9400b4540 msgr2=0x7fe9400b49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.176+0000 7fe946581640 1 --2- 172.21.15.76:0/2727312987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9400b4540 0x7fe9400b49a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe9340072c0 tx=0x7fe9340076c0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.176+0000 7fe946581640 1 -- 172.21.15.76:0/2727312987 shutdown_connections 2024-03-08T06:01:35.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.176+0000 7fe946581640 1 --2- 172.21.15.76:0/2727312987 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe92c06b4d0 0x7fe92c06d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.176+0000 7fe946581640 1 --2- 172.21.15.76:0/2727312987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe94015b8b0 0x7fe94015bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.176+0000 7fe946581640 1 --2- 172.21.15.76:0/2727312987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9400b4540 0x7fe9400b49a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.176+0000 7fe946581640 1 --2- 172.21.15.76:0/2727312987 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe940149200 0x7fe9400c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.176+0000 7fe946581640 1 -- 172.21.15.76:0/2727312987 >> 172.21.15.76:0/2727312987 conn(0x7fe9400cf740 msgr2=0x7fe94006ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:35.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.176+0000 7fe946581640 1 -- 172.21.15.76:0/2727312987 shutdown_connections 2024-03-08T06:01:35.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.176+0000 7fe946581640 1 -- 172.21.15.76:0/2727312987 wait complete. 2024-03-08T06:01:35.188 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:35.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.287+0000 7f9db598e640 1 Processor -- start 2024-03-08T06:01:35.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.287+0000 7f9db598e640 1 -- start start 2024-03-08T06:01:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.288+0000 7f9db598e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9db015dde0 0x7f9db0162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.288+0000 7f9db598e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db00d0b90 0x7f9db00d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.288+0000 7f9db598e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db0162790 con 0x7f9db00b6d00 2024-03-08T06:01:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.288+0000 7f9db598e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db00cf180 con 0x7f9db015dde0 2024-03-08T06:01:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.288+0000 7f9db598e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db00d1ed0 con 0x7f9db00d0b90 2024-03-08T06:01:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.288+0000 7f9daf7fe640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db00d0b90 0x7f9db00d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.288+0000 7f9daeffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9db015dde0 0x7f9db0162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.288+0000 7f9daf7fe640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db00d0b90 0x7f9db00d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:56938/0 (socket says 172.21.15.76:56938) 2024-03-08T06:01:35.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.288+0000 7f9daf7fe640 1 -- 172.21.15.76:0/818660711 learned_addr learned my addr 172.21.15.76:0/818660711 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:35.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.289+0000 7f9daeffd640 1 -- 172.21.15.76:0/818660711 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db00d0b90 msgr2=0x7f9db00d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:35.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.289+0000 7f9daeffd640 1 --2- 172.21.15.76:0/818660711 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db00d0b90 0x7f9db00d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.289+0000 7f9daeffd640 1 -- 172.21.15.76:0/818660711 >> v1:172.21.15.81:6789/0 conn(0x7f9db00b6d00 legacy=0x7f9db00d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.289+0000 7f9daeffd640 1 -- 172.21.15.76:0/818660711 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db00d21b0 con 0x7f9db015dde0 2024-03-08T06:01:35.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.289+0000 7f9daeffd640 1 --2- 172.21.15.76:0/818660711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db015dde0 0x7f9db0162250 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9da00097a0 tx=0x7f9da002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a575538cddc30be server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:35.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.289+0000 7f9dadffb640 1 -- 172.21.15.76:0/818660711 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da00315a0 con 0x7f9db015dde0 2024-03-08T06:01:35.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.290+0000 7f9dadffb640 1 -- 172.21.15.76:0/818660711 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da0037070 con 0x7f9db015dde0 2024-03-08T06:01:35.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.290+0000 7f9dadffb640 1 -- 172.21.15.76:0/818660711 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0031960 con 0x7f9db015dde0 2024-03-08T06:01:35.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.290+0000 7f9db598e640 1 -- 172.21.15.76:0/818660711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db015dde0 msgr2=0x7f9db0162250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.290+0000 7f9db598e640 1 --2- 172.21.15.76:0/818660711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db015dde0 0x7f9db0162250 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9da00097a0 tx=0x7f9da002b840 comp rx=0 tx=0).stop 2024-03-08T06:01:35.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.290+0000 7f9db598e640 1 -- 172.21.15.76:0/818660711 shutdown_connections 2024-03-08T06:01:35.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.290+0000 7f9db598e640 1 --2- 172.21.15.76:0/818660711 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db00d0b90 0x7f9db00d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.290+0000 7f9db598e640 1 --2- 172.21.15.76:0/818660711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db015dde0 0x7f9db0162250 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.290+0000 7f9db598e640 1 -- 172.21.15.76:0/818660711 >> 172.21.15.76:0/818660711 conn(0x7f9db00b56e0 msgr2=0x7f9db00b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:35.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.290+0000 7f9db598e640 1 -- 172.21.15.76:0/818660711 shutdown_connections 2024-03-08T06:01:35.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.290+0000 7f9db598e640 1 -- 172.21.15.76:0/818660711 wait complete. 2024-03-08T06:01:35.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.291+0000 7f9db598e640 1 Processor -- start 2024-03-08T06:01:35.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.292+0000 7f9db598e640 1 -- start start 2024-03-08T06:01:35.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.292+0000 7f9db598e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db00b6d00 0x7f9db00c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.292+0000 7f9db598e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db00d0b90 0x7f9db00caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.292+0000 7f9db598e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db015dde0 0x7f9db00b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.292+0000 7f9db598e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db00d2ef0 con 0x7f9db00b6d00 2024-03-08T06:01:35.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.292+0000 7f9db598e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db00ceea0 con 0x7f9db015dde0 2024-03-08T06:01:35.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.292+0000 7f9db598e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db00d3070 con 0x7f9db00d0b90 2024-03-08T06:01:35.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.292+0000 7f9daeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db00b6d00 0x7f9db00c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.292+0000 7f9daeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db00b6d00 0x7f9db00c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53494/0 (socket says 172.21.15.76:53494) 2024-03-08T06:01:35.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.292+0000 7f9daeffd640 1 -- 172.21.15.76:0/388155158 learned_addr learned my addr 172.21.15.76:0/388155158 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:35.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.292+0000 7f9dae7fc640 1 --2- 172.21.15.76:0/388155158 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db00d0b90 0x7f9db00caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.292+0000 7f9daf7fe640 1 --2- 172.21.15.76:0/388155158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db015dde0 0x7f9db00b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.293+0000 7f9daeffd640 1 -- 172.21.15.76:0/388155158 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db00d0b90 msgr2=0x7f9db00caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.293+0000 7f9daeffd640 1 --2- 172.21.15.76:0/388155158 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db00d0b90 0x7f9db00caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.293+0000 7f9daeffd640 1 -- 172.21.15.76:0/388155158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db015dde0 msgr2=0x7f9db00b87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.293+0000 7f9daeffd640 1 --2- 172.21.15.76:0/388155158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db015dde0 0x7f9db00b87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.293+0000 7f9daeffd640 1 -- 172.21.15.76:0/388155158 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db015e360 con 0x7f9db00b6d00 2024-03-08T06:01:35.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.293+0000 7f9dae7fc640 1 --2- 172.21.15.76:0/388155158 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db00d0b90 0x7f9db00caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:35.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.293+0000 7f9daf7fe640 1 --2- 172.21.15.76:0/388155158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db015dde0 0x7f9db00b87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:35.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.293+0000 7f9daeffd640 1 --2- 172.21.15.76:0/388155158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db00b6d00 0x7f9db00c6700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9da00098c0 tx=0x7f9da00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:35.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.293+0000 7f9d97fff640 1 -- 172.21.15.76:0/388155158 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0032630 con 0x7f9db00b6d00 2024-03-08T06:01:35.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.293+0000 7f9d97fff640 1 -- 172.21.15.76:0/388155158 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da00327d0 con 0x7f9db00b6d00 2024-03-08T06:01:35.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.293+0000 7f9db598e640 1 -- 172.21.15.76:0/388155158 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db015e640 con 0x7f9db00b6d00 2024-03-08T06:01:35.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.293+0000 7f9d97fff640 1 -- 172.21.15.76:0/388155158 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0032bc0 con 0x7f9db00b6d00 2024-03-08T06:01:35.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.293+0000 7f9db598e640 1 -- 172.21.15.76:0/388155158 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db015ec00 con 0x7f9db00b6d00 2024-03-08T06:01:35.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.294+0000 7f9db598e640 1 -- 172.21.15.76:0/388155158 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db004f030 con 0x7f9db00b6d00 2024-03-08T06:01:35.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.294+0000 7f9d97fff640 1 -- 172.21.15.76:0/388155158 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9da004e430 con 0x7f9db00b6d00 2024-03-08T06:01:35.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.295+0000 7f9d97fff640 1 --2- 172.21.15.76:0/388155158 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9d9c06b4d0 0x7f9d9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.295+0000 7f9dae7fc640 1 --2- 172.21.15.76:0/388155158 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9d9c06b4d0 0x7f9d9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.295+0000 7f9d97fff640 1 -- 172.21.15.76:0/388155158 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f9da002f030 con 0x7f9db00b6d00 2024-03-08T06:01:35.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.295+0000 7f9dae7fc640 1 --2- 172.21.15.76:0/388155158 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9d9c06b4d0 0x7f9d9c06d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9db0161230 tx=0x7f9da4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:35.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.299+0000 7f9d97fff640 1 -- 172.21.15.76:0/388155158 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9da0040070 con 0x7f9db00b6d00 2024-03-08T06:01:35.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.472+0000 7f9db598e640 1 -- 172.21.15.76:0/388155158 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9db00d1870 con 0x7f9db00b6d00 2024-03-08T06:01:35.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.473+0000 7f9d97fff640 1 -- 172.21.15.76:0/388155158 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18757 (secure 0 0 0) 0x7f9da007aee0 con 0x7f9db00b6d00 2024-03-08T06:01:35.474 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:35.474 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":28,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:01:17.313272+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:35.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.475+0000 7f9db598e640 1 -- 172.21.15.76:0/388155158 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9d9c06b4d0 msgr2=0x7f9d9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.475+0000 7f9db598e640 1 --2- 172.21.15.76:0/388155158 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9d9c06b4d0 0x7f9d9c06d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9db0161230 tx=0x7f9da4009040 comp rx=0 tx=0).stop 2024-03-08T06:01:35.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.475+0000 7f9db598e640 1 -- 172.21.15.76:0/388155158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db00b6d00 msgr2=0x7f9db00c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.475+0000 7f9db598e640 1 --2- 172.21.15.76:0/388155158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db00b6d00 0x7f9db00c6700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9da00098c0 tx=0x7f9da00046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.475+0000 7f9db598e640 1 -- 172.21.15.76:0/388155158 shutdown_connections 2024-03-08T06:01:35.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.475+0000 7f9db598e640 1 --2- 172.21.15.76:0/388155158 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9d9c06b4d0 0x7f9d9c06d990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.475+0000 7f9db598e640 1 --2- 172.21.15.76:0/388155158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db015dde0 0x7f9db00b87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.475+0000 7f9db598e640 1 --2- 172.21.15.76:0/388155158 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9db00d0b90 0x7f9db00caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.475+0000 7f9db598e640 1 --2- 172.21.15.76:0/388155158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db00b6d00 0x7f9db00c6700 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.475+0000 7f9db598e640 1 -- 172.21.15.76:0/388155158 >> 172.21.15.76:0/388155158 conn(0x7f9db00b56e0 msgr2=0x7f9db00cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:35.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.476+0000 7f9db598e640 1 -- 172.21.15.76:0/388155158 shutdown_connections 2024-03-08T06:01:35.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.476+0000 7f9db598e640 1 -- 172.21.15.76:0/388155158 wait complete. 2024-03-08T06:01:35.487 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-08T06:01:35.487 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:01:35.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.587+0000 7f447fe84640 1 Processor -- start 2024-03-08T06:01:35.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.587+0000 7f447fe84640 1 -- start start 2024-03-08T06:01:35.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.588+0000 7f447fe84640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f44780cfa00 0x7f44780af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.588+0000 7f447fe84640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f447806eee0 0x7f44780caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.588+0000 7f447fe84640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44780d0150 con 0x7f447806ac80 2024-03-08T06:01:35.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.588+0000 7f447fe84640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44780aff00 con 0x7f44780cfa00 2024-03-08T06:01:35.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.588+0000 7f447fe84640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44780d0430 con 0x7f447806eee0 2024-03-08T06:01:35.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.588+0000 7f447dc20640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f447806ac80 0x7f447806b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46802/0 (socket says 172.21.15.76:46802) 2024-03-08T06:01:35.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.588+0000 7f447dc20640 1 -- 172.21.15.76:0/4255177217 learned_addr learned my addr 172.21.15.76:0/4255177217 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:35.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.588+0000 7f447e421640 1 --2- 172.21.15.76:0/4255177217 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f447806eee0 0x7f44780caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.588+0000 7f447d41f640 1 --2- 172.21.15.76:0/4255177217 >> v2:172.21.15.81:3300/0 conn(0x7f44780cfa00 0x7f44780af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447cc1e640 1 -- 172.21.15.76:0/4255177217 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882960531 0 0) 0x7f44780d0150 con 0x7f447806ac80 2024-03-08T06:01:35.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447cc1e640 1 -- 172.21.15.76:0/4255177217 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4464003520 con 0x7f447806ac80 2024-03-08T06:01:35.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447cc1e640 1 -- 172.21.15.76:0/4255177217 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3253475183 0 0) 0x7f4464003520 con 0x7f447806ac80 2024-03-08T06:01:35.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447cc1e640 1 -- 172.21.15.76:0/4255177217 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f447806eee0 msgr2=0x7f44780caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:35.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447cc1e640 1 --2- 172.21.15.76:0/4255177217 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f447806eee0 0x7f44780caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447cc1e640 1 -- 172.21.15.76:0/4255177217 >> v2:172.21.15.81:3300/0 conn(0x7f44780cfa00 msgr2=0x7f44780af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447cc1e640 1 --2- 172.21.15.76:0/4255177217 >> v2:172.21.15.81:3300/0 conn(0x7f44780cfa00 0x7f44780af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447cc1e640 1 -- 172.21.15.76:0/4255177217 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447806e6e0 con 0x7f447806ac80 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447cc1e640 1 -- 172.21.15.76:0/4255177217 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f44740019f0 con 0x7f447806ac80 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447cc1e640 1 -- 172.21.15.76:0/4255177217 >> v1:172.21.15.76:6789/0 conn(0x7f447806ac80 legacy=0x7f447806b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447cc1e640 1 --2- 172.21.15.76:0/4255177217 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f447806eee0 0x7f4464003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447cc1e640 1 --2- 172.21.15.76:0/4255177217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4464003fa0 0x7f4464004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447cc1e640 1 --2- 172.21.15.76:0/4255177217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4464004d40 0x7f44640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447cc1e640 1 -- 172.21.15.76:0/4255177217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44780aff00 con 0x7f4464004d40 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447cc1e640 1 -- 172.21.15.76:0/4255177217 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44780d0430 con 0x7f4464003fa0 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447cc1e640 1 -- 172.21.15.76:0/4255177217 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446400b9f0 con 0x7f447806eee0 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447e421640 1 --2- 172.21.15.76:0/4255177217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4464004d40 0x7f44640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447dc20640 1 --2- 172.21.15.76:0/4255177217 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f447806eee0 0x7f4464003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447d41f640 1 --2- 172.21.15.76:0/4255177217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4464003fa0 0x7f4464004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447e421640 1 -- 172.21.15.76:0/4255177217 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f447806eee0 msgr2=0x7f4464003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447e421640 1 --2- 172.21.15.76:0/4255177217 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f447806eee0 0x7f4464003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447e421640 1 -- 172.21.15.76:0/4255177217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4464003fa0 msgr2=0x7f4464004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:35.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447e421640 1 --2- 172.21.15.76:0/4255177217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4464003fa0 0x7f4464004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.589+0000 7f447e421640 1 -- 172.21.15.76:0/4255177217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4464003520 con 0x7f4464004d40 2024-03-08T06:01:35.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.590+0000 7f447e421640 1 --2- 172.21.15.76:0/4255177217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4464004d40 0x7f44640051c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4470009ac0 tx=0x7f4464005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb0a226c409e6dd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:35.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.590+0000 7f447cc1e640 1 -- 172.21.15.76:0/4255177217 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470039070 con 0x7f4464004d40 2024-03-08T06:01:35.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.590+0000 7f447cc1e640 1 -- 172.21.15.76:0/4255177217 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447002c6b0 con 0x7f4464004d40 2024-03-08T06:01:35.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.590+0000 7f447cc1e640 1 -- 172.21.15.76:0/4255177217 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470009690 con 0x7f4464004d40 2024-03-08T06:01:35.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.591+0000 7f447fe84640 1 -- 172.21.15.76:0/4255177217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4464004d40 msgr2=0x7f44640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.591+0000 7f447fe84640 1 --2- 172.21.15.76:0/4255177217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4464004d40 0x7f44640051c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4470009ac0 tx=0x7f4464005830 comp rx=0 tx=0).stop 2024-03-08T06:01:35.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.591+0000 7f447e421640 1 -- 172.21.15.76:0/4255177217 reap_dead start 2024-03-08T06:01:35.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.591+0000 7f447fe84640 1 -- 172.21.15.76:0/4255177217 shutdown_connections 2024-03-08T06:01:35.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.591+0000 7f447fe84640 1 -- 172.21.15.76:0/4255177217 >> 172.21.15.76:0/4255177217 conn(0x7f4478099a60 msgr2=0x7f44780d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:35.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.591+0000 7f447fe84640 1 -- 172.21.15.76:0/4255177217 shutdown_connections 2024-03-08T06:01:35.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.591+0000 7f447fe84640 1 -- 172.21.15.76:0/4255177217 wait complete. 2024-03-08T06:01:35.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.592+0000 7f447fe84640 1 Processor -- start 2024-03-08T06:01:35.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.592+0000 7f447fe84640 1 -- start start 2024-03-08T06:01:35.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447fe84640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f447814ce40 0x7f44781599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447fe84640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f447814d220 0x7f4478159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447fe84640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44780b1be0 0x7f44780b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447fe84640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447815e280 con 0x7f44780b1be0 2024-03-08T06:01:35.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447fe84640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44780cfe60 con 0x7f447814d220 2024-03-08T06:01:35.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447fe84640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447806f370 con 0x7f447814ce40 2024-03-08T06:01:35.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447e421640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44780b1be0 0x7f44780b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447e421640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44780b1be0 0x7f44780b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53506/0 (socket says 172.21.15.76:53506) 2024-03-08T06:01:35.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447e421640 1 -- 172.21.15.76:0/1629967070 learned_addr learned my addr 172.21.15.76:0/1629967070 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:35.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447d41f640 1 --2- 172.21.15.76:0/1629967070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f447814d220 0x7f4478159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447dc20640 1 --2- 172.21.15.76:0/1629967070 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f447814ce40 0x7f44781599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447e421640 1 -- 172.21.15.76:0/1629967070 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f447814ce40 msgr2=0x7f44781599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447e421640 1 --2- 172.21.15.76:0/1629967070 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f447814ce40 0x7f44781599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447e421640 1 -- 172.21.15.76:0/1629967070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f447814d220 msgr2=0x7f4478159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447e421640 1 --2- 172.21.15.76:0/1629967070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f447814d220 0x7f4478159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447e421640 1 -- 172.21.15.76:0/1629967070 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780b28d0 con 0x7f44780b1be0 2024-03-08T06:01:35.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447dc20640 1 --2- 172.21.15.76:0/1629967070 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f447814ce40 0x7f44781599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:35.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447e421640 1 --2- 172.21.15.76:0/1629967070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44780b1be0 0x7f44780b2090 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4470009ac0 tx=0x7f4470004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:35.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f446e7fc640 1 -- 172.21.15.76:0/1629967070 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470039070 con 0x7f44780b1be0 2024-03-08T06:01:35.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f446e7fc640 1 -- 172.21.15.76:0/1629967070 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4470043070 con 0x7f44780b1be0 2024-03-08T06:01:35.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447fe84640 1 -- 172.21.15.76:0/1629967070 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44780b2bb0 con 0x7f44780b1be0 2024-03-08T06:01:35.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f446e7fc640 1 -- 172.21.15.76:0/1629967070 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470009690 con 0x7f44780b1be0 2024-03-08T06:01:35.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.593+0000 7f447fe84640 1 -- 172.21.15.76:0/1629967070 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44780c43e0 con 0x7f44780b1be0 2024-03-08T06:01:35.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.594+0000 7f447fe84640 1 -- 172.21.15.76:0/1629967070 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447804f030 con 0x7f44780b1be0 2024-03-08T06:01:35.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.595+0000 7f446e7fc640 1 -- 172.21.15.76:0/1629967070 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4470003830 con 0x7f44780b1be0 2024-03-08T06:01:35.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.595+0000 7f446e7fc640 1 --2- 172.21.15.76:0/1629967070 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f445c06b4d0 0x7f445c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.595+0000 7f446e7fc640 1 -- 172.21.15.76:0/1629967070 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f44700ab170 con 0x7f44780b1be0 2024-03-08T06:01:35.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.595+0000 7f447dc20640 1 --2- 172.21.15.76:0/1629967070 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f445c06b4d0 0x7f445c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.596+0000 7f447dc20640 1 --2- 172.21.15.76:0/1629967070 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f445c06b4d0 0x7f445c06d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f44780b1b70 tx=0x7f4474006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:35.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.599+0000 7f446e7fc640 1 -- 172.21.15.76:0/1629967070 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4470079d40 con 0x7f44780b1be0 2024-03-08T06:01:35.757 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:01:35.759 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:01:35.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.808+0000 7f447fe84640 1 -- 172.21.15.76:0/1629967070 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44780cd040 con 0x7f44780b1be0 2024-03-08T06:01:35.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.808+0000 7f446e7fc640 1 -- 172.21.15.76:0/1629967070 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f4470079640 con 0x7f44780b1be0 2024-03-08T06:01:35.809 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:35.809 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4249563566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4249563566},{"type":"v1","addr":"172.21.15.76:6835","nonce":4249563566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1514641617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1514641617},{"type":"v1","addr":"172.21.15.76:6837","nonce":1514641617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6834/1332442802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1332442802},{"type":"v1","addr":"172.21.15.81:6834","nonce":1332442802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:00:35.504231+0000","modified":"2024-03-08T06:00:57.594023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6835/2357927814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":2357927814},{"type":"v1","addr":"172.21.15.81:6835","nonce":2357927814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T06:01:35.809 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-03-08T06:01:35.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.810+0000 7f447fe84640 1 -- 172.21.15.76:0/1629967070 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f445c06b4d0 msgr2=0x7f445c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.810+0000 7f447fe84640 1 --2- 172.21.15.76:0/1629967070 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f445c06b4d0 0x7f445c06d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f44780b1b70 tx=0x7f4474006040 comp rx=0 tx=0).stop 2024-03-08T06:01:35.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.810+0000 7f447fe84640 1 -- 172.21.15.76:0/1629967070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44780b1be0 msgr2=0x7f44780b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.810+0000 7f447fe84640 1 --2- 172.21.15.76:0/1629967070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44780b1be0 0x7f44780b2090 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4470009ac0 tx=0x7f4470004730 comp rx=0 tx=0).stop 2024-03-08T06:01:35.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.810+0000 7f447fe84640 1 -- 172.21.15.76:0/1629967070 shutdown_connections 2024-03-08T06:01:35.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.810+0000 7f447fe84640 1 --2- 172.21.15.76:0/1629967070 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f445c06b4d0 0x7f445c06d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.810+0000 7f447fe84640 1 --2- 172.21.15.76:0/1629967070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44780b1be0 0x7f44780b2090 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.810+0000 7f447fe84640 1 --2- 172.21.15.76:0/1629967070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f447814d220 0x7f4478159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.810+0000 7f447fe84640 1 --2- 172.21.15.76:0/1629967070 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f447814ce40 0x7f44781599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.810+0000 7f447fe84640 1 -- 172.21.15.76:0/1629967070 >> 172.21.15.76:0/1629967070 conn(0x7f4478099a60 msgr2=0x7f44780ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:35.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.810+0000 7f447fe84640 1 -- 172.21.15.76:0/1629967070 shutdown_connections 2024-03-08T06:01:35.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.810+0000 7f447fe84640 1 -- 172.21.15.76:0/1629967070 wait complete. 2024-03-08T06:01:35.822 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:01:35.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.962+0000 7fc928215640 1 Processor -- start 2024-03-08T06:01:35.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.963+0000 7fc928215640 1 -- start start 2024-03-08T06:01:35.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.963+0000 7fc928215640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc92006ac80 0x7fc9200cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.963+0000 7fc928215640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc9200af9c0 0x7fc92006b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.963+0000 7fc928215640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92006b5a0 con 0x7fc92006eee0 2024-03-08T06:01:35.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.963+0000 7fc928215640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9200d0150 con 0x7fc92006ac80 2024-03-08T06:01:35.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.963+0000 7fc928215640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9200d0430 con 0x7fc9200af9c0 2024-03-08T06:01:35.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.964+0000 7fc925fb1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc92006ac80 0x7fc9200cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.964+0000 7fc9267b2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc92006eee0 0x7fc9200caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46818/0 (socket says 172.21.15.76:46818) 2024-03-08T06:01:35.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.964+0000 7fc9267b2640 1 -- 172.21.15.76:0/4136833362 learned_addr learned my addr 172.21.15.76:0/4136833362 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:35.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.964+0000 7fc9257b0640 1 --2- 172.21.15.76:0/4136833362 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc9200af9c0 0x7fc92006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.964+0000 7fc925fb1640 1 -- 172.21.15.76:0/4136833362 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc9200af9c0 msgr2=0x7fc92006b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:35.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.964+0000 7fc925fb1640 1 --2- 172.21.15.76:0/4136833362 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc9200af9c0 0x7fc92006b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.964+0000 7fc925fb1640 1 -- 172.21.15.76:0/4136833362 >> v1:172.21.15.76:6789/0 conn(0x7fc92006eee0 legacy=0x7fc9200caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.964+0000 7fc924faf640 1 -- 172.21.15.76:0/4136833362 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916052124 0 0) 0x7fc92006b5a0 con 0x7fc92006eee0 2024-03-08T06:01:35.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.964+0000 7fc925fb1640 1 -- 172.21.15.76:0/4136833362 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92006e6e0 con 0x7fc92006ac80 2024-03-08T06:01:35.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.964+0000 7fc9257b0640 1 --2- 172.21.15.76:0/4136833362 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc9200af9c0 0x7fc92006b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:35.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.965+0000 7fc925fb1640 1 --2- 172.21.15.76:0/4136833362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92006ac80 0x7fc9200cfa00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc9100097a0 tx=0x7fc91002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef2b804c6557a7f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:35.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.965+0000 7fc924faf640 1 -- 172.21.15.76:0/4136833362 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc910035020 con 0x7fc92006ac80 2024-03-08T06:01:35.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.965+0000 7fc924faf640 1 -- 172.21.15.76:0/4136833362 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc910031830 con 0x7fc92006ac80 2024-03-08T06:01:35.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.965+0000 7fc924faf640 1 -- 172.21.15.76:0/4136833362 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc910031c00 con 0x7fc92006ac80 2024-03-08T06:01:35.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.965+0000 7fc928215640 1 -- 172.21.15.76:0/4136833362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92006ac80 msgr2=0x7fc9200cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.965+0000 7fc928215640 1 --2- 172.21.15.76:0/4136833362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92006ac80 0x7fc9200cfa00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc9100097a0 tx=0x7fc91002b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.965+0000 7fc928215640 1 -- 172.21.15.76:0/4136833362 shutdown_connections 2024-03-08T06:01:35.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.965+0000 7fc928215640 1 --2- 172.21.15.76:0/4136833362 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc9200af9c0 0x7fc92006b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.965+0000 7fc928215640 1 --2- 172.21.15.76:0/4136833362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92006ac80 0x7fc9200cfa00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.965+0000 7fc928215640 1 -- 172.21.15.76:0/4136833362 >> 172.21.15.76:0/4136833362 conn(0x7fc920099a60 msgr2=0x7fc9200d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:35.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.965+0000 7fc928215640 1 -- 172.21.15.76:0/4136833362 shutdown_connections 2024-03-08T06:01:35.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.965+0000 7fc928215640 1 -- 172.21.15.76:0/4136833362 wait complete. 2024-03-08T06:01:35.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.966+0000 7fc928215640 1 Processor -- start 2024-03-08T06:01:35.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.967+0000 7fc928215640 1 -- start start 2024-03-08T06:01:35.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.967+0000 7fc928215640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc92006ac80 0x7fc9200c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.967+0000 7fc928215640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92006eee0 0x7fc9200b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.967+0000 7fc928215640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9200af9c0 0x7fc9200b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.967+0000 7fc928215640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92015e280 con 0x7fc92006eee0 2024-03-08T06:01:35.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.967+0000 7fc928215640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9200afe20 con 0x7fc9200af9c0 2024-03-08T06:01:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.967+0000 7fc928215640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9200cb5b0 con 0x7fc92006ac80 2024-03-08T06:01:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.967+0000 7fc9257b0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92006eee0 0x7fc9200b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.967+0000 7fc9257b0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92006eee0 0x7fc9200b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53534/0 (socket says 172.21.15.76:53534) 2024-03-08T06:01:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.967+0000 7fc9257b0640 1 -- 172.21.15.76:0/3532821660 learned_addr learned my addr 172.21.15.76:0/3532821660 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.967+0000 7fc925fb1640 1 --2- 172.21.15.76:0/3532821660 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc92006ac80 0x7fc9200c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.968+0000 7fc9267b2640 1 --2- 172.21.15.76:0/3532821660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9200af9c0 0x7fc9200b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.968+0000 7fc9257b0640 1 -- 172.21.15.76:0/3532821660 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc92006ac80 msgr2=0x7fc9200c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.968+0000 7fc9257b0640 1 --2- 172.21.15.76:0/3532821660 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc92006ac80 0x7fc9200c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.968+0000 7fc9257b0640 1 -- 172.21.15.76:0/3532821660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9200af9c0 msgr2=0x7fc9200b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.968+0000 7fc9257b0640 1 --2- 172.21.15.76:0/3532821660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9200af9c0 0x7fc9200b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.968+0000 7fc9257b0640 1 -- 172.21.15.76:0/3532821660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9200b27c0 con 0x7fc92006eee0 2024-03-08T06:01:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.968+0000 7fc9257b0640 1 --2- 172.21.15.76:0/3532821660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92006eee0 0x7fc9200b4c90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc91c00c820 tx=0x7fc91c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.968+0000 7fc9167fc640 1 -- 172.21.15.76:0/3532821660 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c015070 con 0x7fc92006eee0 2024-03-08T06:01:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.968+0000 7fc9167fc640 1 -- 172.21.15.76:0/3532821660 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91c007a30 con 0x7fc92006eee0 2024-03-08T06:01:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.968+0000 7fc928215640 1 -- 172.21.15.76:0/3532821660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9200b2a60 con 0x7fc92006eee0 2024-03-08T06:01:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.968+0000 7fc9167fc640 1 -- 172.21.15.76:0/3532821660 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c007de0 con 0x7fc92006eee0 2024-03-08T06:01:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.968+0000 7fc928215640 1 -- 172.21.15.76:0/3532821660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9200b3020 con 0x7fc92006eee0 2024-03-08T06:01:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.969+0000 7fc928215640 1 -- 172.21.15.76:0/3532821660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92004f030 con 0x7fc92006eee0 2024-03-08T06:01:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.969+0000 7fc9167fc640 1 -- 172.21.15.76:0/3532821660 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc91c018070 con 0x7fc92006eee0 2024-03-08T06:01:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.970+0000 7fc9167fc640 1 --2- 172.21.15.76:0/3532821660 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc90406b500 0x7fc90406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.970+0000 7fc9167fc640 1 -- 172.21.15.76:0/3532821660 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc91c08a420 con 0x7fc92006eee0 2024-03-08T06:01:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.970+0000 7fc925fb1640 1 --2- 172.21.15.76:0/3532821660 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc90406b500 0x7fc90406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.970+0000 7fc925fb1640 1 --2- 172.21.15.76:0/3532821660 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc90406b500 0x7fc90406d9c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc910036000 tx=0x7fc910008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:35.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:35.974+0000 7fc9167fc640 1 -- 172.21.15.76:0/3532821660 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc91c058f30 con 0x7fc92006eee0 2024-03-08T06:01:36.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.180+0000 7fc928215640 1 -- 172.21.15.76:0/3532821660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9200cd040 con 0x7fc92006eee0 2024-03-08T06:01:36.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.181+0000 7fc9167fc640 1 -- 172.21.15.76:0/3532821660 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7fc91c058830 con 0x7fc92006eee0 2024-03-08T06:01:36.181 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:36.181 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4249563566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4249563566},{"type":"v1","addr":"172.21.15.76:6835","nonce":4249563566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1514641617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1514641617},{"type":"v1","addr":"172.21.15.76:6837","nonce":1514641617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6834/1332442802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1332442802},{"type":"v1","addr":"172.21.15.81:6834","nonce":1332442802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:00:35.504231+0000","modified":"2024-03-08T06:00:57.594023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6835/2357927814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":2357927814},{"type":"v1","addr":"172.21.15.81:6835","nonce":2357927814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T06:01:36.181 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-03-08T06:01:36.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.183+0000 7fc928215640 1 -- 172.21.15.76:0/3532821660 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc90406b500 msgr2=0x7fc90406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.183+0000 7fc928215640 1 --2- 172.21.15.76:0/3532821660 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc90406b500 0x7fc90406d9c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc910036000 tx=0x7fc910008030 comp rx=0 tx=0).stop 2024-03-08T06:01:36.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.183+0000 7fc928215640 1 -- 172.21.15.76:0/3532821660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92006eee0 msgr2=0x7fc9200b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.183+0000 7fc928215640 1 --2- 172.21.15.76:0/3532821660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92006eee0 0x7fc9200b4c90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc91c00c820 tx=0x7fc91c00cce0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.183+0000 7fc928215640 1 -- 172.21.15.76:0/3532821660 shutdown_connections 2024-03-08T06:01:36.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.183+0000 7fc928215640 1 --2- 172.21.15.76:0/3532821660 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc90406b500 0x7fc90406d9c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.183+0000 7fc928215640 1 --2- 172.21.15.76:0/3532821660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9200af9c0 0x7fc9200b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.183+0000 7fc928215640 1 --2- 172.21.15.76:0/3532821660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92006eee0 0x7fc9200b4c90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.183+0000 7fc928215640 1 --2- 172.21.15.76:0/3532821660 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc92006ac80 0x7fc9200c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.183+0000 7fc928215640 1 -- 172.21.15.76:0/3532821660 >> 172.21.15.76:0/3532821660 conn(0x7fc920099a60 msgr2=0x7fc9200ad600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:36.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.183+0000 7fc928215640 1 -- 172.21.15.76:0/3532821660 shutdown_connections 2024-03-08T06:01:36.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.183+0000 7fc928215640 1 -- 172.21.15.76:0/3532821660 wait complete. 2024-03-08T06:01:36.194 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:36.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.295+0000 7f12f9ed4640 1 Processor -- start 2024-03-08T06:01:36.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.295+0000 7f12f9ed4640 1 -- start start 2024-03-08T06:01:36.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.296+0000 7f12f9ed4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12f4071ee0 0x7f12f40b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.296+0000 7f12f9ed4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f12f40722c0 0x7f12f415d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.296+0000 7f12f9ed4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f40b3450 con 0x7f12f406c300 2024-03-08T06:01:36.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.296+0000 7f12f9ed4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f406b730 con 0x7f12f40722c0 2024-03-08T06:01:36.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.296+0000 7f12f9ed4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f406ba10 con 0x7f12f4071ee0 2024-03-08T06:01:36.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.296+0000 7f12f3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f12f40722c0 0x7f12f415d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.296+0000 7f12f2ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f12f406c300 0x7f12f406dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46820/0 (socket says 172.21.15.76:46820) 2024-03-08T06:01:36.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.296+0000 7f12f2ffd640 1 -- 172.21.15.76:0/3691078004 learned_addr learned my addr 172.21.15.76:0/3691078004 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:36.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.296+0000 7f12f37fe640 1 --2- 172.21.15.76:0/3691078004 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12f4071ee0 0x7f12f40b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.296+0000 7f12f3fff640 1 -- 172.21.15.76:0/3691078004 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12f4071ee0 msgr2=0x7f12f40b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:36.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.296+0000 7f12f3fff640 1 --2- 172.21.15.76:0/3691078004 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12f4071ee0 0x7f12f40b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.296+0000 7f12f3fff640 1 -- 172.21.15.76:0/3691078004 >> v1:172.21.15.76:6789/0 conn(0x7f12f406c300 legacy=0x7f12f406dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.296+0000 7f12f3fff640 1 -- 172.21.15.76:0/3691078004 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f406e2b0 con 0x7f12f40722c0 2024-03-08T06:01:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.297+0000 7f12f3fff640 1 --2- 172.21.15.76:0/3691078004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12f40722c0 0x7f12f415d850 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009d90 tx=0x7f12ec02bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f37874bb05bb79 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.297+0000 7f12f27fc640 1 -- 172.21.15.76:0/3691078004 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec039070 con 0x7f12f40722c0 2024-03-08T06:01:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.297+0000 7f12f27fc640 1 -- 172.21.15.76:0/3691078004 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec02c980 con 0x7f12f40722c0 2024-03-08T06:01:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.297+0000 7f12f27fc640 1 -- 172.21.15.76:0/3691078004 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec02cd50 con 0x7f12f40722c0 2024-03-08T06:01:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.298+0000 7f12f9ed4640 1 -- 172.21.15.76:0/3691078004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12f40722c0 msgr2=0x7f12f415d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.298+0000 7f12f9ed4640 1 --2- 172.21.15.76:0/3691078004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12f40722c0 0x7f12f415d850 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009d90 tx=0x7f12ec02bc50 comp rx=0 tx=0).stop 2024-03-08T06:01:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.298+0000 7f12f9ed4640 1 -- 172.21.15.76:0/3691078004 shutdown_connections 2024-03-08T06:01:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.298+0000 7f12f9ed4640 1 --2- 172.21.15.76:0/3691078004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12f40722c0 0x7f12f415d850 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.298+0000 7f12f9ed4640 1 --2- 172.21.15.76:0/3691078004 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12f4071ee0 0x7f12f40b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.298+0000 7f12f9ed4640 1 -- 172.21.15.76:0/3691078004 >> 172.21.15.76:0/3691078004 conn(0x7f12f40af650 msgr2=0x7f12f40b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.298+0000 7f12f9ed4640 1 -- 172.21.15.76:0/3691078004 shutdown_connections 2024-03-08T06:01:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.298+0000 7f12f9ed4640 1 -- 172.21.15.76:0/3691078004 wait complete. 2024-03-08T06:01:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.299+0000 7f12f9ed4640 1 Processor -- start 2024-03-08T06:01:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.299+0000 7f12f9ed4640 1 -- start start 2024-03-08T06:01:36.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f9ed4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12f406c300 0x7f12f41418e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f9ed4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12f4071ee0 0x7f12f415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f9ed4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12f40722c0 0x7f12f40b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f9ed4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f415dd90 con 0x7f12f4071ee0 2024-03-08T06:01:36.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f9ed4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f40ad090 con 0x7f12f406c300 2024-03-08T06:01:36.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f9ed4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f415df10 con 0x7f12f40722c0 2024-03-08T06:01:36.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12f4071ee0 0x7f12f415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12f4071ee0 0x7f12f415f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53554/0 (socket says 172.21.15.76:53554) 2024-03-08T06:01:36.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f2ffd640 1 -- 172.21.15.76:0/938447899 learned_addr learned my addr 172.21.15.76:0/938447899 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:36.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f3fff640 1 --2- 172.21.15.76:0/938447899 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12f40722c0 0x7f12f40b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f37fe640 1 --2- 172.21.15.76:0/938447899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12f406c300 0x7f12f41418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f2ffd640 1 -- 172.21.15.76:0/938447899 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12f40722c0 msgr2=0x7f12f40b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f2ffd640 1 --2- 172.21.15.76:0/938447899 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12f40722c0 0x7f12f40b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f2ffd640 1 -- 172.21.15.76:0/938447899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12f406c300 msgr2=0x7f12f41418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f2ffd640 1 --2- 172.21.15.76:0/938447899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12f406c300 0x7f12f41418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f2ffd640 1 -- 172.21.15.76:0/938447899 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f415fd10 con 0x7f12f4071ee0 2024-03-08T06:01:36.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f2ffd640 1 --2- 172.21.15.76:0/938447899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12f4071ee0 0x7f12f415f3c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f12e800c780 tx=0x7f12e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:36.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12dbfff640 1 -- 172.21.15.76:0/938447899 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e8018070 con 0x7f12f4071ee0 2024-03-08T06:01:36.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12dbfff640 1 -- 172.21.15.76:0/938447899 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e80072d0 con 0x7f12f4071ee0 2024-03-08T06:01:36.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f9ed4640 1 -- 172.21.15.76:0/938447899 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f415fff0 con 0x7f12f4071ee0 2024-03-08T06:01:36.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12dbfff640 1 -- 172.21.15.76:0/938447899 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e80047e0 con 0x7f12f4071ee0 2024-03-08T06:01:36.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.300+0000 7f12f9ed4640 1 -- 172.21.15.76:0/938447899 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f41605b0 con 0x7f12f4071ee0 2024-03-08T06:01:36.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.301+0000 7f12f9ed4640 1 -- 172.21.15.76:0/938447899 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f404f030 con 0x7f12f4071ee0 2024-03-08T06:01:36.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.302+0000 7f12dbfff640 1 -- 172.21.15.76:0/938447899 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f12e801b070 con 0x7f12f4071ee0 2024-03-08T06:01:36.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.302+0000 7f12dbfff640 1 --2- 172.21.15.76:0/938447899 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f12e006b4d0 0x7f12e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.302+0000 7f12dbfff640 1 -- 172.21.15.76:0/938447899 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f12e80899e0 con 0x7f12f4071ee0 2024-03-08T06:01:36.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.302+0000 7f12f37fe640 1 --2- 172.21.15.76:0/938447899 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f12e006b4d0 0x7f12e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.303+0000 7f12f37fe640 1 --2- 172.21.15.76:0/938447899 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f12e006b4d0 0x7f12e006d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009b40 tx=0x7f12e4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:36.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.307+0000 7f12dbfff640 1 -- 172.21.15.76:0/938447899 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f12e8057e10 con 0x7f12f4071ee0 2024-03-08T06:01:36.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.482+0000 7f12f9ed4640 1 -- 172.21.15.76:0/938447899 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12f415ad80 con 0x7f12f4071ee0 2024-03-08T06:01:36.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.483+0000 7f12dbfff640 1 -- 172.21.15.76:0/938447899 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18757 (secure 0 0 0) 0x7f12e805beb0 con 0x7f12f4071ee0 2024-03-08T06:01:36.484 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:36.485 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":28,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:01:17.313272+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:00:33.926651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:00:34.474030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T06:00:38.207853+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:36.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.485+0000 7f12f9ed4640 1 -- 172.21.15.76:0/938447899 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f12e006b4d0 msgr2=0x7f12e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.485+0000 7f12f9ed4640 1 --2- 172.21.15.76:0/938447899 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f12e006b4d0 0x7f12e006d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009b40 tx=0x7f12e4009190 comp rx=0 tx=0).stop 2024-03-08T06:01:36.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.485+0000 7f12f9ed4640 1 -- 172.21.15.76:0/938447899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12f4071ee0 msgr2=0x7f12f415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.485+0000 7f12f9ed4640 1 --2- 172.21.15.76:0/938447899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12f4071ee0 0x7f12f415f3c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f12e800c780 tx=0x7f12e800cc40 comp rx=0 tx=0).stop 2024-03-08T06:01:36.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.485+0000 7f12f9ed4640 1 -- 172.21.15.76:0/938447899 shutdown_connections 2024-03-08T06:01:36.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.485+0000 7f12f9ed4640 1 --2- 172.21.15.76:0/938447899 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f12e006b4d0 0x7f12e006d990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.485+0000 7f12f9ed4640 1 --2- 172.21.15.76:0/938447899 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12f40722c0 0x7f12f40b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.485+0000 7f12f9ed4640 1 --2- 172.21.15.76:0/938447899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12f4071ee0 0x7f12f415f3c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.486+0000 7f12f9ed4640 1 --2- 172.21.15.76:0/938447899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12f406c300 0x7f12f41418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.486+0000 7f12f9ed4640 1 -- 172.21.15.76:0/938447899 >> 172.21.15.76:0/938447899 conn(0x7f12f40af650 msgr2=0x7f12f415c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:36.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.486+0000 7f12f9ed4640 1 -- 172.21.15.76:0/938447899 shutdown_connections 2024-03-08T06:01:36.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.486+0000 7f12f9ed4640 1 -- 172.21.15.76:0/938447899 wait complete. 2024-03-08T06:01:36.497 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-08T06:01:36.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.598+0000 7f41c5239640 1 Processor -- start 2024-03-08T06:01:36.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.598+0000 7f41c5239640 1 -- start start 2024-03-08T06:01:36.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.599+0000 7f41c5239640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f41c015f5c0 0x7f41c0071890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.599+0000 7f41c5239640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41c015c600 0x7f41c015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.599+0000 7f41c5239640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41c015fc10 con 0x7f41c0159770 2024-03-08T06:01:36.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.599+0000 7f41c5239640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41c0071dd0 con 0x7f41c015f5c0 2024-03-08T06:01:36.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.599+0000 7f41c5239640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c015c170 con 0x7f41c015c600 2024-03-08T06:01:36.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.599+0000 7f41bed9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f41c0159770 0x7f41c00cf4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46836/0 (socket says 172.21.15.76:46836) 2024-03-08T06:01:36.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.599+0000 7f41bed9d640 1 -- 172.21.15.76:0/524002166 learned_addr learned my addr 172.21.15.76:0/524002166 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:36.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.599+0000 7f41bf59e640 1 --2- 172.21.15.76:0/524002166 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41c015c600 0x7f41c015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.599+0000 7f41be59c640 1 --2- 172.21.15.76:0/524002166 >> v2:172.21.15.81:3300/0 conn(0x7f41c015f5c0 0x7f41c0071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.599+0000 7f41bdd9b640 1 -- 172.21.15.76:0/524002166 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276403319 0 0) 0x7f41c015fc10 con 0x7f41c0159770 2024-03-08T06:01:36.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.599+0000 7f41bdd9b640 1 -- 172.21.15.76:0/524002166 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41ac003520 con 0x7f41c0159770 2024-03-08T06:01:36.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.599+0000 7f41bf59e640 1 -- 172.21.15.76:0/524002166 >> v2:172.21.15.81:3300/0 conn(0x7f41c015f5c0 msgr2=0x7f41c0071890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:36.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.599+0000 7f41bf59e640 1 --2- 172.21.15.76:0/524002166 >> v2:172.21.15.81:3300/0 conn(0x7f41c015f5c0 0x7f41c0071890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.599+0000 7f41bf59e640 1 -- 172.21.15.76:0/524002166 >> v1:172.21.15.76:6789/0 conn(0x7f41c0159770 legacy=0x7f41c00cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.599+0000 7f41bf59e640 1 -- 172.21.15.76:0/524002166 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c015fef0 con 0x7f41c015c600 2024-03-08T06:01:36.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.599+0000 7f41be59c640 1 --2- 172.21.15.76:0/524002166 >> v2:172.21.15.81:3300/0 conn(0x7f41c015f5c0 0x7f41c0071890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:36.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.600+0000 7f41bf59e640 1 --2- 172.21.15.76:0/524002166 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41c015c600 0x7f41c015ca90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009d90 tx=0x7f41b002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a241ad5902b35a5a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:36.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.600+0000 7f41bdd9b640 1 -- 172.21.15.76:0/524002166 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b0039070 con 0x7f41c015c600 2024-03-08T06:01:36.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.600+0000 7f41bdd9b640 1 -- 172.21.15.76:0/524002166 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b002cc50 con 0x7f41c015c600 2024-03-08T06:01:36.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.600+0000 7f41bdd9b640 1 -- 172.21.15.76:0/524002166 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b0034440 con 0x7f41c015c600 2024-03-08T06:01:36.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.601+0000 7f41c5239640 1 -- 172.21.15.76:0/524002166 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41c015c600 msgr2=0x7f41c015ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.601+0000 7f41c5239640 1 --2- 172.21.15.76:0/524002166 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41c015c600 0x7f41c015ca90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009d90 tx=0x7f41b002bea0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.601+0000 7f41c5239640 1 -- 172.21.15.76:0/524002166 shutdown_connections 2024-03-08T06:01:36.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.601+0000 7f41c5239640 1 --2- 172.21.15.76:0/524002166 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41c015c600 0x7f41c015ca90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.601+0000 7f41c5239640 1 --2- 172.21.15.76:0/524002166 >> v2:172.21.15.81:3300/0 conn(0x7f41c015f5c0 0x7f41c0071890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.601+0000 7f41c5239640 1 -- 172.21.15.76:0/524002166 >> 172.21.15.76:0/524002166 conn(0x7f41c006f530 msgr2=0x7f41c006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:36.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.601+0000 7f41c5239640 1 -- 172.21.15.76:0/524002166 shutdown_connections 2024-03-08T06:01:36.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.601+0000 7f41c5239640 1 -- 172.21.15.76:0/524002166 wait complete. 2024-03-08T06:01:36.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.602+0000 7f41c5239640 1 Processor -- start 2024-03-08T06:01:36.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.602+0000 7f41c5239640 1 -- start start 2024-03-08T06:01:36.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.603+0000 7f41c5239640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41c0159770 0x7f41c01f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.603+0000 7f41c5239640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41c015c600 0x7f41c014eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.603+0000 7f41c5239640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41c015f5c0 0x7f41c00b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.603+0000 7f41c5239640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c00cfbb0 con 0x7f41c0159770 2024-03-08T06:01:36.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.603+0000 7f41c5239640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c01604b0 con 0x7f41c015c600 2024-03-08T06:01:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.603+0000 7f41c5239640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c00ce250 con 0x7f41c015f5c0 2024-03-08T06:01:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.603+0000 7f41bed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41c0159770 0x7f41c01f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.603+0000 7f41bed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41c0159770 0x7f41c01f3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53558/0 (socket says 172.21.15.76:53558) 2024-03-08T06:01:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.603+0000 7f41bed9d640 1 -- 172.21.15.76:0/3540032514 learned_addr learned my addr 172.21.15.76:0/3540032514 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.603+0000 7f41be59c640 1 --2- 172.21.15.76:0/3540032514 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41c015c600 0x7f41c014eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.603+0000 7f41bf59e640 1 --2- 172.21.15.76:0/3540032514 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41c015f5c0 0x7f41c00b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.604+0000 7f41bed9d640 1 -- 172.21.15.76:0/3540032514 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41c015f5c0 msgr2=0x7f41c00b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.604+0000 7f41bed9d640 1 --2- 172.21.15.76:0/3540032514 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41c015f5c0 0x7f41c00b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.604+0000 7f41bed9d640 1 -- 172.21.15.76:0/3540032514 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41c015c600 msgr2=0x7f41c014eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.604+0000 7f41bed9d640 1 --2- 172.21.15.76:0/3540032514 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41c015c600 0x7f41c014eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.604+0000 7f41bed9d640 1 -- 172.21.15.76:0/3540032514 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c00b73b0 con 0x7f41c0159770 2024-03-08T06:01:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.604+0000 7f41bed9d640 1 --2- 172.21.15.76:0/3540032514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41c0159770 0x7f41c01f3970 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f41b800c460 tx=0x7f41b800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.604+0000 7f41b77fe640 1 -- 172.21.15.76:0/3540032514 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8017070 con 0x7f41c0159770 2024-03-08T06:01:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.604+0000 7f41b77fe640 1 -- 172.21.15.76:0/3540032514 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b8005ce0 con 0x7f41c0159770 2024-03-08T06:01:36.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.604+0000 7f41c5239640 1 -- 172.21.15.76:0/3540032514 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c006bfe0 con 0x7f41c0159770 2024-03-08T06:01:36.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.604+0000 7f41b77fe640 1 -- 172.21.15.76:0/3540032514 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8007290 con 0x7f41c0159770 2024-03-08T06:01:36.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.604+0000 7f41c5239640 1 -- 172.21.15.76:0/3540032514 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c006c5a0 con 0x7f41c0159770 2024-03-08T06:01:36.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.605+0000 7f41c5239640 1 -- 172.21.15.76:0/3540032514 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c004f030 con 0x7f41c0159770 2024-03-08T06:01:36.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.605+0000 7f41b77fe640 1 -- 172.21.15.76:0/3540032514 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41b801a070 con 0x7f41c0159770 2024-03-08T06:01:36.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.606+0000 7f41b77fe640 1 --2- 172.21.15.76:0/3540032514 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f41a006b4d0 0x7f41a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.606+0000 7f41be59c640 1 --2- 172.21.15.76:0/3540032514 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f41a006b4d0 0x7f41a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.606+0000 7f41b77fe640 1 -- 172.21.15.76:0/3540032514 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f41b8089870 con 0x7f41c0159770 2024-03-08T06:01:36.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.606+0000 7f41be59c640 1 --2- 172.21.15.76:0/3540032514 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f41a006b4d0 0x7f41a006d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009560 tx=0x7f41a8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:36.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.610+0000 7f41b77fe640 1 -- 172.21.15.76:0/3540032514 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f41b8057d30 con 0x7f41c0159770 2024-03-08T06:01:36.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.817+0000 7f41c5239640 1 -- 172.21.15.76:0/3540032514 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f41c00703a0 con 0x7f41c0159770 2024-03-08T06:01:36.819 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-03-08T06:01:36.818+0000 7fcc89c70640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-08T06:01:36.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.821+0000 7f41b77fe640 1 -- 172.21.15.76:0/3540032514 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f41b805e080 con 0x7f41c0159770 2024-03-08T06:01:36.822 INFO:teuthology.orchestra.run.smithi076.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-08T06:01:36.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.824+0000 7f41c5239640 1 -- 172.21.15.76:0/3540032514 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f41a006b4d0 msgr2=0x7f41a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.824+0000 7f41c5239640 1 --2- 172.21.15.76:0/3540032514 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f41a006b4d0 0x7f41a006d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009560 tx=0x7f41a8009190 comp rx=0 tx=0).stop 2024-03-08T06:01:36.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.824+0000 7f41c5239640 1 -- 172.21.15.76:0/3540032514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41c0159770 msgr2=0x7f41c01f3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.824+0000 7f41c5239640 1 --2- 172.21.15.76:0/3540032514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41c0159770 0x7f41c01f3970 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f41b800c460 tx=0x7f41b800c920 comp rx=0 tx=0).stop 2024-03-08T06:01:36.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.824+0000 7f41c5239640 1 -- 172.21.15.76:0/3540032514 shutdown_connections 2024-03-08T06:01:36.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.824+0000 7f41c5239640 1 --2- 172.21.15.76:0/3540032514 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f41a006b4d0 0x7f41a006d990 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.824+0000 7f41c5239640 1 --2- 172.21.15.76:0/3540032514 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f41c015f5c0 0x7f41c00b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.824+0000 7f41c5239640 1 --2- 172.21.15.76:0/3540032514 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41c015c600 0x7f41c014eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.824+0000 7f41c5239640 1 --2- 172.21.15.76:0/3540032514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41c0159770 0x7f41c01f3970 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.824+0000 7f41c5239640 1 -- 172.21.15.76:0/3540032514 >> 172.21.15.76:0/3540032514 conn(0x7f41c006f530 msgr2=0x7f41c015ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:36.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.824+0000 7f41c5239640 1 -- 172.21.15.76:0/3540032514 shutdown_connections 2024-03-08T06:01:36.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.824+0000 7f41c5239640 1 -- 172.21.15.76:0/3540032514 wait complete. 2024-03-08T06:01:36.829 INFO:tasks.ceph.mds.d.smithi081.stderr: -4961> 2024-03-08T06:00:40.536+0000 7f935f27d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-08T06:01:36.836 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-08T06:01:36.859 INFO:tasks.ceph.mds.d.smithi081.stdout:starting mds.d at 2024-03-08T06:01:36.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.939+0000 7fbe8dd3e640 1 Processor -- start 2024-03-08T06:01:36.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.939+0000 7fbe8dd3e640 1 -- start start 2024-03-08T06:01:36.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.940+0000 7fbe8dd3e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbe880b9840 0x7fbe880b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.940+0000 7fbe8dd3e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe880b5f40 con 0x7fbe8815dbc0 2024-03-08T06:01:36.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.940+0000 7fbe8dd3e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe880d27d0 con 0x7fbe880b5b20 2024-03-08T06:01:36.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.940+0000 7fbe8dd3e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe880d2ab0 con 0x7fbe880b9840 2024-03-08T06:01:36.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.940+0000 7fbe877fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbe8815dbc0 0x7fbe880d1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46844/0 (socket says 172.21.15.76:46844) 2024-03-08T06:01:36.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.940+0000 7fbe877fe640 1 -- 172.21.15.76:0/182836535 learned_addr learned my addr 172.21.15.76:0/182836535 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:36.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.940+0000 7fbe86ffd640 1 --2- 172.21.15.76:0/182836535 >> v2:172.21.15.81:3300/0 conn(0x7fbe880b9840 0x7fbe880b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.940+0000 7fbe867fc640 1 -- 172.21.15.76:0/182836535 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636426814 0 0) 0x7fbe880b5f40 con 0x7fbe8815dbc0 2024-03-08T06:01:36.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.940+0000 7fbe867fc640 1 -- 172.21.15.76:0/182836535 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe70003520 con 0x7fbe8815dbc0 2024-03-08T06:01:36.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.941+0000 7fbe86ffd640 1 -- 172.21.15.76:0/182836535 >> v1:172.21.15.81:6789/0 conn(0x7fbe880b5b20 legacy=0x7fbe880b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.941+0000 7fbe86ffd640 1 -- 172.21.15.76:0/182836535 >> v1:172.21.15.76:6789/0 conn(0x7fbe8815dbc0 legacy=0x7fbe880d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.941+0000 7fbe86ffd640 1 -- 172.21.15.76:0/182836535 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe880d2d90 con 0x7fbe880b9840 2024-03-08T06:01:36.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.941+0000 7fbe86ffd640 1 --2- 172.21.15.76:0/182836535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe880b9840 0x7fbe880b55e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c0097a0 tx=0x7fbe7c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=715944f27217c21c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:36.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.941+0000 7fbe867fc640 1 -- 172.21.15.76:0/182836535 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c034070 con 0x7fbe880b9840 2024-03-08T06:01:36.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.942+0000 7fbe867fc640 1 -- 172.21.15.76:0/182836535 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7c030610 con 0x7fbe880b9840 2024-03-08T06:01:36.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.942+0000 7fbe867fc640 1 -- 172.21.15.76:0/182836535 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c0309e0 con 0x7fbe880b9840 2024-03-08T06:01:36.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.942+0000 7fbe8dd3e640 1 -- 172.21.15.76:0/182836535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe880b9840 msgr2=0x7fbe880b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.942+0000 7fbe8dd3e640 1 --2- 172.21.15.76:0/182836535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe880b9840 0x7fbe880b55e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c0097a0 tx=0x7fbe7c02b960 comp rx=0 tx=0).stop 2024-03-08T06:01:36.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.942+0000 7fbe8dd3e640 1 -- 172.21.15.76:0/182836535 shutdown_connections 2024-03-08T06:01:36.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.942+0000 7fbe8dd3e640 1 --2- 172.21.15.76:0/182836535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe880b9840 0x7fbe880b55e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.942+0000 7fbe8dd3e640 1 -- 172.21.15.76:0/182836535 >> 172.21.15.76:0/182836535 conn(0x7fbe880670d0 msgr2=0x7fbe880d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:36.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.942+0000 7fbe8dd3e640 1 -- 172.21.15.76:0/182836535 shutdown_connections 2024-03-08T06:01:36.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.942+0000 7fbe8dd3e640 1 -- 172.21.15.76:0/182836535 wait complete. 2024-03-08T06:01:36.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.943+0000 7fbe8dd3e640 1 Processor -- start 2024-03-08T06:01:36.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.943+0000 7fbe8dd3e640 1 -- start start 2024-03-08T06:01:36.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.944+0000 7fbe8dd3e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe880b5b20 0x7fbe88158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.944+0000 7fbe8dd3e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe880b9840 0x7fbe88156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.944+0000 7fbe8dd3e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbe8815dbc0 0x7fbe88157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.944+0000 7fbe8dd3e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8815dfa0 con 0x7fbe880b9840 2024-03-08T06:01:36.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.944+0000 7fbe8dd3e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe880d3870 con 0x7fbe880b5b20 2024-03-08T06:01:36.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.944+0000 7fbe8dd3e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8815e120 con 0x7fbe8815dbc0 2024-03-08T06:01:36.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.944+0000 7fbe86ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe880b9840 0x7fbe88156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.944+0000 7fbe86ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe880b9840 0x7fbe88156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53560/0 (socket says 172.21.15.76:53560) 2024-03-08T06:01:36.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.944+0000 7fbe86ffd640 1 -- 172.21.15.76:0/2538525706 learned_addr learned my addr 172.21.15.76:0/2538525706 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:36.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.944+0000 7fbe877fe640 1 --2- 172.21.15.76:0/2538525706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe880b5b20 0x7fbe88158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.944+0000 7fbe87fff640 1 --2- 172.21.15.76:0/2538525706 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbe8815dbc0 0x7fbe88157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.944+0000 7fbe86ffd640 1 -- 172.21.15.76:0/2538525706 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbe8815dbc0 msgr2=0x7fbe88157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.944+0000 7fbe86ffd640 1 --2- 172.21.15.76:0/2538525706 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbe8815dbc0 0x7fbe88157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.944+0000 7fbe86ffd640 1 -- 172.21.15.76:0/2538525706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe880b5b20 msgr2=0x7fbe88158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:36.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.944+0000 7fbe86ffd640 1 --2- 172.21.15.76:0/2538525706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe880b5b20 0x7fbe88158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:36.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.944+0000 7fbe86ffd640 1 -- 172.21.15.76:0/2538525706 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe88148e90 con 0x7fbe880b9840 2024-03-08T06:01:36.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.945+0000 7fbe86ffd640 1 --2- 172.21.15.76:0/2538525706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe880b9840 0x7fbe88156c20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c0098c0 tx=0x7fbe7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:36.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.945+0000 7fbe6ffff640 1 -- 172.21.15.76:0/2538525706 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c034040 con 0x7fbe880b9840 2024-03-08T06:01:36.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.945+0000 7fbe6ffff640 1 -- 172.21.15.76:0/2538525706 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7c030410 con 0x7fbe880b9840 2024-03-08T06:01:36.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.945+0000 7fbe8dd3e640 1 -- 172.21.15.76:0/2538525706 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe88149170 con 0x7fbe880b9840 2024-03-08T06:01:36.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.945+0000 7fbe6ffff640 1 -- 172.21.15.76:0/2538525706 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c039c50 con 0x7fbe880b9840 2024-03-08T06:01:36.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.945+0000 7fbe8dd3e640 1 -- 172.21.15.76:0/2538525706 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe88149730 con 0x7fbe880b9840 2024-03-08T06:01:36.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.946+0000 7fbe8dd3e640 1 -- 172.21.15.76:0/2538525706 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe880b5940 con 0x7fbe880b9840 2024-03-08T06:01:36.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.946+0000 7fbe6ffff640 1 -- 172.21.15.76:0/2538525706 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe7c037070 con 0x7fbe880b9840 2024-03-08T06:01:36.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.946+0000 7fbe6ffff640 1 --2- 172.21.15.76:0/2538525706 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbe7406b4d0 0x7fbe7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:36.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.947+0000 7fbe6ffff640 1 -- 172.21.15.76:0/2538525706 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6821+0+0 (secure 0 0 0) 0x7fbe7c0ac560 con 0x7fbe880b9840 2024-03-08T06:01:36.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.947+0000 7fbe877fe640 1 --2- 172.21.15.76:0/2538525706 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbe7406b4d0 0x7fbe7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:36.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.947+0000 7fbe877fe640 1 --2- 172.21.15.76:0/2538525706 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbe7406b4d0 0x7fbe7406d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbe780011e0 tx=0x7fbe78007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:36.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:36.951+0000 7fbe6ffff640 1 -- 172.21.15.76:0/2538525706 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe880b5940 con 0x7fbe880b9840 2024-03-08T06:01:37.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.159+0000 7fbe8dd3e640 1 -- 172.21.15.76:0/2538525706 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbe8804f030 con 0x7fbe880b9840 2024-03-08T06:01:37.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.822+0000 7fbe6ffff640 1 -- 172.21.15.76:0/2538525706 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbe7c07a8b0 con 0x7fbe880b9840 2024-03-08T06:01:37.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.825+0000 7fbe8dd3e640 1 -- 172.21.15.76:0/2538525706 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbe7406b4d0 msgr2=0x7fbe7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:37.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.825+0000 7fbe8dd3e640 1 --2- 172.21.15.76:0/2538525706 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbe7406b4d0 0x7fbe7406d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbe780011e0 tx=0x7fbe78007d30 comp rx=0 tx=0).stop 2024-03-08T06:01:37.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.825+0000 7fbe8dd3e640 1 -- 172.21.15.76:0/2538525706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe880b9840 msgr2=0x7fbe88156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:37.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.825+0000 7fbe8dd3e640 1 --2- 172.21.15.76:0/2538525706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe880b9840 0x7fbe88156c20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c0098c0 tx=0x7fbe7c0046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:37.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.825+0000 7fbe8dd3e640 1 -- 172.21.15.76:0/2538525706 shutdown_connections 2024-03-08T06:01:37.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.825+0000 7fbe8dd3e640 1 --2- 172.21.15.76:0/2538525706 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbe7406b4d0 0x7fbe7406d990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:37.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.825+0000 7fbe8dd3e640 1 --2- 172.21.15.76:0/2538525706 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbe8815dbc0 0x7fbe88157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:37.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.825+0000 7fbe8dd3e640 1 --2- 172.21.15.76:0/2538525706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe880b9840 0x7fbe88156c20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:37.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.825+0000 7fbe8dd3e640 1 --2- 172.21.15.76:0/2538525706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe880b5b20 0x7fbe88158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:37.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.825+0000 7fbe8dd3e640 1 -- 172.21.15.76:0/2538525706 >> 172.21.15.76:0/2538525706 conn(0x7fbe880670d0 msgr2=0x7fbe880b6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:37.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.825+0000 7fbe8dd3e640 1 -- 172.21.15.76:0/2538525706 shutdown_connections 2024-03-08T06:01:37.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.825+0000 7fbe8dd3e640 1 -- 172.21.15.76:0/2538525706 wait complete. 2024-03-08T06:01:37.838 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-08T06:01:37.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.944+0000 7f6205076640 1 Processor -- start 2024-03-08T06:01:37.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.944+0000 7f6205076640 1 -- start start 2024-03-08T06:01:37.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.944+0000 7f6205076640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f62000ceef0 0x7f6200144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:37.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.944+0000 7f6205076640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f62000cc8e0 0x7f62000ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:37.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.944+0000 7f6205076640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620015f530 con 0x7f62001447f0 2024-03-08T06:01:37.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.944+0000 7f6205076640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620006e9d0 con 0x7f62000ceef0 2024-03-08T06:01:37.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.944+0000 7f6205076640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620015f810 con 0x7f62000cc8e0 2024-03-08T06:01:37.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.945+0000 7f61fe59c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f62000ceef0 0x7f6200144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:37.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.945+0000 7f61ff59e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f62000cc8e0 0x7f62000ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:37.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.945+0000 7f61fed9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f62001447f0 0x7f62000ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:34530/0 (socket says 172.21.15.76:34530) 2024-03-08T06:01:37.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.945+0000 7f61fed9d640 1 -- 172.21.15.76:0/1683711540 learned_addr learned my addr 172.21.15.76:0/1683711540 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:37.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.945+0000 7f61fe59c640 1 --2- 172.21.15.76:0/1683711540 >> v2:172.21.15.81:3300/0 conn(0x7f62000ceef0 0x7f6200144bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.76:44996/0 (socket says 172.21.15.76:44996) 2024-03-08T06:01:37.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.945+0000 7f61fe59c640 1 -- 172.21.15.76:0/1683711540 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f62000cc8e0 msgr2=0x7f62000ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:37.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.945+0000 7f61fe59c640 1 --2- 172.21.15.76:0/1683711540 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f62000cc8e0 0x7f62000ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:37.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.945+0000 7f61fe59c640 1 -- 172.21.15.76:0/1683711540 >> v1:172.21.15.81:6789/0 conn(0x7f62001447f0 legacy=0x7f62000ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:37.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.945+0000 7f61fe59c640 1 -- 172.21.15.76:0/1683711540 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620015faf0 con 0x7f62000ceef0 2024-03-08T06:01:37.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.945+0000 7f61fdd9b640 1 -- 172.21.15.76:0/1683711540 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344175054 0 0) 0x7f620015f530 con 0x7f62001447f0 2024-03-08T06:01:37.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.945+0000 7f61ff59e640 1 --2- 172.21.15.76:0/1683711540 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f62000cc8e0 0x7f62000ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:37.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.946+0000 7f61fe59c640 1 --2- 172.21.15.76:0/1683711540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62000ceef0 0x7f6200144bd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f61f40097a0 tx=0x7f61f402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84d5e000d7755571 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:37.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.946+0000 7f61fdd9b640 1 -- 172.21.15.76:0/1683711540 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4035020 con 0x7f62000ceef0 2024-03-08T06:01:37.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.946+0000 7f61fdd9b640 1 -- 172.21.15.76:0/1683711540 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f4031830 con 0x7f62000ceef0 2024-03-08T06:01:37.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.946+0000 7f61fdd9b640 1 -- 172.21.15.76:0/1683711540 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4031c00 con 0x7f62000ceef0 2024-03-08T06:01:37.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.946+0000 7f6205076640 1 -- 172.21.15.76:0/1683711540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62000ceef0 msgr2=0x7f6200144bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:37.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.946+0000 7f6205076640 1 --2- 172.21.15.76:0/1683711540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62000ceef0 0x7f6200144bd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f61f40097a0 tx=0x7f61f402b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:37.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.947+0000 7f6205076640 1 -- 172.21.15.76:0/1683711540 shutdown_connections 2024-03-08T06:01:37.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.947+0000 7f6205076640 1 --2- 172.21.15.76:0/1683711540 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f62000cc8e0 0x7f62000ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:37.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.947+0000 7f6205076640 1 --2- 172.21.15.76:0/1683711540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62000ceef0 0x7f6200144bd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:37.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.947+0000 7f6205076640 1 -- 172.21.15.76:0/1683711540 >> 172.21.15.76:0/1683711540 conn(0x7f620006ac80 msgr2=0x7f620006b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:37.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.947+0000 7f6205076640 1 -- 172.21.15.76:0/1683711540 shutdown_connections 2024-03-08T06:01:37.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.947+0000 7f6205076640 1 -- 172.21.15.76:0/1683711540 wait complete. 2024-03-08T06:01:37.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.948+0000 7f6205076640 1 Processor -- start 2024-03-08T06:01:37.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.948+0000 7f6205076640 1 -- start start 2024-03-08T06:01:37.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f6205076640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62000cc8e0 0x7f62001f7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:37.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f6205076640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62000ceef0 0x7f62000c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:37.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f6205076640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f62001447f0 0x7f62000b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:37.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f6205076640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62001602f0 con 0x7f62000cc8e0 2024-03-08T06:01:37.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f6205076640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62000cf3a0 con 0x7f62000ceef0 2024-03-08T06:01:37.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f6205076640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6200145360 con 0x7f62001447f0 2024-03-08T06:01:37.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f61fed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62000cc8e0 0x7f62001f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:37.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f61fed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62000cc8e0 0x7f62001f7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53566/0 (socket says 172.21.15.76:53566) 2024-03-08T06:01:37.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f61fed9d640 1 -- 172.21.15.76:0/711065653 learned_addr learned my addr 172.21.15.76:0/711065653 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:37.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f61fe59c640 1 --2- 172.21.15.76:0/711065653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62000ceef0 0x7f62000c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:37.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f61ff59e640 1 --2- 172.21.15.76:0/711065653 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f62001447f0 0x7f62000b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:37.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f61fed9d640 1 -- 172.21.15.76:0/711065653 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f62001447f0 msgr2=0x7f62000b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:37.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f61fed9d640 1 --2- 172.21.15.76:0/711065653 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f62001447f0 0x7f62000b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:37.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f61fed9d640 1 -- 172.21.15.76:0/711065653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62000ceef0 msgr2=0x7f62000c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:37.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f61fed9d640 1 --2- 172.21.15.76:0/711065653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62000ceef0 0x7f62000c4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:37.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f61fed9d640 1 -- 172.21.15.76:0/711065653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6200139230 con 0x7f62000cc8e0 2024-03-08T06:01:37.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f61fed9d640 1 --2- 172.21.15.76:0/711065653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62000cc8e0 0x7f62001f7380 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f61f000c4d0 tx=0x7f61f000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:37.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f61e77fe640 1 -- 172.21.15.76:0/711065653 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f00074e0 con 0x7f62000cc8e0 2024-03-08T06:01:37.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.949+0000 7f61e77fe640 1 -- 172.21.15.76:0/711065653 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f00047e0 con 0x7f62000cc8e0 2024-03-08T06:01:37.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.950+0000 7f6205076640 1 -- 172.21.15.76:0/711065653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62000c5170 con 0x7f62000cc8e0 2024-03-08T06:01:37.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.950+0000 7f61e77fe640 1 -- 172.21.15.76:0/711065653 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0004bd0 con 0x7f62000cc8e0 2024-03-08T06:01:37.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.950+0000 7f6205076640 1 -- 172.21.15.76:0/711065653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62000b28b0 con 0x7f62000cc8e0 2024-03-08T06:01:37.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.951+0000 7f6205076640 1 -- 172.21.15.76:0/711065653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620004f030 con 0x7f62000cc8e0 2024-03-08T06:01:37.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.951+0000 7f61e77fe640 1 -- 172.21.15.76:0/711065653 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f61f0011070 con 0x7f62000cc8e0 2024-03-08T06:01:37.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.951+0000 7f61e77fe640 1 --2- 172.21.15.76:0/711065653 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f61ec06b4d0 0x7f61ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:37.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.951+0000 7f61e77fe640 1 -- 172.21.15.76:0/711065653 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6821+0+0 (secure 0 0 0) 0x7f61f008a140 con 0x7f62000cc8e0 2024-03-08T06:01:37.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.951+0000 7f61fe59c640 1 --2- 172.21.15.76:0/711065653 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f61ec06b4d0 0x7f61ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:37.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.952+0000 7f61fe59c640 1 --2- 172.21.15.76:0/711065653 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f61ec06b4d0 0x7f61ec06d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f61f4036000 tx=0x7f61f403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:37.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:37.956+0000 7f61e77fe640 1 -- 172.21.15.76:0/711065653 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f61f0058bf0 con 0x7f62000cc8e0 2024-03-08T06:01:38.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.126+0000 7f6205076640 1 -- 172.21.15.76:0/711065653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f62000cb540 con 0x7f62000cc8e0 2024-03-08T06:01:38.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.829+0000 7f61e77fe640 1 -- 172.21.15.76:0/711065653 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) v1 ==== 181+0+0 (secure 0 0 0) 0x7f61f00584f0 con 0x7f62000cc8e0 2024-03-08T06:01:38.830 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' removed 2024-03-08T06:01:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.832+0000 7f6205076640 1 -- 172.21.15.76:0/711065653 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f61ec06b4d0 msgr2=0x7f61ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.832+0000 7f6205076640 1 --2- 172.21.15.76:0/711065653 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f61ec06b4d0 0x7f61ec06d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f61f4036000 tx=0x7f61f403d040 comp rx=0 tx=0).stop 2024-03-08T06:01:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.832+0000 7f6205076640 1 -- 172.21.15.76:0/711065653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62000cc8e0 msgr2=0x7f62001f7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.832+0000 7f6205076640 1 --2- 172.21.15.76:0/711065653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62000cc8e0 0x7f62001f7380 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f61f000c4d0 tx=0x7f61f000c8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.833+0000 7f6205076640 1 -- 172.21.15.76:0/711065653 shutdown_connections 2024-03-08T06:01:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.833+0000 7f6205076640 1 --2- 172.21.15.76:0/711065653 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f61ec06b4d0 0x7f61ec06d990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.833+0000 7f6205076640 1 --2- 172.21.15.76:0/711065653 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f62001447f0 0x7f62000b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.833+0000 7f6205076640 1 --2- 172.21.15.76:0/711065653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62000ceef0 0x7f62000c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.833+0000 7f6205076640 1 --2- 172.21.15.76:0/711065653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62000cc8e0 0x7f62001f7380 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.833+0000 7f6205076640 1 -- 172.21.15.76:0/711065653 >> 172.21.15.76:0/711065653 conn(0x7f620006ac80 msgr2=0x7f62000cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.833+0000 7f6205076640 1 -- 172.21.15.76:0/711065653 shutdown_connections 2024-03-08T06:01:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.833+0000 7f6205076640 1 -- 172.21.15.76:0/711065653 wait complete. 2024-03-08T06:01:38.848 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-08T06:01:38.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.958+0000 7fd99663b640 1 Processor -- start 2024-03-08T06:01:38.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.958+0000 7fd99663b640 1 -- start start 2024-03-08T06:01:38.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.959+0000 7fd99663b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd9900cf640 0x7fd99015ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:38.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.959+0000 7fd99663b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd9900d05f0 0x7fd9900d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:38.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.959+0000 7fd99663b640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99015ae20 con 0x7fd99015aa40 2024-03-08T06:01:38.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.959+0000 7fd99663b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd99015e990 con 0x7fd9900cf640 2024-03-08T06:01:38.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.959+0000 7fd99663b640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9900706d0 con 0x7fd9900d05f0 2024-03-08T06:01:38.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.959+0000 7fd98f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd9900cf640 0x7fd99015ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:38.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.959+0000 7fd98ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd99015aa40 0x7fd99006fdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48426/0 (socket says 172.21.15.76:48426) 2024-03-08T06:01:38.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.959+0000 7fd98ffff640 1 -- 172.21.15.76:0/2012656876 learned_addr learned my addr 172.21.15.76:0/2012656876 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:38.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.960+0000 7fd994bd8640 1 --2- 172.21.15.76:0/2012656876 >> v2:172.21.15.81:3300/0 conn(0x7fd9900d05f0 0x7fd9900d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:38.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.960+0000 7fd98f7fe640 1 -- 172.21.15.76:0/2012656876 >> v2:172.21.15.81:3300/0 conn(0x7fd9900d05f0 msgr2=0x7fd9900d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:38.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.960+0000 7fd98f7fe640 1 --2- 172.21.15.76:0/2012656876 >> v2:172.21.15.81:3300/0 conn(0x7fd9900d05f0 0x7fd9900d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:38.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.960+0000 7fd98f7fe640 1 -- 172.21.15.76:0/2012656876 >> v1:172.21.15.76:6789/0 conn(0x7fd99015aa40 legacy=0x7fd99006fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:38.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.960+0000 7fd98f7fe640 1 -- 172.21.15.76:0/2012656876 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9900709b0 con 0x7fd9900cf640 2024-03-08T06:01:38.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.960+0000 7fd98effd640 1 -- 172.21.15.76:0/2012656876 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286941013 0 0) 0x7fd980001110 con 0x7fd99015aa40 2024-03-08T06:01:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.960+0000 7fd98f7fe640 1 --2- 172.21.15.76:0/2012656876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9900cf640 0x7fd99015ee20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd99015eb10 tx=0x7fd98402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71057832f81893eb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.960+0000 7fd98effd640 1 -- 172.21.15.76:0/2012656876 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd984035020 con 0x7fd9900cf640 2024-03-08T06:01:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.960+0000 7fd98effd640 1 -- 172.21.15.76:0/2012656876 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd984031830 con 0x7fd9900cf640 2024-03-08T06:01:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.960+0000 7fd98effd640 1 -- 172.21.15.76:0/2012656876 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd984031c00 con 0x7fd9900cf640 2024-03-08T06:01:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.961+0000 7fd99663b640 1 -- 172.21.15.76:0/2012656876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9900cf640 msgr2=0x7fd99015ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.961+0000 7fd99663b640 1 --2- 172.21.15.76:0/2012656876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9900cf640 0x7fd99015ee20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd99015eb10 tx=0x7fd98402b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.961+0000 7fd99663b640 1 -- 172.21.15.76:0/2012656876 shutdown_connections 2024-03-08T06:01:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.961+0000 7fd99663b640 1 --2- 172.21.15.76:0/2012656876 >> v2:172.21.15.81:3300/0 conn(0x7fd9900d05f0 0x7fd9900d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.961+0000 7fd99663b640 1 --2- 172.21.15.76:0/2012656876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9900cf640 0x7fd99015ee20 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.961+0000 7fd99663b640 1 -- 172.21.15.76:0/2012656876 >> 172.21.15.76:0/2012656876 conn(0x7fd9900b2390 msgr2=0x7fd9900b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.961+0000 7fd99663b640 1 -- 172.21.15.76:0/2012656876 shutdown_connections 2024-03-08T06:01:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.961+0000 7fd99663b640 1 -- 172.21.15.76:0/2012656876 wait complete. 2024-03-08T06:01:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.962+0000 7fd99663b640 1 Processor -- start 2024-03-08T06:01:38.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.962+0000 7fd99663b640 1 -- start start 2024-03-08T06:01:38.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.963+0000 7fd99663b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9900cf640 0x7fd9900c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:38.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.963+0000 7fd99663b640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd9900d05f0 0x7fd99015cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:38.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.963+0000 7fd99663b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd99015aa40 0x7fd9901f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:38.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.963+0000 7fd99663b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9900d0fa0 con 0x7fd9900cf640 2024-03-08T06:01:38.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.963+0000 7fd99663b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd990070c90 con 0x7fd99015aa40 2024-03-08T06:01:38.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.963+0000 7fd99663b640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99015f360 con 0x7fd9900d05f0 2024-03-08T06:01:38.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.963+0000 7fd98ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9900cf640 0x7fd9900c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:38.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.963+0000 7fd98ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9900cf640 0x7fd9900c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45358/0 (socket says 172.21.15.76:45358) 2024-03-08T06:01:38.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.963+0000 7fd98ffff640 1 -- 172.21.15.76:0/1603780300 learned_addr learned my addr 172.21.15.76:0/1603780300 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:38.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.963+0000 7fd994bd8640 1 --2- 172.21.15.76:0/1603780300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd99015aa40 0x7fd9901f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:38.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.964+0000 7fd98f7fe640 1 --2- 172.21.15.76:0/1603780300 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd9900d05f0 0x7fd99015cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:38.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.964+0000 7fd98ffff640 1 -- 172.21.15.76:0/1603780300 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd9900d05f0 msgr2=0x7fd99015cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:38.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.964+0000 7fd98ffff640 1 --2- 172.21.15.76:0/1603780300 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd9900d05f0 0x7fd99015cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:38.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.964+0000 7fd98ffff640 1 -- 172.21.15.76:0/1603780300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd99015aa40 msgr2=0x7fd9901f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:38.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.964+0000 7fd98ffff640 1 --2- 172.21.15.76:0/1603780300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd99015aa40 0x7fd9901f17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:38.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.964+0000 7fd98ffff640 1 -- 172.21.15.76:0/1603780300 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9901f20b0 con 0x7fd9900cf640 2024-03-08T06:01:38.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.964+0000 7fd98f7fe640 1 --2- 172.21.15.76:0/1603780300 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd9900d05f0 0x7fd99015cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:38.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.964+0000 7fd98ffff640 1 --2- 172.21.15.76:0/1603780300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9900cf640 0x7fd9900c7d30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd98000c780 tx=0x7fd98000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:38.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.964+0000 7fd98cff9640 1 -- 172.21.15.76:0/1603780300 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd980018070 con 0x7fd9900cf640 2024-03-08T06:01:38.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.964+0000 7fd98cff9640 1 -- 172.21.15.76:0/1603780300 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9800072d0 con 0x7fd9900cf640 2024-03-08T06:01:38.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.964+0000 7fd99663b640 1 -- 172.21.15.76:0/1603780300 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9901f23f0 con 0x7fd9900cf640 2024-03-08T06:01:38.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.964+0000 7fd98cff9640 1 -- 172.21.15.76:0/1603780300 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9800047e0 con 0x7fd9900cf640 2024-03-08T06:01:38.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.964+0000 7fd99663b640 1 -- 172.21.15.76:0/1603780300 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99015d470 con 0x7fd9900cf640 2024-03-08T06:01:38.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.965+0000 7fd99663b640 1 -- 172.21.15.76:0/1603780300 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd99004f030 con 0x7fd9900cf640 2024-03-08T06:01:38.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.965+0000 7fd98cff9640 1 -- 172.21.15.76:0/1603780300 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd98001b070 con 0x7fd9900cf640 2024-03-08T06:01:38.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.966+0000 7fd98cff9640 1 --2- 172.21.15.76:0/1603780300 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd97c06b4d0 0x7fd97c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:38.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.966+0000 7fd98cff9640 1 -- 172.21.15.76:0/1603780300 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6383+0+0 (secure 0 0 0) 0x7fd9800897f0 con 0x7fd9900cf640 2024-03-08T06:01:38.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.966+0000 7fd98f7fe640 1 --2- 172.21.15.76:0/1603780300 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd97c06b4d0 0x7fd97c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:38.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.966+0000 7fd98f7fe640 1 --2- 172.21.15.76:0/1603780300 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd97c06b4d0 0x7fd97c06d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd984036000 tx=0x7fd984008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:38.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:38.970+0000 7fd98cff9640 1 -- 172.21.15.76:0/1603780300 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9901f23f0 con 0x7fd9900cf640 2024-03-08T06:01:39.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.146+0000 7fd99663b640 1 -- 172.21.15.76:0/1603780300 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd9900d0d50 con 0x7fd9900cf640 2024-03-08T06:01:39.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.832+0000 7fd98cff9640 1 -- 172.21.15.76:0/1603780300 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd99015ae20 con 0x7fd9900cf640 2024-03-08T06:01:39.833 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' removed 2024-03-08T06:01:39.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.835+0000 7fd99663b640 1 -- 172.21.15.76:0/1603780300 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd97c06b4d0 msgr2=0x7fd97c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:39.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.835+0000 7fd99663b640 1 --2- 172.21.15.76:0/1603780300 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd97c06b4d0 0x7fd97c06d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd984036000 tx=0x7fd984008030 comp rx=0 tx=0).stop 2024-03-08T06:01:39.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.835+0000 7fd99663b640 1 -- 172.21.15.76:0/1603780300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9900cf640 msgr2=0x7fd9900c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:39.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.835+0000 7fd99663b640 1 --2- 172.21.15.76:0/1603780300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9900cf640 0x7fd9900c7d30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd98000c780 tx=0x7fd98000cc40 comp rx=0 tx=0).stop 2024-03-08T06:01:39.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.835+0000 7fd99663b640 1 -- 172.21.15.76:0/1603780300 shutdown_connections 2024-03-08T06:01:39.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.835+0000 7fd99663b640 1 --2- 172.21.15.76:0/1603780300 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd97c06b4d0 0x7fd97c06d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:39.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.835+0000 7fd99663b640 1 --2- 172.21.15.76:0/1603780300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd99015aa40 0x7fd9901f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:39.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.835+0000 7fd99663b640 1 --2- 172.21.15.76:0/1603780300 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd9900d05f0 0x7fd99015cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:39.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.835+0000 7fd99663b640 1 --2- 172.21.15.76:0/1603780300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9900cf640 0x7fd9900c7d30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:39.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.835+0000 7fd99663b640 1 -- 172.21.15.76:0/1603780300 >> 172.21.15.76:0/1603780300 conn(0x7fd9900b2390 msgr2=0x7fd990069940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:39.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.835+0000 7fd99663b640 1 -- 172.21.15.76:0/1603780300 shutdown_connections 2024-03-08T06:01:39.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.835+0000 7fd99663b640 1 -- 172.21.15.76:0/1603780300 wait complete. 2024-03-08T06:01:39.851 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-08T06:01:39.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.956+0000 7fd1d66a7640 1 Processor -- start 2024-03-08T06:01:39.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.956+0000 7fd1d66a7640 1 -- start start 2024-03-08T06:01:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.957+0000 7fd1d66a7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd1d0154d90 0x7fd1d00b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.957+0000 7fd1d66a7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd1d0155170 0x7fd1d015af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.957+0000 7fd1d66a7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d015e740 con 0x7fd1d015b460 2024-03-08T06:01:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.957+0000 7fd1d66a7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d00b2d60 con 0x7fd1d0154d90 2024-03-08T06:01:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.957+0000 7fd1d66a7640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d015ba90 con 0x7fd1d0155170 2024-03-08T06:01:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.957+0000 7fd1cffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd1d0154d90 0x7fd1d00b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.957+0000 7fd1d4c44640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd1d015b460 0x7fd1d006eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48428/0 (socket says 172.21.15.76:48428) 2024-03-08T06:01:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.957+0000 7fd1d4c44640 1 -- 172.21.15.76:0/2247171994 learned_addr learned my addr 172.21.15.76:0/2247171994 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.957+0000 7fd1cffff640 1 -- 172.21.15.76:0/2247171994 >> v2:172.21.15.81:3300/0 conn(0x7fd1d0155170 msgr2=0x7fd1d015af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.957+0000 7fd1cffff640 1 --2- 172.21.15.76:0/2247171994 >> v2:172.21.15.81:3300/0 conn(0x7fd1d0155170 0x7fd1d015af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:39.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.957+0000 7fd1cffff640 1 -- 172.21.15.76:0/2247171994 >> v1:172.21.15.76:6789/0 conn(0x7fd1d015b460 legacy=0x7fd1d006eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:39.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.957+0000 7fd1ceffd640 1 -- 172.21.15.76:0/2247171994 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895049656 0 0) 0x7fd1d015e740 con 0x7fd1d015b460 2024-03-08T06:01:39.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.957+0000 7fd1cffff640 1 -- 172.21.15.76:0/2247171994 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d015bd70 con 0x7fd1d0154d90 2024-03-08T06:01:39.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.958+0000 7fd1cffff640 1 --2- 172.21.15.76:0/2247171994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1d0154d90 0x7fd1d00b31f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00097a0 tx=0x7fd1c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f971e275e1d1aa1d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:39.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.958+0000 7fd1ceffd640 1 -- 172.21.15.76:0/2247171994 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0035020 con 0x7fd1d0154d90 2024-03-08T06:01:39.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.958+0000 7fd1ceffd640 1 -- 172.21.15.76:0/2247171994 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c0031830 con 0x7fd1d0154d90 2024-03-08T06:01:39.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.958+0000 7fd1ceffd640 1 -- 172.21.15.76:0/2247171994 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0031c00 con 0x7fd1d0154d90 2024-03-08T06:01:39.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.959+0000 7fd1d66a7640 1 -- 172.21.15.76:0/2247171994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1d0154d90 msgr2=0x7fd1d00b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:39.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.959+0000 7fd1d66a7640 1 --2- 172.21.15.76:0/2247171994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1d0154d90 0x7fd1d00b31f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00097a0 tx=0x7fd1c002b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:39.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.959+0000 7fd1d66a7640 1 -- 172.21.15.76:0/2247171994 shutdown_connections 2024-03-08T06:01:39.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.959+0000 7fd1d66a7640 1 --2- 172.21.15.76:0/2247171994 >> v2:172.21.15.81:3300/0 conn(0x7fd1d0155170 0x7fd1d015af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:39.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.959+0000 7fd1d66a7640 1 --2- 172.21.15.76:0/2247171994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1d0154d90 0x7fd1d00b31f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:39.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.959+0000 7fd1d66a7640 1 -- 172.21.15.76:0/2247171994 >> 172.21.15.76:0/2247171994 conn(0x7fd1d006ac80 msgr2=0x7fd1d006b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:39.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.959+0000 7fd1d66a7640 1 -- 172.21.15.76:0/2247171994 shutdown_connections 2024-03-08T06:01:39.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.959+0000 7fd1d66a7640 1 -- 172.21.15.76:0/2247171994 wait complete. 2024-03-08T06:01:39.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.960+0000 7fd1d66a7640 1 Processor -- start 2024-03-08T06:01:39.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.960+0000 7fd1d66a7640 1 -- start start 2024-03-08T06:01:39.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1d66a7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1d0154d90 0x7fd1d0147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:39.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1d66a7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd1d0155170 0x7fd1d0157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:39.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1d66a7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1d015b460 0x7fd1d00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:39.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1d66a7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d00b3730 con 0x7fd1d015b460 2024-03-08T06:01:39.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1d66a7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d00b29f0 con 0x7fd1d0154d90 2024-03-08T06:01:39.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1d66a7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d0147640 con 0x7fd1d0155170 2024-03-08T06:01:39.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1d4c44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1d015b460 0x7fd1d00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:39.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1d4c44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1d015b460 0x7fd1d00cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45386/0 (socket says 172.21.15.76:45386) 2024-03-08T06:01:39.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1d4c44640 1 -- 172.21.15.76:0/520478603 learned_addr learned my addr 172.21.15.76:0/520478603 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:39.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1cffff640 1 --2- 172.21.15.76:0/520478603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1d0154d90 0x7fd1d0147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:39.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1cf7fe640 1 --2- 172.21.15.76:0/520478603 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd1d0155170 0x7fd1d0157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:39.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1d4c44640 1 -- 172.21.15.76:0/520478603 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd1d0155170 msgr2=0x7fd1d0157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:39.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1d4c44640 1 --2- 172.21.15.76:0/520478603 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd1d0155170 0x7fd1d0157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:39.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1d4c44640 1 -- 172.21.15.76:0/520478603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1d0154d90 msgr2=0x7fd1d0147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:39.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1d4c44640 1 --2- 172.21.15.76:0/520478603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1d0154d90 0x7fd1d0147100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:39.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1d4c44640 1 -- 172.21.15.76:0/520478603 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d00aef40 con 0x7fd1d015b460 2024-03-08T06:01:39.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1cf7fe640 1 --2- 172.21.15.76:0/520478603 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd1d0155170 0x7fd1d0157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:39.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.961+0000 7fd1d4c44640 1 --2- 172.21.15.76:0/520478603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1d015b460 0x7fd1d00cf930 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800c700 tx=0x7fd1c800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:39.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.962+0000 7fd1ccff9640 1 -- 172.21.15.76:0/520478603 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8018070 con 0x7fd1d015b460 2024-03-08T06:01:39.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.962+0000 7fd1ccff9640 1 -- 172.21.15.76:0/520478603 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c800ce10 con 0x7fd1d015b460 2024-03-08T06:01:39.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.962+0000 7fd1d66a7640 1 -- 172.21.15.76:0/520478603 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d00af220 con 0x7fd1d015b460 2024-03-08T06:01:39.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.962+0000 7fd1ccff9640 1 -- 172.21.15.76:0/520478603 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c800a860 con 0x7fd1d015b460 2024-03-08T06:01:39.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.962+0000 7fd1d66a7640 1 -- 172.21.15.76:0/520478603 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d00af7e0 con 0x7fd1d015b460 2024-03-08T06:01:39.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.963+0000 7fd1d66a7640 1 -- 172.21.15.76:0/520478603 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d004f030 con 0x7fd1d015b460 2024-03-08T06:01:39.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.963+0000 7fd1ccff9640 1 -- 172.21.15.76:0/520478603 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1c801b070 con 0x7fd1d015b460 2024-03-08T06:01:39.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.963+0000 7fd1ccff9640 1 --2- 172.21.15.76:0/520478603 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd1bc06b4d0 0x7fd1bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:39.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.963+0000 7fd1ccff9640 1 -- 172.21.15.76:0/520478603 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd1c8089760 con 0x7fd1d015b460 2024-03-08T06:01:39.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.963+0000 7fd1cffff640 1 --2- 172.21.15.76:0/520478603 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd1bc06b4d0 0x7fd1bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:39.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.964+0000 7fd1cffff640 1 --2- 172.21.15.76:0/520478603 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd1bc06b4d0 0x7fd1bc06d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0036000 tx=0x7fd1c003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:39.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:39.968+0000 7fd1ccff9640 1 -- 172.21.15.76:0/520478603 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1c80585e0 con 0x7fd1d015b460 2024-03-08T06:01:40.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:40.137+0000 7fd1d66a7640 1 -- 172.21.15.76:0/520478603 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd1d00b38b0 con 0x7fd1d015b460 2024-03-08T06:01:40.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:40.838+0000 7fd1ccff9640 1 -- 172.21.15.76:0/520478603 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) v1 ==== 178+0+0 (secure 0 0 0) 0x7fd1c8057ee0 con 0x7fd1d015b460 2024-03-08T06:01:40.839 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data_ec' removed 2024-03-08T06:01:40.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:40.840+0000 7fd1d66a7640 1 -- 172.21.15.76:0/520478603 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd1bc06b4d0 msgr2=0x7fd1bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:40.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:40.840+0000 7fd1d66a7640 1 --2- 172.21.15.76:0/520478603 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd1bc06b4d0 0x7fd1bc06d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0036000 tx=0x7fd1c003c040 comp rx=0 tx=0).stop 2024-03-08T06:01:40.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:40.840+0000 7fd1d66a7640 1 -- 172.21.15.76:0/520478603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1d015b460 msgr2=0x7fd1d00cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:40.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:40.840+0000 7fd1d66a7640 1 --2- 172.21.15.76:0/520478603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1d015b460 0x7fd1d00cf930 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800c700 tx=0x7fd1c800cbc0 comp rx=0 tx=0).stop 2024-03-08T06:01:40.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:40.840+0000 7fd1d66a7640 1 -- 172.21.15.76:0/520478603 shutdown_connections 2024-03-08T06:01:40.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:40.840+0000 7fd1d66a7640 1 --2- 172.21.15.76:0/520478603 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd1bc06b4d0 0x7fd1bc06d990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:40.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:40.840+0000 7fd1d66a7640 1 --2- 172.21.15.76:0/520478603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1d015b460 0x7fd1d00cf930 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:40.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:40.840+0000 7fd1d66a7640 1 --2- 172.21.15.76:0/520478603 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd1d0155170 0x7fd1d0157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:40.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:40.840+0000 7fd1d66a7640 1 --2- 172.21.15.76:0/520478603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1d0154d90 0x7fd1d0147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:40.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:40.840+0000 7fd1d66a7640 1 -- 172.21.15.76:0/520478603 >> 172.21.15.76:0/520478603 conn(0x7fd1d006ac80 msgr2=0x7fd1d0152e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:40.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:40.841+0000 7fd1d66a7640 1 -- 172.21.15.76:0/520478603 shutdown_connections 2024-03-08T06:01:40.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:40.841+0000 7fd1d66a7640 1 -- 172.21.15.76:0/520478603 wait complete. 2024-03-08T06:01:40.857 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-08T06:01:40.858 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-08T06:01:40.858 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-08T06:01:40.858 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:01:40.858 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-08T06:01:40.858 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-08T06:01:40.859 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-08T06:01:40.859 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:01:40.859 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-08T06:01:40.859 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-08T06:01:40.859 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-08T06:01:40.859 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:01:40.860 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-08T06:01:40.860 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-08T06:01:40.860 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-08T06:01:40.860 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:01:40.870 INFO:tasks.ceph.mds.a:Stopped 2024-03-08T06:01:40.870 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-08T06:01:40.871 INFO:tasks.ceph.mds.c:Stopped 2024-03-08T06:01:40.871 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-08T06:01:40.872 INFO:tasks.ceph.mds.b:Stopped 2024-03-08T06:01:40.872 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-08T06:01:40.873 INFO:tasks.ceph.mds.d:Stopped 2024-03-08T06:01:40.873 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-08T06:01:40.913 INFO:tasks.ceph.mds.a:Started 2024-03-08T06:01:40.913 DEBUG:teuthology.parallel:result is None 2024-03-08T06:01:40.914 INFO:tasks.ceph.mds.c:Started 2024-03-08T06:01:40.914 DEBUG:teuthology.parallel:result is None 2024-03-08T06:01:40.915 INFO:tasks.ceph.mds.b:Started 2024-03-08T06:01:40.915 DEBUG:teuthology.parallel:result is None 2024-03-08T06:01:40.916 INFO:tasks.ceph.mds.d:Started 2024-03-08T06:01:40.916 DEBUG:teuthology.parallel:result is None 2024-03-08T06:01:40.916 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-08T06:01:40.990 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-03-08T06:01:40.990 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2024-03-08T06:01:40.999 INFO:tasks.ceph.mds.b.smithi081.stdout:starting mds.b at 2024-03-08T06:01:40.999 INFO:tasks.ceph.mds.d.smithi081.stdout:starting mds.d at 2024-03-08T06:01:41.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.021+0000 7f184bcfb640 1 Processor -- start 2024-03-08T06:01:41.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.022+0000 7f184bcfb640 1 -- start start 2024-03-08T06:01:41.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.022+0000 7f184bcfb640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1844161210 0x7f18440cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:41.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.022+0000 7f184bcfb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1844071670 0x7f18440cc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:41.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.022+0000 7f184bcfb640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1844070470 con 0x7f184406d040 2024-03-08T06:01:41.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.022+0000 7f184bcfb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f184406bfa0 con 0x7f1844071670 2024-03-08T06:01:41.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.022+0000 7f184bcfb640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184406c280 con 0x7f1844161210 2024-03-08T06:01:41.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.023+0000 7f184a298640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1844071670 0x7f18440cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:41.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.023+0000 7f184a298640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1844071670 0x7f18440cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45432/0 (socket says 172.21.15.76:45432) 2024-03-08T06:01:41.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.023+0000 7f184a298640 1 -- 172.21.15.76:0/536602628 learned_addr learned my addr 172.21.15.76:0/536602628 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:41.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.023+0000 7f1849a97640 1 --2- 172.21.15.76:0/536602628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1844161210 0x7f18440cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:41.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.023+0000 7f184a298640 1 -- 172.21.15.76:0/536602628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1844161210 msgr2=0x7f18440cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:41.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.023+0000 7f184a298640 1 --2- 172.21.15.76:0/536602628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1844161210 0x7f18440cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:41.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.023+0000 7f184a298640 1 -- 172.21.15.76:0/536602628 >> v1:172.21.15.81:6789/0 conn(0x7f184406d040 legacy=0x7f1844070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:41.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.023+0000 7f184a298640 1 -- 172.21.15.76:0/536602628 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184406c560 con 0x7f1844071670 2024-03-08T06:01:41.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.024+0000 7f184a298640 1 --2- 172.21.15.76:0/536602628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1844071670 0x7f18440cc060 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f183c009e50 tx=0x7f183c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5fb00e0ac3702c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:41.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.024+0000 7f1848a95640 1 -- 172.21.15.76:0/536602628 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c02cb00 con 0x7f1844071670 2024-03-08T06:01:41.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.024+0000 7f1848a95640 1 -- 172.21.15.76:0/536602628 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c02cca0 con 0x7f1844071670 2024-03-08T06:01:41.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.024+0000 7f1848a95640 1 -- 172.21.15.76:0/536602628 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c034610 con 0x7f1844071670 2024-03-08T06:01:41.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.024+0000 7f184bcfb640 1 -- 172.21.15.76:0/536602628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1844071670 msgr2=0x7f18440cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:41.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.024+0000 7f184bcfb640 1 --2- 172.21.15.76:0/536602628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1844071670 0x7f18440cc060 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f183c009e50 tx=0x7f183c02bf60 comp rx=0 tx=0).stop 2024-03-08T06:01:41.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.024+0000 7f184bcfb640 1 -- 172.21.15.76:0/536602628 shutdown_connections 2024-03-08T06:01:41.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.024+0000 7f184bcfb640 1 --2- 172.21.15.76:0/536602628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1844071670 0x7f18440cc060 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:41.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.024+0000 7f184bcfb640 1 --2- 172.21.15.76:0/536602628 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1844161210 0x7f18440cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:41.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.024+0000 7f184bcfb640 1 -- 172.21.15.76:0/536602628 >> 172.21.15.76:0/536602628 conn(0x7f184406f530 msgr2=0x7f184406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:41.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.024+0000 7f184bcfb640 1 -- 172.21.15.76:0/536602628 shutdown_connections 2024-03-08T06:01:41.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.024+0000 7f184bcfb640 1 -- 172.21.15.76:0/536602628 wait complete. 2024-03-08T06:01:41.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.025+0000 7f184bcfb640 1 Processor -- start 2024-03-08T06:01:41.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.026+0000 7f184bcfb640 1 -- start start 2024-03-08T06:01:41.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.026+0000 7f184bcfb640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f184406d040 0x7f184415cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:41.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.026+0000 7f184bcfb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1844071670 0x7f184415d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:41.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.026+0000 7f184bcfb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1844161210 0x7f184415dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:41.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.026+0000 7f184bcfb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184406d4d0 con 0x7f1844161210 2024-03-08T06:01:41.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.026+0000 7f184bcfb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184406cb20 con 0x7f1844071670 2024-03-08T06:01:41.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.026+0000 7f184bcfb640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184406d650 con 0x7f184406d040 2024-03-08T06:01:41.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.026+0000 7f184a298640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1844161210 0x7f184415dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:41.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.026+0000 7f184a298640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1844161210 0x7f184415dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45436/0 (socket says 172.21.15.76:45436) 2024-03-08T06:01:41.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.026+0000 7f184a298640 1 -- 172.21.15.76:0/2382484756 learned_addr learned my addr 172.21.15.76:0/2382484756 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:41.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.027+0000 7f1849a97640 1 --2- 172.21.15.76:0/2382484756 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f184406d040 0x7f184415cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:41.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.027+0000 7f1849296640 1 --2- 172.21.15.76:0/2382484756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1844071670 0x7f184415d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:41.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.027+0000 7f184a298640 1 -- 172.21.15.76:0/2382484756 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f184406d040 msgr2=0x7f184415cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:41.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.027+0000 7f184a298640 1 --2- 172.21.15.76:0/2382484756 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f184406d040 0x7f184415cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:41.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.027+0000 7f184a298640 1 -- 172.21.15.76:0/2382484756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1844071670 msgr2=0x7f184415d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:41.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.027+0000 7f184a298640 1 --2- 172.21.15.76:0/2382484756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1844071670 0x7f184415d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:41.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.027+0000 7f184a298640 1 -- 172.21.15.76:0/2382484756 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18441f39a0 con 0x7f1844161210 2024-03-08T06:01:41.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.027+0000 7f1849a97640 1 --2- 172.21.15.76:0/2382484756 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f184406d040 0x7f184415cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:41.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.027+0000 7f1849296640 1 --2- 172.21.15.76:0/2382484756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1844071670 0x7f184415d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:41.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.027+0000 7f184a298640 1 --2- 172.21.15.76:0/2382484756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1844161210 0x7f184415dde0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f183c009e50 tx=0x7f183c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:41.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.027+0000 7f183a7fc640 1 -- 172.21.15.76:0/2382484756 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c043020 con 0x7f1844161210 2024-03-08T06:01:41.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.027+0000 7f183a7fc640 1 -- 172.21.15.76:0/2382484756 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c02ce50 con 0x7f1844161210 2024-03-08T06:01:41.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.027+0000 7f184bcfb640 1 -- 172.21.15.76:0/2382484756 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18440c6900 con 0x7f1844161210 2024-03-08T06:01:41.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.027+0000 7f183a7fc640 1 -- 172.21.15.76:0/2382484756 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c009510 con 0x7f1844161210 2024-03-08T06:01:41.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.027+0000 7f184bcfb640 1 -- 172.21.15.76:0/2382484756 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18440c6ec0 con 0x7f1844161210 2024-03-08T06:01:41.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.028+0000 7f184bcfb640 1 -- 172.21.15.76:0/2382484756 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18441f3b70 con 0x7f1844161210 2024-03-08T06:01:41.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.029+0000 7f183a7fc640 1 -- 172.21.15.76:0/2382484756 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f183c046020 con 0x7f1844161210 2024-03-08T06:01:41.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.029+0000 7f183a7fc640 1 --2- 172.21.15.76:0/2382484756 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f182806b4d0 0x7f182806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:41.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.029+0000 7f183a7fc640 1 -- 172.21.15.76:0/2382484756 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f183c0ac7a0 con 0x7f1844161210 2024-03-08T06:01:41.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.029+0000 7f1849a97640 1 --2- 172.21.15.76:0/2382484756 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f182806b4d0 0x7f182806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:41.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.029+0000 7f1849a97640 1 --2- 172.21.15.76:0/2382484756 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f182806b4d0 0x7f182806d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1840006fd0 tx=0x7f1840008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:41.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.033+0000 7f183a7fc640 1 -- 172.21.15.76:0/2382484756 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f183c04d030 con 0x7f1844161210 2024-03-08T06:01:41.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.216+0000 7f184bcfb640 1 -- 172.21.15.76:0/2382484756 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f18440b7850 con 0x7f1844161210 2024-03-08T06:01:41.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.848+0000 7f183a7fc640 1 -- 172.21.15.76:0/2382484756 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) v1 ==== 97+0+0 (secure 0 0 0) 0x7f183c07b700 con 0x7f1844161210 2024-03-08T06:01:41.849 INFO:teuthology.orchestra.run.smithi076.stderr: removed all blocklist entries 2024-03-08T06:01:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.851+0000 7f184bcfb640 1 -- 172.21.15.76:0/2382484756 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f182806b4d0 msgr2=0x7f182806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.851+0000 7f184bcfb640 1 --2- 172.21.15.76:0/2382484756 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f182806b4d0 0x7f182806d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1840006fd0 tx=0x7f1840008040 comp rx=0 tx=0).stop 2024-03-08T06:01:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.851+0000 7f184bcfb640 1 -- 172.21.15.76:0/2382484756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1844161210 msgr2=0x7f184415dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.851+0000 7f184bcfb640 1 --2- 172.21.15.76:0/2382484756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1844161210 0x7f184415dde0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f183c009e50 tx=0x7f183c004a40 comp rx=0 tx=0).stop 2024-03-08T06:01:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.851+0000 7f184bcfb640 1 -- 172.21.15.76:0/2382484756 shutdown_connections 2024-03-08T06:01:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.851+0000 7f184bcfb640 1 --2- 172.21.15.76:0/2382484756 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f182806b4d0 0x7f182806d990 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.851+0000 7f184bcfb640 1 --2- 172.21.15.76:0/2382484756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1844161210 0x7f184415dde0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.851+0000 7f184bcfb640 1 --2- 172.21.15.76:0/2382484756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1844071670 0x7f184415d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.851+0000 7f184bcfb640 1 --2- 172.21.15.76:0/2382484756 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f184406d040 0x7f184415cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.851+0000 7f184bcfb640 1 -- 172.21.15.76:0/2382484756 >> 172.21.15.76:0/2382484756 conn(0x7f184406f530 msgr2=0x7f1844160210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.851+0000 7f184bcfb640 1 -- 172.21.15.76:0/2382484756 shutdown_connections 2024-03-08T06:01:41.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.851+0000 7f184bcfb640 1 -- 172.21.15.76:0/2382484756 wait complete. 2024-03-08T06:01:41.867 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-08T06:01:41.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.968+0000 7f7046881640 1 Processor -- start 2024-03-08T06:01:41.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.968+0000 7f7046881640 1 -- start start 2024-03-08T06:01:41.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.969+0000 7f7046881640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f70400d0b90 0x7f7040168bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:41.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.969+0000 7f7046881640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70400cf180 con 0x7f70400b6d00 2024-03-08T06:01:41.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.969+0000 7f7046881640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70400d1ed0 con 0x7f704015dde0 2024-03-08T06:01:41.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.969+0000 7f7046881640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70400d21b0 con 0x7f70400d0b90 2024-03-08T06:01:41.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.969+0000 7f703f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f70400b6d00 0x7f70400d0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48440/0 (socket says 172.21.15.76:48440) 2024-03-08T06:01:41.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.969+0000 7f703f7fe640 1 -- 172.21.15.76:0/3266080007 learned_addr learned my addr 172.21.15.76:0/3266080007 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:41.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703effd640 1 -- 172.21.15.76:0/3266080007 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4005446768 0 0) 0x7f70400cf180 con 0x7f70400b6d00 2024-03-08T06:01:41.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703effd640 1 -- 172.21.15.76:0/3266080007 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7028003520 con 0x7f70400b6d00 2024-03-08T06:01:41.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f7044e1e640 1 --2- 172.21.15.76:0/3266080007 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f70400d0b90 0x7f7040168bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:41.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703effd640 1 -- 172.21.15.76:0/3266080007 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 678989411 0 0) 0x7f7028003520 con 0x7f70400b6d00 2024-03-08T06:01:41.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703effd640 1 -- 172.21.15.76:0/3266080007 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f70400d0b90 msgr2=0x7f7040168bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:41.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703effd640 1 --2- 172.21.15.76:0/3266080007 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f70400d0b90 0x7f7040168bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:41.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703effd640 1 -- 172.21.15.76:0/3266080007 >> v1:172.21.15.81:6789/0 conn(0x7f704015dde0 legacy=0x7f7040162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:41.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703effd640 1 -- 172.21.15.76:0/3266080007 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70400cec10 con 0x7f70400b6d00 2024-03-08T06:01:41.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703effd640 1 -- 172.21.15.76:0/3266080007 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f7034001670 con 0x7f70400b6d00 2024-03-08T06:01:41.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f7044e1e640 1 --2- 172.21.15.76:0/3266080007 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f70400d0b90 0x7f7040168bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:41.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703effd640 1 -- 172.21.15.76:0/3266080007 >> v1:172.21.15.76:6789/0 conn(0x7f70400b6d00 legacy=0x7f70400d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:41.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703effd640 1 --2- 172.21.15.76:0/3266080007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7028003a50 0x7f7028003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:41.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703effd640 1 --2- 172.21.15.76:0/3266080007 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f70400d0b90 0x7f7028004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:41.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703effd640 1 --2- 172.21.15.76:0/3266080007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7028004d40 0x7f70280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:41.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703effd640 1 -- 172.21.15.76:0/3266080007 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70400d21b0 con 0x7f7028004d40 2024-03-08T06:01:41.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703effd640 1 -- 172.21.15.76:0/3266080007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702800bc00 con 0x7f7028003a50 2024-03-08T06:01:41.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703effd640 1 -- 172.21.15.76:0/3266080007 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f702800be00 con 0x7f70400d0b90 2024-03-08T06:01:41.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f7044e1e640 1 --2- 172.21.15.76:0/3266080007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7028004d40 0x7f70280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:41.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703f7fe640 1 --2- 172.21.15.76:0/3266080007 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f70400d0b90 0x7f7028004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:41.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f703ffff640 1 --2- 172.21.15.76:0/3266080007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7028003a50 0x7f7028003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:41.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f7044e1e640 1 -- 172.21.15.76:0/3266080007 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f70400d0b90 msgr2=0x7f7028004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:41.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f7044e1e640 1 --2- 172.21.15.76:0/3266080007 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f70400d0b90 0x7f7028004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:41.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f7044e1e640 1 -- 172.21.15.76:0/3266080007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7028003a50 msgr2=0x7f7028003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:41.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f7044e1e640 1 --2- 172.21.15.76:0/3266080007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7028003a50 0x7f7028003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:41.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.970+0000 7f7044e1e640 1 -- 172.21.15.76:0/3266080007 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7028003520 con 0x7f7028004d40 2024-03-08T06:01:41.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.971+0000 7f7044e1e640 1 --2- 172.21.15.76:0/3266080007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7028004d40 0x7f70280051c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f70400d0fa0 tx=0x7f7028005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c59bfeb7f61c2c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:41.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.971+0000 7f703effd640 1 -- 172.21.15.76:0/3266080007 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038039070 con 0x7f7028004d40 2024-03-08T06:01:41.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.971+0000 7f703effd640 1 -- 172.21.15.76:0/3266080007 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703802ccd0 con 0x7f7028004d40 2024-03-08T06:01:41.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.972+0000 7f703effd640 1 -- 172.21.15.76:0/3266080007 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038009c80 con 0x7f7028004d40 2024-03-08T06:01:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.972+0000 7f7046881640 1 -- 172.21.15.76:0/3266080007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7028004d40 msgr2=0x7f70280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.972+0000 7f7046881640 1 --2- 172.21.15.76:0/3266080007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7028004d40 0x7f70280051c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f70400d0fa0 tx=0x7f7028005830 comp rx=0 tx=0).stop 2024-03-08T06:01:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.972+0000 7f7044e1e640 1 -- 172.21.15.76:0/3266080007 reap_dead start 2024-03-08T06:01:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.972+0000 7f7046881640 1 -- 172.21.15.76:0/3266080007 shutdown_connections 2024-03-08T06:01:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.972+0000 7f7046881640 1 -- 172.21.15.76:0/3266080007 >> 172.21.15.76:0/3266080007 conn(0x7f70400b56e0 msgr2=0x7f70400b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.973+0000 7f7046881640 1 -- 172.21.15.76:0/3266080007 shutdown_connections 2024-03-08T06:01:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.973+0000 7f7046881640 1 -- 172.21.15.76:0/3266080007 wait complete. 2024-03-08T06:01:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.974+0000 7f7046881640 1 Processor -- start 2024-03-08T06:01:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.974+0000 7f7046881640 1 -- start start 2024-03-08T06:01:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f7046881640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7028004d40 0x7f7040162d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f7046881640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7040165670 0x7f70400c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f7046881640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f704015bf60 0x7f704015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f7046881640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7040166d40 con 0x7f7028004d40 2024-03-08T06:01:41.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f7046881640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7040166bc0 con 0x7f7040165670 2024-03-08T06:01:41.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f7046881640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7040166ec0 con 0x7f704015bf60 2024-03-08T06:01:41.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f703ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7028004d40 0x7f7040162d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:41.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f703ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7028004d40 0x7f7040162d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45448/0 (socket says 172.21.15.76:45448) 2024-03-08T06:01:41.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f703ffff640 1 -- 172.21.15.76:0/1583072151 learned_addr learned my addr 172.21.15.76:0/1583072151 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:41.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f703f7fe640 1 --2- 172.21.15.76:0/1583072151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7040165670 0x7f70400c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:41.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f7044e1e640 1 --2- 172.21.15.76:0/1583072151 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f704015bf60 0x7f704015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:41.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f703ffff640 1 -- 172.21.15.76:0/1583072151 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f704015bf60 msgr2=0x7f704015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:41.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f703ffff640 1 --2- 172.21.15.76:0/1583072151 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f704015bf60 0x7f704015c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:41.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f703ffff640 1 -- 172.21.15.76:0/1583072151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7040165670 msgr2=0x7f70400c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:41.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f703ffff640 1 --2- 172.21.15.76:0/1583072151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7040165670 0x7f70400c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:41.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f703ffff640 1 -- 172.21.15.76:0/1583072151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70400ca410 con 0x7f7028004d40 2024-03-08T06:01:41.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f703ffff640 1 --2- 172.21.15.76:0/1583072151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7028004d40 0x7f7040162d20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f703000f880 tx=0x7f703000fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:41.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f703cff9640 1 -- 172.21.15.76:0/1583072151 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70300050d0 con 0x7f7028004d40 2024-03-08T06:01:41.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f703cff9640 1 -- 172.21.15.76:0/1583072151 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7030018070 con 0x7f7028004d40 2024-03-08T06:01:41.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f703cff9640 1 -- 172.21.15.76:0/1583072151 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703000d730 con 0x7f7028004d40 2024-03-08T06:01:41.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f7046881640 1 -- 172.21.15.76:0/1583072151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7040165e00 con 0x7f7028004d40 2024-03-08T06:01:41.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.975+0000 7f7046881640 1 -- 172.21.15.76:0/1583072151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70401663c0 con 0x7f7028004d40 2024-03-08T06:01:41.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.976+0000 7f7046881640 1 -- 172.21.15.76:0/1583072151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70400cf5d0 con 0x7f7028004d40 2024-03-08T06:01:41.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.977+0000 7f703cff9640 1 -- 172.21.15.76:0/1583072151 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f703000d8d0 con 0x7f7028004d40 2024-03-08T06:01:41.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.977+0000 7f703cff9640 1 --2- 172.21.15.76:0/1583072151 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f702c06b4d0 0x7f702c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:41.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.977+0000 7f703cff9640 1 -- 172.21.15.76:0/1583072151 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7030089d10 con 0x7f7028004d40 2024-03-08T06:01:41.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.977+0000 7f703f7fe640 1 --2- 172.21.15.76:0/1583072151 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f702c06b4d0 0x7f702c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:41.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.978+0000 7f703f7fe640 1 --2- 172.21.15.76:0/1583072151 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f702c06b4d0 0x7f702c06d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f704015cd30 tx=0x7f7034001c40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:41.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:41.982+0000 7f703cff9640 1 -- 172.21.15.76:0/1583072151 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70400cf5d0 con 0x7f7028004d40 2024-03-08T06:01:42.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.215+0000 7f7046881640 1 -- 172.21.15.76:0/1583072151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f704004f030 con 0x7f7028004d40 2024-03-08T06:01:42.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.215+0000 7f703cff9640 1 -- 172.21.15.76:0/1583072151 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f7030058d50 con 0x7f7028004d40 2024-03-08T06:01:42.216 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:42.216 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-08T06:01:42.216 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_dump": [ 2024-03-08T06:01:42.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.216 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.a", 2024-03-08T06:01:42.216 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBKqeplZeJjARAAlfjyyegTL7rXiO5oZlnk0w==", 2024-03-08T06:01:42.216 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.216 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-03-08T06:01:42.216 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-03-08T06:01:42.217 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-03-08T06:01:42.217 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-03-08T06:01:42.217 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.217 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.b", 2024-03-08T06:01:42.217 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBKqepla2TDDRAAKz8pGbF08Fj6g6wH9plqEQ==", 2024-03-08T06:01:42.217 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.217 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-03-08T06:01:42.217 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-03-08T06:01:42.217 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-03-08T06:01:42.217 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-03-08T06:01:42.217 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.218 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.c", 2024-03-08T06:01:42.218 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBKqepljg3KCBAAloYX61Kt+Mo/LxsQ6ZWbkA==", 2024-03-08T06:01:42.218 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.218 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-03-08T06:01:42.218 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-03-08T06:01:42.218 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-03-08T06:01:42.218 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-03-08T06:01:42.218 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.218 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.d", 2024-03-08T06:01:42.218 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBKqepl8AMiFRAAzfoDnTWrB8kMbPrjIhseVg==", 2024-03-08T06:01:42.218 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.219 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-03-08T06:01:42.219 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-03-08T06:01:42.219 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-03-08T06:01:42.219 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-03-08T06:01:42.219 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.219 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.0", 2024-03-08T06:01:42.219 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBNqeplwpn1OBAAACkPVBFCXbuovt8Eu6gswQ==", 2024-03-08T06:01:42.219 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.219 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-03-08T06:01:42.219 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-03-08T06:01:42.219 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-03-08T06:01:42.219 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.220 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.1", 2024-03-08T06:01:42.220 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBQqeplSbr8NxAASBqBdSC3wDAkaXnJ4aveEg==", 2024-03-08T06:01:42.220 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.220 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-03-08T06:01:42.220 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-03-08T06:01:42.220 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-03-08T06:01:42.220 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.220 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.2", 2024-03-08T06:01:42.220 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBTqepl9kB2NhAArI6PJK6a9jPuby86K+sviA==", 2024-03-08T06:01:42.220 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.220 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-03-08T06:01:42.221 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-03-08T06:01:42.221 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-03-08T06:01:42.221 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.221 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.3", 2024-03-08T06:01:42.221 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBWqeplLwQ4NRAAjR9l83jSoKb8cJrIBcxZMg==", 2024-03-08T06:01:42.221 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.221 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-03-08T06:01:42.221 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-03-08T06:01:42.221 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-03-08T06:01:42.221 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.221 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.4", 2024-03-08T06:01:42.221 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBcqeplqQsLOxAABHoIXSPMqfFZK0rZmV8GbQ==", 2024-03-08T06:01:42.222 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.222 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-03-08T06:01:42.222 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-03-08T06:01:42.222 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-03-08T06:01:42.222 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.222 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.5", 2024-03-08T06:01:42.222 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBfqepl+o58ORAAC0EruVCdKFdUokPiKS8OAA==", 2024-03-08T06:01:42.222 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.222 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-03-08T06:01:42.222 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-03-08T06:01:42.222 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-03-08T06:01:42.222 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.223 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.6", 2024-03-08T06:01:42.223 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBiqeplabHNNhAA7Uso+ALm1TwwkbtUVkmMvw==", 2024-03-08T06:01:42.223 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.223 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-03-08T06:01:42.223 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-03-08T06:01:42.223 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-03-08T06:01:42.223 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.223 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.7", 2024-03-08T06:01:42.223 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBlqepl16BeNBAAkHN6ceewaEZzsTZcqTqmDA==", 2024-03-08T06:01:42.223 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.223 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-03-08T06:01:42.223 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-03-08T06:01:42.223 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-03-08T06:01:42.224 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.224 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.0", 2024-03-08T06:01:42.224 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBKqepl3elrGRAAXLq7DBlUer23iSXOoBZatA==", 2024-03-08T06:01:42.224 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.224 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-03-08T06:01:42.224 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-03-08T06:01:42.224 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-03-08T06:01:42.224 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-03-08T06:01:42.224 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.224 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.1", 2024-03-08T06:01:42.224 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBKqeplhteqIhAAuwfBLYJRuYCaz3BTclKqwg==", 2024-03-08T06:01:42.224 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.225 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-03-08T06:01:42.225 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-03-08T06:01:42.225 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-03-08T06:01:42.225 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-03-08T06:01:42.225 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.225 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.2", 2024-03-08T06:01:42.225 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBKqepls1FEHxAA+hIAqCWzjGxFRYdAgdDRQw==", 2024-03-08T06:01:42.225 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.225 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-03-08T06:01:42.225 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-03-08T06:01:42.225 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-03-08T06:01:42.225 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-03-08T06:01:42.225 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.226 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.3", 2024-03-08T06:01:42.226 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBKqepl38B0KBAAwHBqLkwC4Sl9N9jDYFvfUg==", 2024-03-08T06:01:42.226 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.226 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-03-08T06:01:42.226 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-03-08T06:01:42.226 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-03-08T06:01:42.226 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-03-08T06:01:42.226 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.226 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.admin", 2024-03-08T06:01:42.226 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBJqeplQukbIBAAtoVurz6VrURxd0H6lZtBUw==", 2024-03-08T06:01:42.226 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.227 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-03-08T06:01:42.227 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-03-08T06:01:42.227 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-03-08T06:01:42.227 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-03-08T06:01:42.227 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.227 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-mds", 2024-03-08T06:01:42.227 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBrqeplFH9aLBAAMcRO2bd3lIXBdLwgvpOAjw==", 2024-03-08T06:01:42.227 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.227 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-mds" 2024-03-08T06:01:42.227 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.227 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-mgr", 2024-03-08T06:01:42.228 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBrqeplcIdaLBAA3lqVHKvZZeHVyM9YmUxc/A==", 2024-03-08T06:01:42.228 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.228 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-08T06:01:42.228 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.228 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-osd", 2024-03-08T06:01:42.228 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBrqepl2I1aLBAA+EgKDoccrJoZ3pfjdEgsNQ==", 2024-03-08T06:01:42.228 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.228 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-osd" 2024-03-08T06:01:42.228 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.228 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-rbd", 2024-03-08T06:01:42.228 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBrqeplNpRaLBAAGe6OHQQZb/Fr0LkS/pgAdg==", 2024-03-08T06:01:42.228 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.229 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-08T06:01:42.229 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.229 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-08T06:01:42.229 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBrqepl2JpaLBAAq07RpFEx18uIr6JGuaQJAQ==", 2024-03-08T06:01:42.229 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.229 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-08T06:01:42.229 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.229 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-rgw", 2024-03-08T06:01:42.229 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBrqepl2qFaLBAAiAreAbjiVBECq9HmN84Mrw==", 2024-03-08T06:01:42.229 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.229 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-08T06:01:42.229 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.230 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.x", 2024-03-08T06:01:42.230 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBJqeplqvWVORAAUysSLEF4wAs62maeFdG60w==", 2024-03-08T06:01:42.230 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.230 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-03-08T06:01:42.230 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2024-03-08T06:01:42.230 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-03-08T06:01:42.230 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:01:42.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:42.230 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.y", 2024-03-08T06:01:42.230 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQBJqeplOJ3CNRAA6hv+A1Bmwarz4eo2wqVNZQ==", 2024-03-08T06:01:42.230 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-03-08T06:01:42.231 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-03-08T06:01:42.231 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2024-03-08T06:01:42.231 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-03-08T06:01:42.231 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.231 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:42.231 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-03-08T06:01:42.231 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-08T06:01:42.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.217+0000 7f7046881640 1 -- 172.21.15.76:0/1583072151 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f702c06b4d0 msgr2=0x7f702c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:42.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.217+0000 7f7046881640 1 --2- 172.21.15.76:0/1583072151 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f702c06b4d0 0x7f702c06d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f704015cd30 tx=0x7f7034001c40 comp rx=0 tx=0).stop 2024-03-08T06:01:42.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.217+0000 7f7046881640 1 -- 172.21.15.76:0/1583072151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7028004d40 msgr2=0x7f7040162d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:42.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.217+0000 7f7046881640 1 --2- 172.21.15.76:0/1583072151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7028004d40 0x7f7040162d20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f703000f880 tx=0x7f703000fc80 comp rx=0 tx=0).stop 2024-03-08T06:01:42.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.217+0000 7f7046881640 1 -- 172.21.15.76:0/1583072151 shutdown_connections 2024-03-08T06:01:42.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.217+0000 7f7046881640 1 --2- 172.21.15.76:0/1583072151 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f702c06b4d0 0x7f702c06d990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.217+0000 7f7046881640 1 --2- 172.21.15.76:0/1583072151 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f704015bf60 0x7f704015c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.217+0000 7f7046881640 1 --2- 172.21.15.76:0/1583072151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7040165670 0x7f70400c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.217+0000 7f7046881640 1 --2- 172.21.15.76:0/1583072151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7028004d40 0x7f7040162d20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.217+0000 7f7046881640 1 -- 172.21.15.76:0/1583072151 >> 172.21.15.76:0/1583072151 conn(0x7f70400b56e0 msgr2=0x7f70400b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:42.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.217+0000 7f7046881640 1 -- 172.21.15.76:0/1583072151 shutdown_connections 2024-03-08T06:01:42.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.217+0000 7f7046881640 1 -- 172.21.15.76:0/1583072151 wait complete. 2024-03-08T06:01:42.233 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-08T06:01:42.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.334+0000 7f1ba63ac640 1 Processor -- start 2024-03-08T06:01:42.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.335+0000 7f1ba63ac640 1 -- start start 2024-03-08T06:01:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.335+0000 7f1ba63ac640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1ba006d8a0 0x7f1ba0162020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.335+0000 7f1ba63ac640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba0162560 0x7f1ba015dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.335+0000 7f1ba63ac640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba0161840 con 0x7f1ba00ce5e0 2024-03-08T06:01:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.335+0000 7f1ba63ac640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba0161b90 con 0x7f1ba006d8a0 2024-03-08T06:01:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.335+0000 7f1ba63ac640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba015eb30 con 0x7f1ba0162560 2024-03-08T06:01:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.336+0000 7f1b9f7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1ba006d8a0 0x7f1ba0162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.336+0000 7f1ba4949640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba0162560 0x7f1ba015dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.336+0000 7f1b9f7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1ba006d8a0 0x7f1ba0162020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.76:41884/0 (socket says 172.21.15.76:41884) 2024-03-08T06:01:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.336+0000 7f1b9ffff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f1ba00ce5e0 0x7f1ba00ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:35958/0 (socket says 172.21.15.76:35958) 2024-03-08T06:01:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.336+0000 7f1b9f7fe640 1 -- 172.21.15.76:0/4071363813 learned_addr learned my addr 172.21.15.76:0/4071363813 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.336+0000 7f1ba4949640 1 -- 172.21.15.76:0/4071363813 >> v2:172.21.15.81:3300/0 conn(0x7f1ba006d8a0 msgr2=0x7f1ba0162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:42.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.336+0000 7f1ba4949640 1 --2- 172.21.15.76:0/4071363813 >> v2:172.21.15.81:3300/0 conn(0x7f1ba006d8a0 0x7f1ba0162020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.336+0000 7f1ba4949640 1 -- 172.21.15.76:0/4071363813 >> v1:172.21.15.81:6789/0 conn(0x7f1ba00ce5e0 legacy=0x7f1ba00ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:42.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.336+0000 7f1ba4949640 1 -- 172.21.15.76:0/4071363813 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba015ee10 con 0x7f1ba0162560 2024-03-08T06:01:42.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.336+0000 7f1b9f7fe640 1 --2- 172.21.15.76:0/4071363813 >> v2:172.21.15.81:3300/0 conn(0x7f1ba006d8a0 0x7f1ba0162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:42.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.337+0000 7f1ba4949640 1 --2- 172.21.15.76:0/4071363813 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba0162560 0x7f1ba015dfb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009d90 tx=0x7f1b9802bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e8eca6e26ded981 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:42.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.337+0000 7f1b9effd640 1 -- 172.21.15.76:0/4071363813 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b98039070 con 0x7f1ba0162560 2024-03-08T06:01:42.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.337+0000 7f1b9effd640 1 -- 172.21.15.76:0/4071363813 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9802cce0 con 0x7f1ba0162560 2024-03-08T06:01:42.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.337+0000 7f1b9effd640 1 -- 172.21.15.76:0/4071363813 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b98034660 con 0x7f1ba0162560 2024-03-08T06:01:42.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.337+0000 7f1ba63ac640 1 -- 172.21.15.76:0/4071363813 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba0162560 msgr2=0x7f1ba015dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:42.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.337+0000 7f1ba63ac640 1 --2- 172.21.15.76:0/4071363813 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba0162560 0x7f1ba015dfb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009d90 tx=0x7f1b9802bf30 comp rx=0 tx=0).stop 2024-03-08T06:01:42.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.338+0000 7f1ba63ac640 1 -- 172.21.15.76:0/4071363813 shutdown_connections 2024-03-08T06:01:42.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.338+0000 7f1ba63ac640 1 --2- 172.21.15.76:0/4071363813 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba0162560 0x7f1ba015dfb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.338+0000 7f1ba63ac640 1 --2- 172.21.15.76:0/4071363813 >> v2:172.21.15.81:3300/0 conn(0x7f1ba006d8a0 0x7f1ba0162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.338+0000 7f1ba63ac640 1 -- 172.21.15.76:0/4071363813 >> 172.21.15.76:0/4071363813 conn(0x7f1ba006ae60 msgr2=0x7f1ba006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:42.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.338+0000 7f1ba63ac640 1 -- 172.21.15.76:0/4071363813 shutdown_connections 2024-03-08T06:01:42.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.338+0000 7f1ba63ac640 1 -- 172.21.15.76:0/4071363813 wait complete. 2024-03-08T06:01:42.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.339+0000 7f1ba63ac640 1 Processor -- start 2024-03-08T06:01:42.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.339+0000 7f1ba63ac640 1 -- start start 2024-03-08T06:01:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1ba63ac640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba006d8a0 0x7f1ba014fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1ba63ac640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ba00ce5e0 0x7f1ba01e9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1ba63ac640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ba0162560 0x7f1ba01e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1ba63ac640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba015e670 con 0x7f1ba00ce5e0 2024-03-08T06:01:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1ba63ac640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba015e4f0 con 0x7f1ba0162560 2024-03-08T06:01:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1ba63ac640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba015e7f0 con 0x7f1ba006d8a0 2024-03-08T06:01:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1b9f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ba00ce5e0 0x7f1ba01e9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1b9f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ba00ce5e0 0x7f1ba01e9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45464/0 (socket says 172.21.15.76:45464) 2024-03-08T06:01:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1b9f7fe640 1 -- 172.21.15.76:0/3543823550 learned_addr learned my addr 172.21.15.76:0/3543823550 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1b9ffff640 1 --2- 172.21.15.76:0/3543823550 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba006d8a0 0x7f1ba014fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1ba4949640 1 --2- 172.21.15.76:0/3543823550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ba0162560 0x7f1ba01e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1b9f7fe640 1 -- 172.21.15.76:0/3543823550 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba006d8a0 msgr2=0x7f1ba014fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1b9f7fe640 1 --2- 172.21.15.76:0/3543823550 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba006d8a0 0x7f1ba014fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1b9f7fe640 1 -- 172.21.15.76:0/3543823550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ba0162560 msgr2=0x7f1ba01e9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1b9f7fe640 1 --2- 172.21.15.76:0/3543823550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ba0162560 0x7f1ba01e9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1b9f7fe640 1 -- 172.21.15.76:0/3543823550 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba00ca7d0 con 0x7f1ba00ce5e0 2024-03-08T06:01:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1b9f7fe640 1 --2- 172.21.15.76:0/3543823550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ba00ce5e0 0x7f1ba01e9870 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400e9e0 tx=0x7f1b9400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1b9cff9640 1 -- 172.21.15.76:0/3543823550 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94017070 con 0x7f1ba00ce5e0 2024-03-08T06:01:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1b9cff9640 1 -- 172.21.15.76:0/3543823550 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9400c9d0 con 0x7f1ba00ce5e0 2024-03-08T06:01:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1ba63ac640 1 -- 172.21.15.76:0/3543823550 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba00caab0 con 0x7f1ba00ce5e0 2024-03-08T06:01:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1b9cff9640 1 -- 172.21.15.76:0/3543823550 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9400cd80 con 0x7f1ba00ce5e0 2024-03-08T06:01:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.340+0000 7f1ba63ac640 1 -- 172.21.15.76:0/3543823550 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba00cb070 con 0x7f1ba00ce5e0 2024-03-08T06:01:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.341+0000 7f1ba63ac640 1 -- 172.21.15.76:0/3543823550 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba004f030 con 0x7f1ba00ce5e0 2024-03-08T06:01:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.342+0000 7f1b9cff9640 1 -- 172.21.15.76:0/3543823550 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b9401a070 con 0x7f1ba00ce5e0 2024-03-08T06:01:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.342+0000 7f1b9cff9640 1 --2- 172.21.15.76:0/3543823550 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b8c06b4d0 0x7f1b8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.342+0000 7f1b9cff9640 1 -- 172.21.15.76:0/3543823550 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1b9408a280 con 0x7f1ba00ce5e0 2024-03-08T06:01:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.342+0000 7f1b9ffff640 1 --2- 172.21.15.76:0/3543823550 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b8c06b4d0 0x7f1b8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:42.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.343+0000 7f1b9ffff640 1 --2- 172.21.15.76:0/3543823550 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b8c06b4d0 0x7f1b8c06d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00ce180 tx=0x7f1b90007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:42.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.346+0000 7f1b9cff9640 1 -- 172.21.15.76:0/3543823550 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b940592c0 con 0x7f1ba00ce5e0 2024-03-08T06:01:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.587+0000 7f1ba63ac640 1 -- 172.21.15.76:0/3543823550 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f1ba00b4330 con 0x7f1ba00ce5e0 2024-03-08T06:01:42.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.589+0000 7f1b9cff9640 1 -- 172.21.15.76:0/3543823550 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1b94058bc0 con 0x7f1ba00ce5e0 2024-03-08T06:01:42.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.591+0000 7f1ba63ac640 1 -- 172.21.15.76:0/3543823550 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b8c06b4d0 msgr2=0x7f1b8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:42.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.591+0000 7f1ba63ac640 1 --2- 172.21.15.76:0/3543823550 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b8c06b4d0 0x7f1b8c06d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00ce180 tx=0x7f1b90007ce0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.591+0000 7f1ba63ac640 1 -- 172.21.15.76:0/3543823550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ba00ce5e0 msgr2=0x7f1ba01e9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:42.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.591+0000 7f1ba63ac640 1 --2- 172.21.15.76:0/3543823550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ba00ce5e0 0x7f1ba01e9870 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400e9e0 tx=0x7f1b9400eea0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.591+0000 7f1ba63ac640 1 -- 172.21.15.76:0/3543823550 shutdown_connections 2024-03-08T06:01:42.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.591+0000 7f1ba63ac640 1 --2- 172.21.15.76:0/3543823550 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1b8c06b4d0 0x7f1b8c06d990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.591+0000 7f1ba63ac640 1 --2- 172.21.15.76:0/3543823550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ba0162560 0x7f1ba01e9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.591+0000 7f1ba63ac640 1 --2- 172.21.15.76:0/3543823550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ba00ce5e0 0x7f1ba01e9870 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.591+0000 7f1ba63ac640 1 --2- 172.21.15.76:0/3543823550 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba006d8a0 0x7f1ba014fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.591+0000 7f1ba63ac640 1 -- 172.21.15.76:0/3543823550 >> 172.21.15.76:0/3543823550 conn(0x7f1ba006ae60 msgr2=0x7f1ba00b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:42.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.591+0000 7f1ba63ac640 1 -- 172.21.15.76:0/3543823550 shutdown_connections 2024-03-08T06:01:42.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.591+0000 7f1ba63ac640 1 -- 172.21.15.76:0/3543823550 wait complete. 2024-03-08T06:01:42.602 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-08T06:01:42.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.703+0000 7f4e75f6c640 1 Processor -- start 2024-03-08T06:01:42.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.703+0000 7f4e75f6c640 1 -- start start 2024-03-08T06:01:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.703+0000 7f4e75f6c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e700b9840 0x7f4e700b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.703+0000 7f4e75f6c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4e700b5b20 0x7f4e700b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.703+0000 7f4e75f6c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e700d27d0 con 0x7f4e7015dbc0 2024-03-08T06:01:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.703+0000 7f4e75f6c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e700b5f90 con 0x7f4e700b5b20 2024-03-08T06:01:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.703+0000 7f4e75f6c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700d2ab0 con 0x7f4e700b9840 2024-03-08T06:01:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4e7015dbc0 0x7f4e700d1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48444/0 (socket says 172.21.15.76:48444) 2024-03-08T06:01:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6f7fe640 1 -- 172.21.15.76:0/1833788492 learned_addr learned my addr 172.21.15.76:0/1833788492 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6effd640 1 --2- 172.21.15.76:0/1833788492 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e700b9840 0x7f4e700b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6ffff640 1 --2- 172.21.15.76:0/1833788492 >> v2:172.21.15.81:3300/0 conn(0x7f4e700b5b20 0x7f4e700b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6e7fc640 1 -- 172.21.15.76:0/1833788492 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722843475 0 0) 0x7f4e700d27d0 con 0x7f4e7015dbc0 2024-03-08T06:01:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6e7fc640 1 -- 172.21.15.76:0/1833788492 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e58003520 con 0x7f4e7015dbc0 2024-03-08T06:01:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6e7fc640 1 -- 172.21.15.76:0/1833788492 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1395342618 0 0) 0x7f4e58003520 con 0x7f4e7015dbc0 2024-03-08T06:01:42.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6e7fc640 1 -- 172.21.15.76:0/1833788492 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e700b9840 msgr2=0x7f4e700b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:42.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6e7fc640 1 --2- 172.21.15.76:0/1833788492 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e700b9840 0x7f4e700b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6e7fc640 1 -- 172.21.15.76:0/1833788492 >> v2:172.21.15.81:3300/0 conn(0x7f4e700b5b20 msgr2=0x7f4e700b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:42.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6e7fc640 1 --2- 172.21.15.76:0/1833788492 >> v2:172.21.15.81:3300/0 conn(0x7f4e700b5b20 0x7f4e700b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6e7fc640 1 -- 172.21.15.76:0/1833788492 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e700d2d90 con 0x7f4e7015dbc0 2024-03-08T06:01:42.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6e7fc640 1 -- 172.21.15.76:0/1833788492 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f4e60001670 con 0x7f4e7015dbc0 2024-03-08T06:01:42.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6e7fc640 1 -- 172.21.15.76:0/1833788492 >> v1:172.21.15.76:6789/0 conn(0x7f4e7015dbc0 legacy=0x7f4e700d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:42.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6e7fc640 1 --2- 172.21.15.76:0/1833788492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e58003a20 0x7f4e58003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:42.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6e7fc640 1 --2- 172.21.15.76:0/1833788492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e58004800 0x7f4e58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:42.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6e7fc640 1 --2- 172.21.15.76:0/1833788492 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e700b9840 0x7f4e580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:42.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6e7fc640 1 -- 172.21.15.76:0/1833788492 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700b5f90 con 0x7f4e58003a20 2024-03-08T06:01:42.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6e7fc640 1 -- 172.21.15.76:0/1833788492 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700d2ab0 con 0x7f4e58004800 2024-03-08T06:01:42.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6e7fc640 1 -- 172.21.15.76:0/1833788492 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5800b9c0 con 0x7f4e700b9840 2024-03-08T06:01:42.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6f7fe640 1 --2- 172.21.15.76:0/1833788492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e58003a20 0x7f4e58003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:42.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6ffff640 1 --2- 172.21.15.76:0/1833788492 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e700b9840 0x7f4e580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:42.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6effd640 1 --2- 172.21.15.76:0/1833788492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e58004800 0x7f4e58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:42.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6f7fe640 1 -- 172.21.15.76:0/1833788492 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e700b9840 msgr2=0x7f4e580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:42.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6f7fe640 1 --2- 172.21.15.76:0/1833788492 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e700b9840 0x7f4e580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6f7fe640 1 -- 172.21.15.76:0/1833788492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e58004800 msgr2=0x7f4e58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:42.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6f7fe640 1 --2- 172.21.15.76:0/1833788492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e58004800 0x7f4e58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.704+0000 7f4e6f7fe640 1 -- 172.21.15.76:0/1833788492 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e58003520 con 0x7f4e58003a20 2024-03-08T06:01:42.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.705+0000 7f4e6f7fe640 1 --2- 172.21.15.76:0/1833788492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e58003a20 0x7f4e58003e40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000c270 tx=0x7f4e58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8624f1fcae847da1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:42.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.705+0000 7f4e6e7fc640 1 -- 172.21.15.76:0/1833788492 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60007d30 con 0x7f4e58003a20 2024-03-08T06:01:42.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.705+0000 7f4e6e7fc640 1 -- 172.21.15.76:0/1833788492 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e60030070 con 0x7f4e58003a20 2024-03-08T06:01:42.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.705+0000 7f4e6e7fc640 1 -- 172.21.15.76:0/1833788492 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6002cd60 con 0x7f4e58003a20 2024-03-08T06:01:42.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.706+0000 7f4e75f6c640 1 -- 172.21.15.76:0/1833788492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e58003a20 msgr2=0x7f4e58003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:42.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.706+0000 7f4e75f6c640 1 --2- 172.21.15.76:0/1833788492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e58003a20 0x7f4e58003e40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000c270 tx=0x7f4e58005830 comp rx=0 tx=0).stop 2024-03-08T06:01:42.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.706+0000 7f4e6ffff640 1 -- 172.21.15.76:0/1833788492 reap_dead start 2024-03-08T06:01:42.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.706+0000 7f4e75f6c640 1 -- 172.21.15.76:0/1833788492 shutdown_connections 2024-03-08T06:01:42.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.706+0000 7f4e75f6c640 1 -- 172.21.15.76:0/1833788492 >> 172.21.15.76:0/1833788492 conn(0x7f4e700670d0 msgr2=0x7f4e700d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:42.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.706+0000 7f4e75f6c640 1 -- 172.21.15.76:0/1833788492 shutdown_connections 2024-03-08T06:01:42.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.706+0000 7f4e75f6c640 1 -- 172.21.15.76:0/1833788492 wait complete. 2024-03-08T06:01:42.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.707+0000 7f4e75f6c640 1 Processor -- start 2024-03-08T06:01:42.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.707+0000 7f4e75f6c640 1 -- start start 2024-03-08T06:01:42.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.708+0000 7f4e75f6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e701f32a0 0x7f4e701e3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:42.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.708+0000 7f4e75f6c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e701f3680 0x7f4e70147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:42.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.708+0000 7f4e75f6c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70148c90 0x7f4e701494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:42.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.708+0000 7f4e75f6c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7015dfa0 con 0x7f4e701f32a0 2024-03-08T06:01:42.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.708+0000 7f4e75f6c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700d3870 con 0x7f4e70148c90 2024-03-08T06:01:42.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.708+0000 7f4e75f6c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7015e120 con 0x7f4e701f3680 2024-03-08T06:01:42.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.708+0000 7f4e6f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e701f32a0 0x7f4e701e3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:42.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.708+0000 7f4e6f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e701f32a0 0x7f4e701e3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45486/0 (socket says 172.21.15.76:45486) 2024-03-08T06:01:42.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.708+0000 7f4e6f7fe640 1 -- 172.21.15.76:0/3898828085 learned_addr learned my addr 172.21.15.76:0/3898828085 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:42.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.708+0000 7f4e6f7fe640 1 -- 172.21.15.76:0/3898828085 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e701f3680 msgr2=0x7f4e70147430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:42.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.708+0000 7f4e6f7fe640 1 --2- 172.21.15.76:0/3898828085 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e701f3680 0x7f4e70147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.708+0000 7f4e6f7fe640 1 -- 172.21.15.76:0/3898828085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70148c90 msgr2=0x7f4e701494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:42.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.708+0000 7f4e6f7fe640 1 --2- 172.21.15.76:0/3898828085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70148c90 0x7f4e701494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.708+0000 7f4e6f7fe640 1 -- 172.21.15.76:0/3898828085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e70147970 con 0x7f4e701f32a0 2024-03-08T06:01:42.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.709+0000 7f4e6f7fe640 1 --2- 172.21.15.76:0/3898828085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e701f32a0 0x7f4e701e3370 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000c190 tx=0x7f4e700d29a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:42.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.709+0000 7f4e57fff640 1 -- 172.21.15.76:0/3898828085 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60001c70 con 0x7f4e701f32a0 2024-03-08T06:01:42.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.709+0000 7f4e57fff640 1 -- 172.21.15.76:0/3898828085 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e60004cb0 con 0x7f4e701f32a0 2024-03-08T06:01:42.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.709+0000 7f4e75f6c640 1 -- 172.21.15.76:0/3898828085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e70147bf0 con 0x7f4e701f32a0 2024-03-08T06:01:42.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.709+0000 7f4e57fff640 1 -- 172.21.15.76:0/3898828085 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60007d30 con 0x7f4e701f32a0 2024-03-08T06:01:42.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.709+0000 7f4e75f6c640 1 -- 172.21.15.76:0/3898828085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e701481b0 con 0x7f4e701f32a0 2024-03-08T06:01:42.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.710+0000 7f4e75f6c640 1 -- 172.21.15.76:0/3898828085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e7004f030 con 0x7f4e701f32a0 2024-03-08T06:01:42.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.710+0000 7f4e57fff640 1 -- 172.21.15.76:0/3898828085 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e60007290 con 0x7f4e701f32a0 2024-03-08T06:01:42.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.711+0000 7f4e57fff640 1 --2- 172.21.15.76:0/3898828085 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4e5c06b4d0 0x7f4e5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:42.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.711+0000 7f4e57fff640 1 -- 172.21.15.76:0/3898828085 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4e600ab570 con 0x7f4e701f32a0 2024-03-08T06:01:42.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.711+0000 7f4e6effd640 1 --2- 172.21.15.76:0/3898828085 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4e5c06b4d0 0x7f4e5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:42.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.711+0000 7f4e6effd640 1 --2- 172.21.15.76:0/3898828085 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4e5c06b4d0 0x7f4e5c06d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009a40 tx=0x7f4e64009320 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:42.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.715+0000 7f4e57fff640 1 -- 172.21.15.76:0/3898828085 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e70147970 con 0x7f4e701f32a0 2024-03-08T06:01:42.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.952+0000 7f4e75f6c640 1 -- 172.21.15.76:0/3898828085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f4e700c9b40 con 0x7f4e701f32a0 2024-03-08T06:01:42.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.954+0000 7f4e57fff640 1 -- 172.21.15.76:0/3898828085 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4e6003f340 con 0x7f4e701f32a0 2024-03-08T06:01:42.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.956+0000 7f4e75f6c640 1 -- 172.21.15.76:0/3898828085 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4e5c06b4d0 msgr2=0x7f4e5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:42.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.956+0000 7f4e75f6c640 1 --2- 172.21.15.76:0/3898828085 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4e5c06b4d0 0x7f4e5c06d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009a40 tx=0x7f4e64009320 comp rx=0 tx=0).stop 2024-03-08T06:01:42.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.956+0000 7f4e75f6c640 1 -- 172.21.15.76:0/3898828085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e701f32a0 msgr2=0x7f4e701e3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:42.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.956+0000 7f4e75f6c640 1 --2- 172.21.15.76:0/3898828085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e701f32a0 0x7f4e701e3370 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000c190 tx=0x7f4e700d29a0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.956+0000 7f4e75f6c640 1 -- 172.21.15.76:0/3898828085 shutdown_connections 2024-03-08T06:01:42.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.956+0000 7f4e75f6c640 1 --2- 172.21.15.76:0/3898828085 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4e5c06b4d0 0x7f4e5c06d990 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.956+0000 7f4e75f6c640 1 --2- 172.21.15.76:0/3898828085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70148c90 0x7f4e701494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.956+0000 7f4e75f6c640 1 --2- 172.21.15.76:0/3898828085 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e701f3680 0x7f4e70147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.956+0000 7f4e75f6c640 1 --2- 172.21.15.76:0/3898828085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e701f32a0 0x7f4e701e3370 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:42.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.956+0000 7f4e75f6c640 1 -- 172.21.15.76:0/3898828085 >> 172.21.15.76:0/3898828085 conn(0x7f4e700670d0 msgr2=0x7f4e7006c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:42.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.956+0000 7f4e75f6c640 1 -- 172.21.15.76:0/3898828085 shutdown_connections 2024-03-08T06:01:42.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:42.956+0000 7f4e75f6c640 1 -- 172.21.15.76:0/3898828085 wait complete. 2024-03-08T06:01:42.968 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-08T06:01:43.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.069+0000 7f6fe5e78640 1 Processor -- start 2024-03-08T06:01:43.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.069+0000 7f6fe5e78640 1 -- start start 2024-03-08T06:01:43.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.069+0000 7f6fe5e78640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6fe015dbc0 0x7f6fe00d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fe5e78640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fe00b9840 0x7f6fe00b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fe5e78640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe00d27d0 con 0x7f6fe00b5b20 2024-03-08T06:01:43.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fe5e78640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe00b5f90 con 0x7f6fe015dbc0 2024-03-08T06:01:43.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fe5e78640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe00d2ab0 con 0x7f6fe00b9840 2024-03-08T06:01:43.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fdffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6fe00b5b20 0x7f6fe006c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48450/0 (socket says 172.21.15.76:48450) 2024-03-08T06:01:43.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fdffff640 1 -- 172.21.15.76:0/2188550902 learned_addr learned my addr 172.21.15.76:0/2188550902 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:43.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fdeffd640 1 --2- 172.21.15.76:0/2188550902 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fe00b9840 0x7f6fe00b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fdf7fe640 1 --2- 172.21.15.76:0/2188550902 >> v2:172.21.15.81:3300/0 conn(0x7f6fe015dbc0 0x7f6fe00d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fde7fc640 1 -- 172.21.15.76:0/2188550902 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486469955 0 0) 0x7f6fe00d27d0 con 0x7f6fe00b5b20 2024-03-08T06:01:43.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fde7fc640 1 -- 172.21.15.76:0/2188550902 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc8003520 con 0x7f6fe00b5b20 2024-03-08T06:01:43.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fde7fc640 1 -- 172.21.15.76:0/2188550902 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1923729269 0 0) 0x7f6fc8003520 con 0x7f6fe00b5b20 2024-03-08T06:01:43.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fde7fc640 1 -- 172.21.15.76:0/2188550902 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fe00b9840 msgr2=0x7f6fe00b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:43.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fde7fc640 1 --2- 172.21.15.76:0/2188550902 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fe00b9840 0x7f6fe00b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fde7fc640 1 -- 172.21.15.76:0/2188550902 >> v2:172.21.15.81:3300/0 conn(0x7f6fe015dbc0 msgr2=0x7f6fe00d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:43.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fde7fc640 1 --2- 172.21.15.76:0/2188550902 >> v2:172.21.15.81:3300/0 conn(0x7f6fe015dbc0 0x7f6fe00d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fde7fc640 1 -- 172.21.15.76:0/2188550902 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe00d2d90 con 0x7f6fe00b5b20 2024-03-08T06:01:43.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fde7fc640 1 -- 172.21.15.76:0/2188550902 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f6fd80019f0 con 0x7f6fe00b5b20 2024-03-08T06:01:43.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fde7fc640 1 -- 172.21.15.76:0/2188550902 >> v1:172.21.15.76:6789/0 conn(0x7f6fe00b5b20 legacy=0x7f6fe006c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:43.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fde7fc640 1 --2- 172.21.15.76:0/2188550902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fc8003a20 0x7f6fc8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fde7fc640 1 --2- 172.21.15.76:0/2188550902 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fe00b9840 0x7f6fc8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fde7fc640 1 --2- 172.21.15.76:0/2188550902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fc8004d40 0x7f6fc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fde7fc640 1 -- 172.21.15.76:0/2188550902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe00b5f90 con 0x7f6fc8004d40 2024-03-08T06:01:43.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fde7fc640 1 -- 172.21.15.76:0/2188550902 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe00d2ab0 con 0x7f6fc8003a20 2024-03-08T06:01:43.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fde7fc640 1 -- 172.21.15.76:0/2188550902 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc800b9c0 con 0x7f6fe00b9840 2024-03-08T06:01:43.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fdffff640 1 --2- 172.21.15.76:0/2188550902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fc8004d40 0x7f6fc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fdf7fe640 1 --2- 172.21.15.76:0/2188550902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fc8003a20 0x7f6fc8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.070+0000 7f6fdeffd640 1 --2- 172.21.15.76:0/2188550902 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fe00b9840 0x7f6fc8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.071+0000 7f6fdffff640 1 -- 172.21.15.76:0/2188550902 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fe00b9840 msgr2=0x7f6fc8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:43.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.071+0000 7f6fdffff640 1 --2- 172.21.15.76:0/2188550902 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fe00b9840 0x7f6fc8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.071+0000 7f6fdffff640 1 -- 172.21.15.76:0/2188550902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fc8003a20 msgr2=0x7f6fc8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:43.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.071+0000 7f6fdffff640 1 --2- 172.21.15.76:0/2188550902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fc8003a20 0x7f6fc8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.071+0000 7f6fdffff640 1 -- 172.21.15.76:0/2188550902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc8003520 con 0x7f6fc8004d40 2024-03-08T06:01:43.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.071+0000 7f6fdffff640 1 --2- 172.21.15.76:0/2188550902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fc8004d40 0x7f6fc80051c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800a880 tx=0x7f6fc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a131d2ad97386130 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:43.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.072+0000 7f6fde7fc640 1 -- 172.21.15.76:0/2188550902 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd800a4a0 con 0x7f6fc8004d40 2024-03-08T06:01:43.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.072+0000 7f6fde7fc640 1 -- 172.21.15.76:0/2188550902 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd8034650 con 0x7f6fc8004d40 2024-03-08T06:01:43.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.072+0000 7f6fde7fc640 1 -- 172.21.15.76:0/2188550902 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8034a40 con 0x7f6fc8004d40 2024-03-08T06:01:43.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.072+0000 7f6fe5e78640 1 -- 172.21.15.76:0/2188550902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fc8004d40 msgr2=0x7f6fc80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:43.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.072+0000 7f6fe5e78640 1 --2- 172.21.15.76:0/2188550902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fc8004d40 0x7f6fc80051c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800a880 tx=0x7f6fc8005830 comp rx=0 tx=0).stop 2024-03-08T06:01:43.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.072+0000 7f6fdffff640 1 -- 172.21.15.76:0/2188550902 reap_dead start 2024-03-08T06:01:43.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.072+0000 7f6fe5e78640 1 -- 172.21.15.76:0/2188550902 shutdown_connections 2024-03-08T06:01:43.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.072+0000 7f6fe5e78640 1 -- 172.21.15.76:0/2188550902 >> 172.21.15.76:0/2188550902 conn(0x7f6fe00670d0 msgr2=0x7f6fe00d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:43.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.072+0000 7f6fe5e78640 1 -- 172.21.15.76:0/2188550902 shutdown_connections 2024-03-08T06:01:43.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.072+0000 7f6fe5e78640 1 -- 172.21.15.76:0/2188550902 wait complete. 2024-03-08T06:01:43.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.073+0000 7f6fe5e78640 1 Processor -- start 2024-03-08T06:01:43.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.073+0000 7f6fe5e78640 1 -- start start 2024-03-08T06:01:43.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.074+0000 7f6fe5e78640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fe01f3280 0x7f6fe01e2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.074+0000 7f6fe5e78640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fe0159dd0 0x7f6fe01f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.074+0000 7f6fe5e78640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fe01f74f0 0x7f6fe00cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.074+0000 7f6fe5e78640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe00d2430 con 0x7f6fe01f74f0 2024-03-08T06:01:43.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.074+0000 7f6fe5e78640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe00d3870 con 0x7f6fe0159dd0 2024-03-08T06:01:43.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.074+0000 7f6fe5e78640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe00d25b0 con 0x7f6fe01f3280 2024-03-08T06:01:43.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.074+0000 7f6fdffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fe01f74f0 0x7f6fe00cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.074+0000 7f6fdffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fe01f74f0 0x7f6fe00cbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45494/0 (socket says 172.21.15.76:45494) 2024-03-08T06:01:43.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.074+0000 7f6fdffff640 1 -- 172.21.15.76:0/2532150848 learned_addr learned my addr 172.21.15.76:0/2532150848 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:43.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.074+0000 7f6fdf7fe640 1 --2- 172.21.15.76:0/2532150848 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fe01f3280 0x7f6fe01e2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.074+0000 7f6fdeffd640 1 --2- 172.21.15.76:0/2532150848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fe0159dd0 0x7f6fe01f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.075+0000 7f6fdffff640 1 -- 172.21.15.76:0/2532150848 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fe01f3280 msgr2=0x7f6fe01e2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:43.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.075+0000 7f6fdffff640 1 --2- 172.21.15.76:0/2532150848 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fe01f3280 0x7f6fe01e2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.075+0000 7f6fdffff640 1 -- 172.21.15.76:0/2532150848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fe0159dd0 msgr2=0x7f6fe01f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:43.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.075+0000 7f6fdffff640 1 --2- 172.21.15.76:0/2532150848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fe0159dd0 0x7f6fe01f6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.075+0000 7f6fdffff640 1 -- 172.21.15.76:0/2532150848 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe015a2d0 con 0x7f6fe01f74f0 2024-03-08T06:01:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.075+0000 7f6fdffff640 1 --2- 172.21.15.76:0/2532150848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fe01f74f0 0x7f6fe00cbc80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800a9a0 tx=0x7f6fe00d2f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.075+0000 7f6fc7fff640 1 -- 172.21.15.76:0/2532150848 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8045070 con 0x7f6fe01f74f0 2024-03-08T06:01:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.075+0000 7f6fc7fff640 1 -- 172.21.15.76:0/2532150848 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd803a040 con 0x7f6fe01f74f0 2024-03-08T06:01:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.075+0000 7f6fe5e78640 1 -- 172.21.15.76:0/2532150848 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe01574f0 con 0x7f6fe01f74f0 2024-03-08T06:01:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.075+0000 7f6fc7fff640 1 -- 172.21.15.76:0/2532150848 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd800a4a0 con 0x7f6fe01f74f0 2024-03-08T06:01:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.075+0000 7f6fe5e78640 1 -- 172.21.15.76:0/2532150848 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe0157ab0 con 0x7f6fe01f74f0 2024-03-08T06:01:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.076+0000 7f6fe5e78640 1 -- 172.21.15.76:0/2532150848 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe004f030 con 0x7f6fe01f74f0 2024-03-08T06:01:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.076+0000 7f6fc7fff640 1 -- 172.21.15.76:0/2532150848 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fd8048070 con 0x7f6fe01f74f0 2024-03-08T06:01:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.077+0000 7f6fc7fff640 1 --2- 172.21.15.76:0/2532150848 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6fcc06b500 0x7f6fcc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.077+0000 7f6fc7fff640 1 -- 172.21.15.76:0/2532150848 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6fd80ab9c0 con 0x7f6fe01f74f0 2024-03-08T06:01:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.077+0000 7f6fdf7fe640 1 --2- 172.21.15.76:0/2532150848 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6fcc06b500 0x7f6fcc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.077+0000 7f6fdf7fe640 1 --2- 172.21.15.76:0/2532150848 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6fcc06b500 0x7f6fcc06d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009ba0 tx=0x7f6fd4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:43.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.081+0000 7f6fc7fff640 1 -- 172.21.15.76:0/2532150848 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6fd80ade20 con 0x7f6fe01f74f0 2024-03-08T06:01:43.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.320+0000 7f6fe5e78640 1 -- 172.21.15.76:0/2532150848 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f6fe00c9f70 con 0x7f6fe01f74f0 2024-03-08T06:01:43.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.322+0000 7f6fc7fff640 1 -- 172.21.15.76:0/2532150848 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6fd807a3e0 con 0x7f6fe01f74f0 2024-03-08T06:01:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.324+0000 7f6fe5e78640 1 -- 172.21.15.76:0/2532150848 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6fcc06b500 msgr2=0x7f6fcc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.324+0000 7f6fe5e78640 1 --2- 172.21.15.76:0/2532150848 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6fcc06b500 0x7f6fcc06d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009ba0 tx=0x7f6fd4009190 comp rx=0 tx=0).stop 2024-03-08T06:01:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.324+0000 7f6fe5e78640 1 -- 172.21.15.76:0/2532150848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fe01f74f0 msgr2=0x7f6fe00cbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.324+0000 7f6fe5e78640 1 --2- 172.21.15.76:0/2532150848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fe01f74f0 0x7f6fe00cbc80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800a9a0 tx=0x7f6fe00d2f60 comp rx=0 tx=0).stop 2024-03-08T06:01:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.324+0000 7f6fe5e78640 1 -- 172.21.15.76:0/2532150848 shutdown_connections 2024-03-08T06:01:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.324+0000 7f6fe5e78640 1 --2- 172.21.15.76:0/2532150848 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6fcc06b500 0x7f6fcc06d9c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.324+0000 7f6fe5e78640 1 --2- 172.21.15.76:0/2532150848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fe01f74f0 0x7f6fe00cbc80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.324+0000 7f6fe5e78640 1 --2- 172.21.15.76:0/2532150848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fe0159dd0 0x7f6fe01f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.324+0000 7f6fe5e78640 1 --2- 172.21.15.76:0/2532150848 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fe01f3280 0x7f6fe01e2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.324+0000 7f6fe5e78640 1 -- 172.21.15.76:0/2532150848 >> 172.21.15.76:0/2532150848 conn(0x7f6fe00670d0 msgr2=0x7f6fe00c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.324+0000 7f6fe5e78640 1 -- 172.21.15.76:0/2532150848 shutdown_connections 2024-03-08T06:01:43.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.324+0000 7f6fe5e78640 1 -- 172.21.15.76:0/2532150848 wait complete. 2024-03-08T06:01:43.336 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-08T06:01:43.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.437+0000 7f302de9d640 1 Processor -- start 2024-03-08T06:01:43.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.437+0000 7f302de9d640 1 -- start start 2024-03-08T06:01:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.438+0000 7f302de9d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30280b3c50 0x7f30280b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.438+0000 7f302de9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f302806ac80 0x7f302806b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.438+0000 7f302de9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f302806f280 0x7f302806f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.438+0000 7f302de9d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30280b4570 con 0x7f302806ac80 2024-03-08T06:01:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.438+0000 7f302de9d640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f302806b720 con 0x7f302806f280 2024-03-08T06:01:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.438+0000 7f302de9d640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302806ba00 con 0x7f30280b3c50 2024-03-08T06:01:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.438+0000 7f3026ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f302806ac80 0x7f302806b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.438+0000 7f3026ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f302806ac80 0x7f302806b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45504/0 (socket says 172.21.15.76:45504) 2024-03-08T06:01:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.438+0000 7f3026ffd640 1 -- 172.21.15.76:0/1390263574 learned_addr learned my addr 172.21.15.76:0/1390263574 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.438+0000 7f30277fe640 1 --2- 172.21.15.76:0/1390263574 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30280b3c50 0x7f30280b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.438+0000 7f3027fff640 1 --2- 172.21.15.76:0/1390263574 >> v2:172.21.15.81:3300/0 conn(0x7f302806f280 0x7f302806f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.438+0000 7f3026ffd640 1 -- 172.21.15.76:0/1390263574 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30280b3c50 msgr2=0x7f30280b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.438+0000 7f3026ffd640 1 --2- 172.21.15.76:0/1390263574 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30280b3c50 0x7f30280b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.438+0000 7f3026ffd640 1 -- 172.21.15.76:0/1390263574 >> v2:172.21.15.81:3300/0 conn(0x7f302806f280 msgr2=0x7f302806f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.438+0000 7f3026ffd640 1 --2- 172.21.15.76:0/1390263574 >> v2:172.21.15.81:3300/0 conn(0x7f302806f280 0x7f302806f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.438+0000 7f3026ffd640 1 -- 172.21.15.76:0/1390263574 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302806bce0 con 0x7f302806ac80 2024-03-08T06:01:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.439+0000 7f3026ffd640 1 --2- 172.21.15.76:0/1390263574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f302806ac80 0x7f302806b0c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f30280b4810 tx=0x7f301c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ad609a528f709f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.439+0000 7f30267fc640 1 -- 172.21.15.76:0/1390263574 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c035020 con 0x7f302806ac80 2024-03-08T06:01:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.439+0000 7f30267fc640 1 -- 172.21.15.76:0/1390263574 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301c031870 con 0x7f302806ac80 2024-03-08T06:01:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.439+0000 7f30267fc640 1 -- 172.21.15.76:0/1390263574 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c031c40 con 0x7f302806ac80 2024-03-08T06:01:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.440+0000 7f302de9d640 1 -- 172.21.15.76:0/1390263574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f302806ac80 msgr2=0x7f302806b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.440+0000 7f302de9d640 1 --2- 172.21.15.76:0/1390263574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f302806ac80 0x7f302806b0c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f30280b4810 tx=0x7f301c02b910 comp rx=0 tx=0).stop 2024-03-08T06:01:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.440+0000 7f302de9d640 1 -- 172.21.15.76:0/1390263574 shutdown_connections 2024-03-08T06:01:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.440+0000 7f302de9d640 1 --2- 172.21.15.76:0/1390263574 >> v2:172.21.15.81:3300/0 conn(0x7f302806f280 0x7f302806f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.440+0000 7f302de9d640 1 --2- 172.21.15.76:0/1390263574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f302806ac80 0x7f302806b0c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.440+0000 7f302de9d640 1 --2- 172.21.15.76:0/1390263574 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30280b3c50 0x7f30280b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.440+0000 7f302de9d640 1 -- 172.21.15.76:0/1390263574 >> 172.21.15.76:0/1390263574 conn(0x7f302806dd40 msgr2=0x7f302806e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:43.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.440+0000 7f302de9d640 1 -- 172.21.15.76:0/1390263574 shutdown_connections 2024-03-08T06:01:43.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.440+0000 7f302de9d640 1 -- 172.21.15.76:0/1390263574 wait complete. 2024-03-08T06:01:43.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.441+0000 7f302de9d640 1 Processor -- start 2024-03-08T06:01:43.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.441+0000 7f302de9d640 1 -- start start 2024-03-08T06:01:43.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f302de9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f302806ac80 0x7f30280cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f302de9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f302806f280 0x7f302814ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f302de9d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30280b3c50 0x7f30280c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f302de9d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3028070500 con 0x7f302806f280 2024-03-08T06:01:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f302de9d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302806c2a0 con 0x7f302806ac80 2024-03-08T06:01:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f302de9d640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30281612c0 con 0x7f30280b3c50 2024-03-08T06:01:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f3026ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f302806f280 0x7f302814ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f3026ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f302806f280 0x7f302814ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45518/0 (socket says 172.21.15.76:45518) 2024-03-08T06:01:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f3026ffd640 1 -- 172.21.15.76:0/503920969 learned_addr learned my addr 172.21.15.76:0/503920969 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f3027fff640 1 --2- 172.21.15.76:0/503920969 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30280b3c50 0x7f30280c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f30277fe640 1 --2- 172.21.15.76:0/503920969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f302806ac80 0x7f30280cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f3026ffd640 1 -- 172.21.15.76:0/503920969 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30280b3c50 msgr2=0x7f30280c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f3026ffd640 1 --2- 172.21.15.76:0/503920969 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30280b3c50 0x7f30280c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f3026ffd640 1 -- 172.21.15.76:0/503920969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f302806ac80 msgr2=0x7f30280cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f3026ffd640 1 --2- 172.21.15.76:0/503920969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f302806ac80 0x7f30280cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f3026ffd640 1 -- 172.21.15.76:0/503920969 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302814d5f0 con 0x7f302806f280 2024-03-08T06:01:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.442+0000 7f30277fe640 1 --2- 172.21.15.76:0/503920969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f302806ac80 0x7f30280cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.443+0000 7f3026ffd640 1 --2- 172.21.15.76:0/503920969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f302806f280 0x7f302814ccc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f301c000c00 tx=0x7f301c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.443+0000 7f300ffff640 1 -- 172.21.15.76:0/503920969 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c031e10 con 0x7f302806f280 2024-03-08T06:01:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.443+0000 7f300ffff640 1 -- 172.21.15.76:0/503920969 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301c03ad60 con 0x7f302806f280 2024-03-08T06:01:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.443+0000 7f302de9d640 1 -- 172.21.15.76:0/503920969 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302814d8d0 con 0x7f302806f280 2024-03-08T06:01:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.443+0000 7f300ffff640 1 -- 172.21.15.76:0/503920969 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c044610 con 0x7f302806f280 2024-03-08T06:01:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.443+0000 7f302de9d640 1 -- 172.21.15.76:0/503920969 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302814de90 con 0x7f302806f280 2024-03-08T06:01:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.444+0000 7f302de9d640 1 -- 172.21.15.76:0/503920969 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f302804f030 con 0x7f302806f280 2024-03-08T06:01:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.444+0000 7f300ffff640 1 -- 172.21.15.76:0/503920969 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f301c04b060 con 0x7f302806f280 2024-03-08T06:01:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.445+0000 7f300ffff640 1 --2- 172.21.15.76:0/503920969 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f301406b500 0x7f301406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.445+0000 7f300ffff640 1 -- 172.21.15.76:0/503920969 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f301c0abfa0 con 0x7f302806f280 2024-03-08T06:01:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.445+0000 7f30277fe640 1 --2- 172.21.15.76:0/503920969 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f301406b500 0x7f301406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.445+0000 7f30277fe640 1 --2- 172.21.15.76:0/503920969 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f301406b500 0x7f301406d9c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f30280cdfa0 tx=0x7f3018008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:43.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.449+0000 7f300ffff640 1 -- 172.21.15.76:0/503920969 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f301c03f070 con 0x7f302806f280 2024-03-08T06:01:43.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.684+0000 7f302de9d640 1 -- 172.21.15.76:0/503920969 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f302806cdf0 con 0x7f302806f280 2024-03-08T06:01:43.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.686+0000 7f300ffff640 1 -- 172.21.15.76:0/503920969 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f301c02f320 con 0x7f302806f280 2024-03-08T06:01:43.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.688+0000 7f302de9d640 1 -- 172.21.15.76:0/503920969 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f301406b500 msgr2=0x7f301406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:43.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.688+0000 7f302de9d640 1 --2- 172.21.15.76:0/503920969 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f301406b500 0x7f301406d9c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f30280cdfa0 tx=0x7f3018008040 comp rx=0 tx=0).stop 2024-03-08T06:01:43.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.688+0000 7f302de9d640 1 -- 172.21.15.76:0/503920969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f302806f280 msgr2=0x7f302814ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:43.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.688+0000 7f302de9d640 1 --2- 172.21.15.76:0/503920969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f302806f280 0x7f302814ccc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f301c000c00 tx=0x7f301c0046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.688+0000 7f302de9d640 1 -- 172.21.15.76:0/503920969 shutdown_connections 2024-03-08T06:01:43.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.688+0000 7f302de9d640 1 --2- 172.21.15.76:0/503920969 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f301406b500 0x7f301406d9c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.688+0000 7f302de9d640 1 --2- 172.21.15.76:0/503920969 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30280b3c50 0x7f30280c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.688+0000 7f302de9d640 1 --2- 172.21.15.76:0/503920969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f302806f280 0x7f302814ccc0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.688+0000 7f302de9d640 1 --2- 172.21.15.76:0/503920969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f302806ac80 0x7f30280cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.688+0000 7f302de9d640 1 -- 172.21.15.76:0/503920969 >> 172.21.15.76:0/503920969 conn(0x7f302806dd40 msgr2=0x7f3028160160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:43.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.688+0000 7f302de9d640 1 -- 172.21.15.76:0/503920969 shutdown_connections 2024-03-08T06:01:43.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.688+0000 7f302de9d640 1 -- 172.21.15.76:0/503920969 wait complete. 2024-03-08T06:01:43.700 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-08T06:01:43.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.801+0000 7fc250488640 1 Processor -- start 2024-03-08T06:01:43.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.801+0000 7fc250488640 1 -- start start 2024-03-08T06:01:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.802+0000 7fc250488640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc24806d040 0x7fc2480cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.802+0000 7fc250488640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc248071670 0x7fc2480cc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.802+0000 7fc250488640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc248070470 con 0x7fc248161210 2024-03-08T06:01:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.802+0000 7fc250488640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24806bfa0 con 0x7fc248071670 2024-03-08T06:01:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.802+0000 7fc250488640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24806c280 con 0x7fc24806d040 2024-03-08T06:01:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.802+0000 7fc24ea25640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc248071670 0x7fc2480cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.802+0000 7fc24ea25640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc248071670 0x7fc2480cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45528/0 (socket says 172.21.15.76:45528) 2024-03-08T06:01:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.802+0000 7fc24ea25640 1 -- 172.21.15.76:0/3566920911 learned_addr learned my addr 172.21.15.76:0/3566920911 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.802+0000 7fc24da23640 1 --2- 172.21.15.76:0/3566920911 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc24806d040 0x7fc2480cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.802+0000 7fc24ea25640 1 -- 172.21.15.76:0/3566920911 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc24806d040 msgr2=0x7fc2480cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.802+0000 7fc24ea25640 1 --2- 172.21.15.76:0/3566920911 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc24806d040 0x7fc2480cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.802+0000 7fc24ea25640 1 -- 172.21.15.76:0/3566920911 >> v1:172.21.15.81:6789/0 conn(0x7fc248161210 legacy=0x7fc248070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.802+0000 7fc24ea25640 1 -- 172.21.15.76:0/3566920911 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24806c560 con 0x7fc248071670 2024-03-08T06:01:43.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.803+0000 7fc24ea25640 1 --2- 172.21.15.76:0/3566920911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc248071670 0x7fc2480cc060 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc240009e50 tx=0x7fc24002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b372caa59cee8d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:43.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.803+0000 7fc24d222640 1 -- 172.21.15.76:0/3566920911 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24002cb00 con 0x7fc248071670 2024-03-08T06:01:43.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.803+0000 7fc24d222640 1 -- 172.21.15.76:0/3566920911 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24002cca0 con 0x7fc248071670 2024-03-08T06:01:43.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.803+0000 7fc24d222640 1 -- 172.21.15.76:0/3566920911 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc240034610 con 0x7fc248071670 2024-03-08T06:01:43.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.803+0000 7fc250488640 1 -- 172.21.15.76:0/3566920911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc248071670 msgr2=0x7fc2480cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:43.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.803+0000 7fc250488640 1 --2- 172.21.15.76:0/3566920911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc248071670 0x7fc2480cc060 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc240009e50 tx=0x7fc24002bf60 comp rx=0 tx=0).stop 2024-03-08T06:01:43.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.803+0000 7fc250488640 1 -- 172.21.15.76:0/3566920911 shutdown_connections 2024-03-08T06:01:43.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.803+0000 7fc250488640 1 --2- 172.21.15.76:0/3566920911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc248071670 0x7fc2480cc060 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.804+0000 7fc250488640 1 --2- 172.21.15.76:0/3566920911 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc24806d040 0x7fc2480cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.804+0000 7fc250488640 1 -- 172.21.15.76:0/3566920911 >> 172.21.15.76:0/3566920911 conn(0x7fc24806f530 msgr2=0x7fc24806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:43.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.804+0000 7fc250488640 1 -- 172.21.15.76:0/3566920911 shutdown_connections 2024-03-08T06:01:43.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.804+0000 7fc250488640 1 -- 172.21.15.76:0/3566920911 wait complete. 2024-03-08T06:01:43.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.805+0000 7fc250488640 1 Processor -- start 2024-03-08T06:01:43.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.805+0000 7fc250488640 1 -- start start 2024-03-08T06:01:43.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.805+0000 7fc250488640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc24806d040 0x7fc24814dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.805+0000 7fc250488640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc248071670 0x7fc2480b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.805+0000 7fc250488640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc248161210 0x7fc2480b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.805+0000 7fc250488640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24806d4d0 con 0x7fc248071670 2024-03-08T06:01:43.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.805+0000 7fc250488640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24806cb20 con 0x7fc248161210 2024-03-08T06:01:43.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.805+0000 7fc250488640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24806d650 con 0x7fc24806d040 2024-03-08T06:01:43.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.805+0000 7fc24da23640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc248071670 0x7fc2480b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.805+0000 7fc24da23640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc248071670 0x7fc2480b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45538/0 (socket says 172.21.15.76:45538) 2024-03-08T06:01:43.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.805+0000 7fc24da23640 1 -- 172.21.15.76:0/285814124 learned_addr learned my addr 172.21.15.76:0/285814124 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:43.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.806+0000 7fc24e224640 1 --2- 172.21.15.76:0/285814124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc24806d040 0x7fc24814dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.806+0000 7fc24ea25640 1 --2- 172.21.15.76:0/285814124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc248161210 0x7fc2480b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.806+0000 7fc24da23640 1 -- 172.21.15.76:0/285814124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc24806d040 msgr2=0x7fc24814dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:43.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.806+0000 7fc24da23640 1 --2- 172.21.15.76:0/285814124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc24806d040 0x7fc24814dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.806+0000 7fc24da23640 1 -- 172.21.15.76:0/285814124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc248161210 msgr2=0x7fc2480b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:43.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.806+0000 7fc24da23640 1 --2- 172.21.15.76:0/285814124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc248161210 0x7fc2480b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:43.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.806+0000 7fc24da23640 1 -- 172.21.15.76:0/285814124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24814e270 con 0x7fc248071670 2024-03-08T06:01:43.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.806+0000 7fc24e224640 1 --2- 172.21.15.76:0/285814124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc24806d040 0x7fc24814dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:43.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.806+0000 7fc24da23640 1 --2- 172.21.15.76:0/285814124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc248071670 0x7fc2480b4d50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc23800e9b0 tx=0x7fc23800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:43.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.806+0000 7fc23effd640 1 -- 172.21.15.76:0/285814124 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238017070 con 0x7fc248071670 2024-03-08T06:01:43.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.806+0000 7fc23effd640 1 -- 172.21.15.76:0/285814124 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23800c9a0 con 0x7fc248071670 2024-03-08T06:01:43.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.806+0000 7fc250488640 1 -- 172.21.15.76:0/285814124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2480b7310 con 0x7fc248071670 2024-03-08T06:01:43.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.806+0000 7fc23effd640 1 -- 172.21.15.76:0/285814124 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23800cd70 con 0x7fc248071670 2024-03-08T06:01:43.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.806+0000 7fc250488640 1 -- 172.21.15.76:0/285814124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2480b5ba0 con 0x7fc248071670 2024-03-08T06:01:43.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.807+0000 7fc250488640 1 -- 172.21.15.76:0/285814124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24804f030 con 0x7fc248071670 2024-03-08T06:01:43.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.807+0000 7fc23effd640 1 -- 172.21.15.76:0/285814124 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc23801a070 con 0x7fc248071670 2024-03-08T06:01:43.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.808+0000 7fc23effd640 1 --2- 172.21.15.76:0/285814124 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc22c06b4d0 0x7fc22c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:43.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.808+0000 7fc23effd640 1 -- 172.21.15.76:0/285814124 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc238089ff0 con 0x7fc248071670 2024-03-08T06:01:43.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.808+0000 7fc24e224640 1 --2- 172.21.15.76:0/285814124 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc22c06b4d0 0x7fc22c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:43.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.808+0000 7fc24e224640 1 --2- 172.21.15.76:0/285814124 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc22c06b4d0 0x7fc22c06d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc2480b7790 tx=0x7fc244009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:43.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:43.812+0000 7fc23effd640 1 -- 172.21.15.76:0/285814124 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc24814e270 con 0x7fc248071670 2024-03-08T06:01:44.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.050+0000 7fc250488640 1 -- 172.21.15.76:0/285814124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fc2480b7850 con 0x7fc248071670 2024-03-08T06:01:44.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.052+0000 7fc23effd640 1 -- 172.21.15.76:0/285814124 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc238059030 con 0x7fc248071670 2024-03-08T06:01:44.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.054+0000 7fc250488640 1 -- 172.21.15.76:0/285814124 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc22c06b4d0 msgr2=0x7fc22c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.054+0000 7fc250488640 1 --2- 172.21.15.76:0/285814124 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc22c06b4d0 0x7fc22c06d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc2480b7790 tx=0x7fc244009040 comp rx=0 tx=0).stop 2024-03-08T06:01:44.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.054+0000 7fc250488640 1 -- 172.21.15.76:0/285814124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc248071670 msgr2=0x7fc2480b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.054+0000 7fc250488640 1 --2- 172.21.15.76:0/285814124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc248071670 0x7fc2480b4d50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc23800e9b0 tx=0x7fc23800ee70 comp rx=0 tx=0).stop 2024-03-08T06:01:44.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.054+0000 7fc250488640 1 -- 172.21.15.76:0/285814124 shutdown_connections 2024-03-08T06:01:44.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.054+0000 7fc250488640 1 --2- 172.21.15.76:0/285814124 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc22c06b4d0 0x7fc22c06d990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.054+0000 7fc250488640 1 --2- 172.21.15.76:0/285814124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc248161210 0x7fc2480b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.054+0000 7fc250488640 1 --2- 172.21.15.76:0/285814124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc248071670 0x7fc2480b4d50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.054+0000 7fc250488640 1 --2- 172.21.15.76:0/285814124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc24806d040 0x7fc24814dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.054+0000 7fc250488640 1 -- 172.21.15.76:0/285814124 >> 172.21.15.76:0/285814124 conn(0x7fc24806f530 msgr2=0x7fc24815f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:44.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.054+0000 7fc250488640 1 -- 172.21.15.76:0/285814124 shutdown_connections 2024-03-08T06:01:44.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.054+0000 7fc250488640 1 -- 172.21.15.76:0/285814124 wait complete. 2024-03-08T06:01:44.065 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-08T06:01:44.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.166+0000 7f6a8e64e640 1 Processor -- start 2024-03-08T06:01:44.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.166+0000 7f6a8e64e640 1 -- start start 2024-03-08T06:01:44.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.167+0000 7f6a8e64e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a88154df0 0x7f6a880b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.167+0000 7f6a8e64e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8815f450 con 0x7f6a880b3640 2024-03-08T06:01:44.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.167+0000 7f6a8e64e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8815e690 con 0x7f6a880cc7e0 2024-03-08T06:01:44.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.167+0000 7f6a8e64e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a880af1e0 con 0x7f6a88154df0 2024-03-08T06:01:44.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.167+0000 7f6a87fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a88154df0 0x7f6a880b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.167+0000 7f6a87fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a88154df0 0x7f6a880b3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45548/0 (socket says 172.21.15.76:45548) 2024-03-08T06:01:44.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.167+0000 7f6a877fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6a880b3640 0x7f6a880cc0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48460/0 (socket says 172.21.15.76:48460) 2024-03-08T06:01:44.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.167+0000 7f6a87fff640 1 -- 172.21.15.76:0/538208167 learned_addr learned my addr 172.21.15.76:0/538208167 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:44.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.167+0000 7f6a87fff640 1 -- 172.21.15.76:0/538208167 >> v1:172.21.15.81:6789/0 conn(0x7f6a880cc7e0 legacy=0x7f6a8815af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:44.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.167+0000 7f6a87fff640 1 -- 172.21.15.76:0/538208167 >> v1:172.21.15.76:6789/0 conn(0x7f6a880b3640 legacy=0x7f6a880cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.167+0000 7f6a87fff640 1 -- 172.21.15.76:0/538208167 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8815e970 con 0x7f6a88154df0 2024-03-08T06:01:44.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.167+0000 7f6a86ffd640 1 -- 172.21.15.76:0/538208167 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 144340715 0 0) 0x7f6a8815f450 con 0x7f6a880b3640 2024-03-08T06:01:44.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.168+0000 7f6a87fff640 1 --2- 172.21.15.76:0/538208167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a88154df0 0x7f6a880b3100 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009870 tx=0x7f6a7802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d4b8e3b41ce6b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:44.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.168+0000 7f6a86ffd640 1 -- 172.21.15.76:0/538208167 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78034070 con 0x7f6a88154df0 2024-03-08T06:01:44.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.168+0000 7f6a86ffd640 1 -- 172.21.15.76:0/538208167 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a78030860 con 0x7f6a88154df0 2024-03-08T06:01:44.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.168+0000 7f6a86ffd640 1 -- 172.21.15.76:0/538208167 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78030c30 con 0x7f6a88154df0 2024-03-08T06:01:44.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.169+0000 7f6a8e64e640 1 -- 172.21.15.76:0/538208167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a88154df0 msgr2=0x7f6a880b3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.169+0000 7f6a8e64e640 1 --2- 172.21.15.76:0/538208167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a88154df0 0x7f6a880b3100 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009870 tx=0x7f6a7802b9a0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.169+0000 7f6a8e64e640 1 -- 172.21.15.76:0/538208167 shutdown_connections 2024-03-08T06:01:44.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.169+0000 7f6a8e64e640 1 --2- 172.21.15.76:0/538208167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a88154df0 0x7f6a880b3100 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.169+0000 7f6a8e64e640 1 -- 172.21.15.76:0/538208167 >> 172.21.15.76:0/538208167 conn(0x7f6a8806ac80 msgr2=0x7f6a8806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:44.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.169+0000 7f6a8e64e640 1 -- 172.21.15.76:0/538208167 shutdown_connections 2024-03-08T06:01:44.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.169+0000 7f6a8e64e640 1 -- 172.21.15.76:0/538208167 wait complete. 2024-03-08T06:01:44.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.170+0000 7f6a8e64e640 1 Processor -- start 2024-03-08T06:01:44.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.170+0000 7f6a8e64e640 1 -- start start 2024-03-08T06:01:44.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a8e64e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6a880b3640 0x7f6a88148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a8e64e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6a880cc7e0 0x7f6a880ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a8e64e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a88154df0 0x7f6a881f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a8e64e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8815ef30 con 0x7f6a88154df0 2024-03-08T06:01:44.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a8e64e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a880b3c80 con 0x7f6a880b3640 2024-03-08T06:01:44.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a8e64e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a880ceb60 con 0x7f6a880cc7e0 2024-03-08T06:01:44.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a8cbeb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a88154df0 0x7f6a881f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a8cbeb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a88154df0 0x7f6a881f4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45560/0 (socket says 172.21.15.76:45560) 2024-03-08T06:01:44.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a8cbeb640 1 -- 172.21.15.76:0/4129230112 learned_addr learned my addr 172.21.15.76:0/4129230112 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:44.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a87fff640 1 --2- 172.21.15.76:0/4129230112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6a880b3640 0x7f6a88148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a877fe640 1 --2- 172.21.15.76:0/4129230112 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6a880cc7e0 0x7f6a880ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a8cbeb640 1 -- 172.21.15.76:0/4129230112 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6a880cc7e0 msgr2=0x7f6a880ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a8cbeb640 1 --2- 172.21.15.76:0/4129230112 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6a880cc7e0 0x7f6a880ae4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a8cbeb640 1 -- 172.21.15.76:0/4129230112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6a880b3640 msgr2=0x7f6a88148100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a8cbeb640 1 --2- 172.21.15.76:0/4129230112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6a880b3640 0x7f6a88148100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a8cbeb640 1 -- 172.21.15.76:0/4129230112 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a881f39b0 con 0x7f6a88154df0 2024-03-08T06:01:44.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a8cbeb640 1 --2- 172.21.15.76:0/4129230112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a88154df0 0x7f6a881f4190 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000c7a0 tx=0x7f6a8000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:44.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a84ff9640 1 -- 172.21.15.76:0/4129230112 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a80003b70 con 0x7f6a88154df0 2024-03-08T06:01:44.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a84ff9640 1 -- 172.21.15.76:0/4129230112 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a80003d30 con 0x7f6a88154df0 2024-03-08T06:01:44.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.171+0000 7f6a8e64e640 1 -- 172.21.15.76:0/4129230112 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8814b750 con 0x7f6a88154df0 2024-03-08T06:01:44.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.172+0000 7f6a84ff9640 1 -- 172.21.15.76:0/4129230112 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8000aa90 con 0x7f6a88154df0 2024-03-08T06:01:44.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.172+0000 7f6a8e64e640 1 -- 172.21.15.76:0/4129230112 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8814bd10 con 0x7f6a88154df0 2024-03-08T06:01:44.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.173+0000 7f6a8e64e640 1 -- 172.21.15.76:0/4129230112 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8804f030 con 0x7f6a88154df0 2024-03-08T06:01:44.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.173+0000 7f6a84ff9640 1 -- 172.21.15.76:0/4129230112 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a8000ac30 con 0x7f6a88154df0 2024-03-08T06:01:44.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.173+0000 7f6a84ff9640 1 --2- 172.21.15.76:0/4129230112 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6a7406b4d0 0x7f6a7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.174+0000 7f6a84ff9640 1 -- 172.21.15.76:0/4129230112 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6a80089ec0 con 0x7f6a88154df0 2024-03-08T06:01:44.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.174+0000 7f6a87fff640 1 --2- 172.21.15.76:0/4129230112 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6a7406b4d0 0x7f6a7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.174+0000 7f6a87fff640 1 --2- 172.21.15.76:0/4129230112 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6a7406b4d0 0x7f6a7406d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a78036000 tx=0x7f6a7803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:44.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.178+0000 7f6a84ff9640 1 -- 172.21.15.76:0/4129230112 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a80058f90 con 0x7f6a88154df0 2024-03-08T06:01:44.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.412+0000 7f6a8e64e640 1 -- 172.21.15.76:0/4129230112 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f6a88154880 con 0x7f6a88154df0 2024-03-08T06:01:44.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.414+0000 7f6a84ff9640 1 -- 172.21.15.76:0/4129230112 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6a80058890 con 0x7f6a88154df0 2024-03-08T06:01:44.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.416+0000 7f6a8e64e640 1 -- 172.21.15.76:0/4129230112 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6a7406b4d0 msgr2=0x7f6a7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.416+0000 7f6a8e64e640 1 --2- 172.21.15.76:0/4129230112 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6a7406b4d0 0x7f6a7406d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a78036000 tx=0x7f6a7803b040 comp rx=0 tx=0).stop 2024-03-08T06:01:44.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.416+0000 7f6a8e64e640 1 -- 172.21.15.76:0/4129230112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a88154df0 msgr2=0x7f6a881f4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.416+0000 7f6a8e64e640 1 --2- 172.21.15.76:0/4129230112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a88154df0 0x7f6a881f4190 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000c7a0 tx=0x7f6a8000cc60 comp rx=0 tx=0).stop 2024-03-08T06:01:44.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.416+0000 7f6a8e64e640 1 -- 172.21.15.76:0/4129230112 shutdown_connections 2024-03-08T06:01:44.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.416+0000 7f6a8e64e640 1 --2- 172.21.15.76:0/4129230112 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6a7406b4d0 0x7f6a7406d990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.416+0000 7f6a8e64e640 1 --2- 172.21.15.76:0/4129230112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a88154df0 0x7f6a881f4190 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.416+0000 7f6a8e64e640 1 --2- 172.21.15.76:0/4129230112 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6a880cc7e0 0x7f6a880ae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.416+0000 7f6a8e64e640 1 --2- 172.21.15.76:0/4129230112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6a880b3640 0x7f6a88148100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.416+0000 7f6a8e64e640 1 -- 172.21.15.76:0/4129230112 >> 172.21.15.76:0/4129230112 conn(0x7f6a8806ac80 msgr2=0x7f6a88152050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:44.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.417+0000 7f6a8e64e640 1 -- 172.21.15.76:0/4129230112 shutdown_connections 2024-03-08T06:01:44.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.417+0000 7f6a8e64e640 1 -- 172.21.15.76:0/4129230112 wait complete. 2024-03-08T06:01:44.428 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:44.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.527+0000 7efef8676640 1 Processor -- start 2024-03-08T06:01:44.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.528+0000 7efef8676640 1 -- start start 2024-03-08T06:01:44.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.528+0000 7efef8676640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef00d1960 0x7efef00d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.528+0000 7efef8676640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef00d2280 con 0x7efef00ce320 2024-03-08T06:01:44.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.528+0000 7efef8676640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef00ced70 con 0x7efef00ce780 2024-03-08T06:01:44.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.528+0000 7efef8676640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef00cc7b0 con 0x7efef00d1960 2024-03-08T06:01:44.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5c11640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efef00ce320 0x7efef006bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48464/0 (socket says 172.21.15.76:48464) 2024-03-08T06:01:44.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5c11640 1 -- 172.21.15.76:0/3015971765 learned_addr learned my addr 172.21.15.76:0/3015971765 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:44.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef6412640 1 --2- 172.21.15.76:0/3015971765 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef00d1960 0x7efef00d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5410640 1 -- 172.21.15.76:0/3015971765 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854462978 0 0) 0x7efef00d2280 con 0x7efef00ce320 2024-03-08T06:01:44.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5410640 1 -- 172.21.15.76:0/3015971765 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efedc003520 con 0x7efef00ce320 2024-03-08T06:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5410640 1 -- 172.21.15.76:0/3015971765 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1567202114 0 0) 0x7efedc003520 con 0x7efef00ce320 2024-03-08T06:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5410640 1 -- 172.21.15.76:0/3015971765 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef00d1960 msgr2=0x7efef00d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5410640 1 --2- 172.21.15.76:0/3015971765 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef00d1960 0x7efef00d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5410640 1 -- 172.21.15.76:0/3015971765 >> v1:172.21.15.81:6789/0 conn(0x7efef00ce780 legacy=0x7efef00ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5410640 1 -- 172.21.15.76:0/3015971765 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef00cf050 con 0x7efef00ce320 2024-03-08T06:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5410640 1 -- 172.21.15.76:0/3015971765 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7efee0001670 con 0x7efef00ce320 2024-03-08T06:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5410640 1 -- 172.21.15.76:0/3015971765 >> v1:172.21.15.76:6789/0 conn(0x7efef00ce320 legacy=0x7efef006bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5410640 1 --2- 172.21.15.76:0/3015971765 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef00d1960 0x7efedc003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5410640 1 --2- 172.21.15.76:0/3015971765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efedc003fd0 0x7efedc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5410640 1 --2- 172.21.15.76:0/3015971765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efedc004d40 0x7efedc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5410640 1 -- 172.21.15.76:0/3015971765 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef00cc7b0 con 0x7efedc004d40 2024-03-08T06:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5410640 1 -- 172.21.15.76:0/3015971765 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc00bb20 con 0x7efedc003fd0 2024-03-08T06:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5410640 1 -- 172.21.15.76:0/3015971765 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc00bd20 con 0x7efef00d1960 2024-03-08T06:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef6c13640 1 --2- 172.21.15.76:0/3015971765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efedc004d40 0x7efedc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef5c11640 1 --2- 172.21.15.76:0/3015971765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efedc003fd0 0x7efedc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.529+0000 7efef6412640 1 --2- 172.21.15.76:0/3015971765 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef00d1960 0x7efedc003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.530+0000 7efef6c13640 1 -- 172.21.15.76:0/3015971765 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef00d1960 msgr2=0x7efedc003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:44.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.530+0000 7efef6c13640 1 --2- 172.21.15.76:0/3015971765 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef00d1960 0x7efedc003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.530+0000 7efef6c13640 1 -- 172.21.15.76:0/3015971765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efedc003fd0 msgr2=0x7efedc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:44.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.530+0000 7efef6c13640 1 --2- 172.21.15.76:0/3015971765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efedc003fd0 0x7efedc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.530+0000 7efef6c13640 1 -- 172.21.15.76:0/3015971765 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc003520 con 0x7efedc004d40 2024-03-08T06:01:44.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.530+0000 7efef6c13640 1 --2- 172.21.15.76:0/3015971765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efedc004d40 0x7efedc0051c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efeec001750 tx=0x7efedc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fac2522bedc224 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:44.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.530+0000 7efef5410640 1 -- 172.21.15.76:0/3015971765 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec001370 con 0x7efedc004d40 2024-03-08T06:01:44.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.531+0000 7efef5410640 1 -- 172.21.15.76:0/3015971765 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec0052d0 con 0x7efedc004d40 2024-03-08T06:01:44.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.531+0000 7efef5410640 1 -- 172.21.15.76:0/3015971765 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec003830 con 0x7efedc004d40 2024-03-08T06:01:44.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.531+0000 7efef8676640 1 -- 172.21.15.76:0/3015971765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efedc004d40 msgr2=0x7efedc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.531+0000 7efef8676640 1 --2- 172.21.15.76:0/3015971765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efedc004d40 0x7efedc0051c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efeec001750 tx=0x7efedc005830 comp rx=0 tx=0).stop 2024-03-08T06:01:44.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.531+0000 7efef6c13640 1 -- 172.21.15.76:0/3015971765 reap_dead start 2024-03-08T06:01:44.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.531+0000 7efef8676640 1 -- 172.21.15.76:0/3015971765 shutdown_connections 2024-03-08T06:01:44.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.531+0000 7efef8676640 1 -- 172.21.15.76:0/3015971765 >> 172.21.15.76:0/3015971765 conn(0x7efef00b34e0 msgr2=0x7efef00b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:44.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.531+0000 7efef8676640 1 -- 172.21.15.76:0/3015971765 shutdown_connections 2024-03-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.531+0000 7efef8676640 1 -- 172.21.15.76:0/3015971765 wait complete. 2024-03-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.532+0000 7efef8676640 1 Processor -- start 2024-03-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.532+0000 7efef8676640 1 -- start start 2024-03-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.533+0000 7efef8676640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efedc004d40 0x7efef01e75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.533+0000 7efef8676640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef01f7f10 0x7efef006d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.533+0000 7efef8676640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efef01f3220 0x7efef01f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.533+0000 7efef8676640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef006c4a0 con 0x7efef01f7f10 2024-03-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.533+0000 7efef8676640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef00cc440 con 0x7efef01f3220 2024-03-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.533+0000 7efef8676640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef00cde00 con 0x7efedc004d40 2024-03-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.533+0000 7efef5c11640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef01f7f10 0x7efef006d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.533+0000 7efef5c11640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef01f7f10 0x7efef006d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45574/0 (socket says 172.21.15.76:45574) 2024-03-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.533+0000 7efef5c11640 1 -- 172.21.15.76:0/2481526053 learned_addr learned my addr 172.21.15.76:0/2481526053 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.533+0000 7efef6412640 1 --2- 172.21.15.76:0/2481526053 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efedc004d40 0x7efef01e75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.533+0000 7efef5c11640 1 -- 172.21.15.76:0/2481526053 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efedc004d40 msgr2=0x7efef01e75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.533+0000 7efef6c13640 1 --2- 172.21.15.76:0/2481526053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efef01f3220 0x7efef01f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.533+0000 7efef5c11640 1 --2- 172.21.15.76:0/2481526053 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efedc004d40 0x7efef01e75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.533+0000 7efef5c11640 1 -- 172.21.15.76:0/2481526053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efef01f3220 msgr2=0x7efef01f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.533+0000 7efef5c11640 1 --2- 172.21.15.76:0/2481526053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efef01f3220 0x7efef01f36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.533+0000 7efef5c11640 1 -- 172.21.15.76:0/2481526053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef01f3be0 con 0x7efef01f7f10 2024-03-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.534+0000 7efef5c11640 1 --2- 172.21.15.76:0/2481526053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef01f7f10 0x7efef006d230 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efee0001b80 tx=0x7efee0007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.534+0000 7efee6ffd640 1 -- 172.21.15.76:0/2481526053 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee000f960 con 0x7efef01f7f10 2024-03-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.534+0000 7efee6ffd640 1 -- 172.21.15.76:0/2481526053 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee0018070 con 0x7efef01f7f10 2024-03-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.534+0000 7efee6ffd640 1 -- 172.21.15.76:0/2481526053 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee000fd60 con 0x7efef01f7f10 2024-03-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.534+0000 7efef8676640 1 -- 172.21.15.76:0/2481526053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef01e7e10 con 0x7efef01f7f10 2024-03-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.534+0000 7efef8676640 1 -- 172.21.15.76:0/2481526053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef01e83d0 con 0x7efef01f7f10 2024-03-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.535+0000 7efef8676640 1 -- 172.21.15.76:0/2481526053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef004f030 con 0x7efef01f7f10 2024-03-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.535+0000 7efee6ffd640 1 -- 172.21.15.76:0/2481526053 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efee001d460 con 0x7efef01f7f10 2024-03-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.535+0000 7efee6ffd640 1 --2- 172.21.15.76:0/2481526053 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7efed406b500 0x7efed406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.536+0000 7efee6ffd640 1 -- 172.21.15.76:0/2481526053 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efee008a110 con 0x7efef01f7f10 2024-03-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.536+0000 7efef6412640 1 --2- 172.21.15.76:0/2481526053 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7efed406b500 0x7efed406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.536+0000 7efef6412640 1 --2- 172.21.15.76:0/2481526053 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7efed406b500 0x7efed406d9c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efee8007440 tx=0x7efee800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:44.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.540+0000 7efee6ffd640 1 -- 172.21.15.76:0/2481526053 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efef01f3be0 con 0x7efef01f7f10 2024-03-08T06:01:44.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.709+0000 7efef8676640 1 -- 172.21.15.76:0/2481526053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efef00d0550 con 0x7efef01f7f10 2024-03-08T06:01:44.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.710+0000 7efee6ffd640 1 -- 172.21.15.76:0/2481526053 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+13080 (secure 0 0 0) 0x7efee00591e0 con 0x7efef01f7f10 2024-03-08T06:01:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:44.711 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":36,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:01:41.855869+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:44.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.712+0000 7efef8676640 1 -- 172.21.15.76:0/2481526053 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7efed406b500 msgr2=0x7efed406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.712+0000 7efef8676640 1 --2- 172.21.15.76:0/2481526053 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7efed406b500 0x7efed406d9c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efee8007440 tx=0x7efee800b040 comp rx=0 tx=0).stop 2024-03-08T06:01:44.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.712+0000 7efef8676640 1 -- 172.21.15.76:0/2481526053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef01f7f10 msgr2=0x7efef006d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.712+0000 7efef8676640 1 --2- 172.21.15.76:0/2481526053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef01f7f10 0x7efef006d230 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efee0001b80 tx=0x7efee0007720 comp rx=0 tx=0).stop 2024-03-08T06:01:44.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.712+0000 7efef8676640 1 -- 172.21.15.76:0/2481526053 shutdown_connections 2024-03-08T06:01:44.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.712+0000 7efef8676640 1 --2- 172.21.15.76:0/2481526053 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7efed406b500 0x7efed406d9c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.712+0000 7efef8676640 1 --2- 172.21.15.76:0/2481526053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efef01f3220 0x7efef01f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.712+0000 7efef8676640 1 --2- 172.21.15.76:0/2481526053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef01f7f10 0x7efef006d230 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.712+0000 7efef8676640 1 --2- 172.21.15.76:0/2481526053 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efedc004d40 0x7efef01e75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.712+0000 7efef8676640 1 -- 172.21.15.76:0/2481526053 >> 172.21.15.76:0/2481526053 conn(0x7efef00b34e0 msgr2=0x7efef006c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:44.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.712+0000 7efef8676640 1 -- 172.21.15.76:0/2481526053 shutdown_connections 2024-03-08T06:01:44.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.712+0000 7efef8676640 1 -- 172.21.15.76:0/2481526053 wait complete. 2024-03-08T06:01:44.723 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:01:44.724 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:01:44.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.827+0000 7f9f83b2e640 1 Processor -- start 2024-03-08T06:01:44.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.827+0000 7f9f83b2e640 1 -- start start 2024-03-08T06:01:44.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.828+0000 7f9f83b2e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9f7c0cc0d0 0x7f9f7c0cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.828+0000 7f9f83b2e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9f7c15af70 0x7f9f7c15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.828+0000 7f9f83b2e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f7c15f450 con 0x7f9f7c154df0 2024-03-08T06:01:44.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.828+0000 7f9f83b2e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f7c0af1e0 con 0x7f9f7c0cc0d0 2024-03-08T06:01:44.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.828+0000 7f9f83b2e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f7c0b3c80 con 0x7f9f7c15af70 2024-03-08T06:01:44.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.828+0000 7f9f810c9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9f7c0cc0d0 0x7f9f7c0cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.828+0000 7f9f810c9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9f7c0cc0d0 0x7f9f7c0cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45586/0 (socket says 172.21.15.76:45586) 2024-03-08T06:01:44.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.828+0000 7f9f810c9640 1 -- 172.21.15.76:0/3667494841 learned_addr learned my addr 172.21.15.76:0/3667494841 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:44.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.828+0000 7f9f820cb640 1 --2- 172.21.15.76:0/3667494841 >> v2:172.21.15.81:3300/0 conn(0x7f9f7c15af70 0x7f9f7c15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.828+0000 7f9f810c9640 1 -- 172.21.15.76:0/3667494841 >> v2:172.21.15.81:3300/0 conn(0x7f9f7c15af70 msgr2=0x7f9f7c15b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:44.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.828+0000 7f9f810c9640 1 --2- 172.21.15.76:0/3667494841 >> v2:172.21.15.81:3300/0 conn(0x7f9f7c15af70 0x7f9f7c15b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.828+0000 7f9f810c9640 1 -- 172.21.15.76:0/3667494841 >> v1:172.21.15.81:6789/0 conn(0x7f9f7c154df0 legacy=0x7f9f7c0b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.828+0000 7f9f810c9640 1 -- 172.21.15.76:0/3667494841 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c15e690 con 0x7f9f7c0cc0d0 2024-03-08T06:01:44.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.829+0000 7f9f810c9640 1 --2- 172.21.15.76:0/3667494841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c0cc0d0 0x7f9f7c0cc580 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0cd910 tx=0x7f9f6c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e8d261d6f55188 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:44.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.829+0000 7f9f808c8640 1 -- 172.21.15.76:0/3667494841 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c035020 con 0x7f9f7c0cc0d0 2024-03-08T06:01:44.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.829+0000 7f9f808c8640 1 -- 172.21.15.76:0/3667494841 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6c031870 con 0x7f9f7c0cc0d0 2024-03-08T06:01:44.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.829+0000 7f9f808c8640 1 -- 172.21.15.76:0/3667494841 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c031c40 con 0x7f9f7c0cc0d0 2024-03-08T06:01:44.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.829+0000 7f9f83b2e640 1 -- 172.21.15.76:0/3667494841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c0cc0d0 msgr2=0x7f9f7c0cc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.829+0000 7f9f83b2e640 1 --2- 172.21.15.76:0/3667494841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c0cc0d0 0x7f9f7c0cc580 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0cd910 tx=0x7f9f6c02b910 comp rx=0 tx=0).stop 2024-03-08T06:01:44.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.830+0000 7f9f83b2e640 1 -- 172.21.15.76:0/3667494841 shutdown_connections 2024-03-08T06:01:44.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.830+0000 7f9f83b2e640 1 --2- 172.21.15.76:0/3667494841 >> v2:172.21.15.81:3300/0 conn(0x7f9f7c15af70 0x7f9f7c15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.830+0000 7f9f83b2e640 1 --2- 172.21.15.76:0/3667494841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c0cc0d0 0x7f9f7c0cc580 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.830+0000 7f9f83b2e640 1 -- 172.21.15.76:0/3667494841 >> 172.21.15.76:0/3667494841 conn(0x7f9f7c06ac80 msgr2=0x7f9f7c06b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:44.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.830+0000 7f9f83b2e640 1 -- 172.21.15.76:0/3667494841 shutdown_connections 2024-03-08T06:01:44.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.830+0000 7f9f83b2e640 1 -- 172.21.15.76:0/3667494841 wait complete. 2024-03-08T06:01:44.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.831+0000 7f9f83b2e640 1 Processor -- start 2024-03-08T06:01:44.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.831+0000 7f9f83b2e640 1 -- start start 2024-03-08T06:01:44.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.831+0000 7f9f83b2e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c0cc0d0 0x7f9f7c13e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.831+0000 7f9f83b2e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9f7c154df0 0x7f9f7c14d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.831+0000 7f9f83b2e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f7c15af70 0x7f9f7c0b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f83b2e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c15ef30 con 0x7f9f7c0cc0d0 2024-03-08T06:01:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f83b2e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c15ec50 con 0x7f9f7c15af70 2024-03-08T06:01:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f83b2e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c1551d0 con 0x7f9f7c154df0 2024-03-08T06:01:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f818ca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c0cc0d0 0x7f9f7c13e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f818ca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c0cc0d0 0x7f9f7c13e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45596/0 (socket says 172.21.15.76:45596) 2024-03-08T06:01:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f818ca640 1 -- 172.21.15.76:0/915631066 learned_addr learned my addr 172.21.15.76:0/915631066 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f820cb640 1 --2- 172.21.15.76:0/915631066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f7c15af70 0x7f9f7c0b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f810c9640 1 --2- 172.21.15.76:0/915631066 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9f7c154df0 0x7f9f7c14d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f818ca640 1 -- 172.21.15.76:0/915631066 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9f7c154df0 msgr2=0x7f9f7c14d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f818ca640 1 --2- 172.21.15.76:0/915631066 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9f7c154df0 0x7f9f7c14d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f818ca640 1 -- 172.21.15.76:0/915631066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f7c15af70 msgr2=0x7f9f7c0b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f818ca640 1 --2- 172.21.15.76:0/915631066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f7c15af70 0x7f9f7c0b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f818ca640 1 -- 172.21.15.76:0/915631066 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c14e780 con 0x7f9f7c0cc0d0 2024-03-08T06:01:44.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f818ca640 1 --2- 172.21.15.76:0/915631066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c0cc0d0 0x7f9f7c13e8f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800c400 tx=0x7f9f7800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:44.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f727fc640 1 -- 172.21.15.76:0/915631066 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f780052d0 con 0x7f9f7c0cc0d0 2024-03-08T06:01:44.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f727fc640 1 -- 172.21.15.76:0/915631066 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f78005490 con 0x7f9f7c0cc0d0 2024-03-08T06:01:44.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f83b2e640 1 -- 172.21.15.76:0/915631066 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f7c14ea60 con 0x7f9f7c0cc0d0 2024-03-08T06:01:44.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f727fc640 1 -- 172.21.15.76:0/915631066 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f78007260 con 0x7f9f7c0cc0d0 2024-03-08T06:01:44.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.832+0000 7f9f83b2e640 1 -- 172.21.15.76:0/915631066 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f7c14f020 con 0x7f9f7c0cc0d0 2024-03-08T06:01:44.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.833+0000 7f9f83b2e640 1 -- 172.21.15.76:0/915631066 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f7c04f030 con 0x7f9f7c0cc0d0 2024-03-08T06:01:44.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.834+0000 7f9f727fc640 1 -- 172.21.15.76:0/915631066 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f78007400 con 0x7f9f7c0cc0d0 2024-03-08T06:01:44.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.834+0000 7f9f727fc640 1 --2- 172.21.15.76:0/915631066 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9f6006b4d0 0x7f9f6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:44.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.834+0000 7f9f810c9640 1 --2- 172.21.15.76:0/915631066 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9f6006b4d0 0x7f9f6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:44.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.834+0000 7f9f727fc640 1 -- 172.21.15.76:0/915631066 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9f78089500 con 0x7f9f7c0cc0d0 2024-03-08T06:01:44.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.835+0000 7f9f810c9640 1 --2- 172.21.15.76:0/915631066 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9f6006b4d0 0x7f9f6006d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c036000 tx=0x7f9f6c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:44.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:44.838+0000 7f9f727fc640 1 -- 172.21.15.76:0/915631066 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f780585d0 con 0x7f9f7c0cc0d0 2024-03-08T06:01:45.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.013+0000 7f9f83b2e640 1 -- 172.21.15.76:0/915631066 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f7c0b2de0 con 0x7f9f7c0cc0d0 2024-03-08T06:01:45.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.014+0000 7f9f727fc640 1 -- 172.21.15.76:0/915631066 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f78057ed0 con 0x7f9f7c0cc0d0 2024-03-08T06:01:45.014 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:01:45.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.015+0000 7f9f83b2e640 1 -- 172.21.15.76:0/915631066 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9f6006b4d0 msgr2=0x7f9f6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:45.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.015+0000 7f9f83b2e640 1 --2- 172.21.15.76:0/915631066 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9f6006b4d0 0x7f9f6006d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c036000 tx=0x7f9f6c008030 comp rx=0 tx=0).stop 2024-03-08T06:01:45.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.015+0000 7f9f83b2e640 1 -- 172.21.15.76:0/915631066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c0cc0d0 msgr2=0x7f9f7c13e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:45.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.015+0000 7f9f83b2e640 1 --2- 172.21.15.76:0/915631066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c0cc0d0 0x7f9f7c13e8f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800c400 tx=0x7f9f7800c8c0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.015+0000 7f9f83b2e640 1 -- 172.21.15.76:0/915631066 shutdown_connections 2024-03-08T06:01:45.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.015+0000 7f9f83b2e640 1 --2- 172.21.15.76:0/915631066 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9f6006b4d0 0x7f9f6006d990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.015+0000 7f9f83b2e640 1 --2- 172.21.15.76:0/915631066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f7c15af70 0x7f9f7c0b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.015+0000 7f9f83b2e640 1 --2- 172.21.15.76:0/915631066 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9f7c154df0 0x7f9f7c14d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.015+0000 7f9f83b2e640 1 --2- 172.21.15.76:0/915631066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c0cc0d0 0x7f9f7c13e8f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.016+0000 7f9f83b2e640 1 -- 172.21.15.76:0/915631066 >> 172.21.15.76:0/915631066 conn(0x7f9f7c06ac80 msgr2=0x7f9f7c0b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:45.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.016+0000 7f9f83b2e640 1 -- 172.21.15.76:0/915631066 shutdown_connections 2024-03-08T06:01:45.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.016+0000 7f9f83b2e640 1 -- 172.21.15.76:0/915631066 wait complete. 2024-03-08T06:01:45.027 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-08T06:01:45.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.127+0000 7fd4e2cff640 1 Processor -- start 2024-03-08T06:01:45.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.127+0000 7fd4e2cff640 1 -- start start 2024-03-08T06:01:45.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.127+0000 7fd4e2cff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4dc071390 0x7fd4dc0cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:45.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.127+0000 7fd4e2cff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd4dc0cd310 0x7fd4dc0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:45.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.128+0000 7fd4e2cff640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4dc06d880 con 0x7fd4dc06d3f0 2024-03-08T06:01:45.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.128+0000 7fd4e2cff640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4dc06e010 con 0x7fd4dc0cd310 2024-03-08T06:01:45.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.128+0000 7fd4e2cff640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc06e2f0 con 0x7fd4dc071390 2024-03-08T06:01:45.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.128+0000 7fd4e0a9b640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4dc071390 0x7fd4dc0cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:45.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.128+0000 7fd4e129c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd4dc0cd310 0x7fd4dc0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:45.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.128+0000 7fd4dbfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd4dc06d3f0 0x7fd4dc071770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:36014/0 (socket says 172.21.15.76:36014) 2024-03-08T06:01:45.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.128+0000 7fd4dbfff640 1 -- 172.21.15.76:0/2938431010 learned_addr learned my addr 172.21.15.76:0/2938431010 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:45.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.128+0000 7fd4db7fe640 1 -- 172.21.15.76:0/2938431010 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354312783 0 0) 0x7fd4dc06d880 con 0x7fd4dc06d3f0 2024-03-08T06:01:45.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.128+0000 7fd4db7fe640 1 -- 172.21.15.76:0/2938431010 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4cc003520 con 0x7fd4dc06d3f0 2024-03-08T06:01:45.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.128+0000 7fd4e0a9b640 1 -- 172.21.15.76:0/2938431010 >> v2:172.21.15.81:3300/0 conn(0x7fd4dc0cd310 msgr2=0x7fd4dc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:45.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.128+0000 7fd4e0a9b640 1 --2- 172.21.15.76:0/2938431010 >> v2:172.21.15.81:3300/0 conn(0x7fd4dc0cd310 0x7fd4dc0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.128+0000 7fd4e0a9b640 1 -- 172.21.15.76:0/2938431010 >> v1:172.21.15.81:6789/0 conn(0x7fd4dc06d3f0 legacy=0x7fd4dc071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:45.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.128+0000 7fd4e0a9b640 1 -- 172.21.15.76:0/2938431010 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc0ce7b0 con 0x7fd4dc071390 2024-03-08T06:01:45.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.129+0000 7fd4e0a9b640 1 --2- 172.21.15.76:0/2938431010 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4dc071390 0x7fd4dc0cf0e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00097a0 tx=0x7fd4d002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f99d0c5a04bb7231 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:45.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.129+0000 7fd4db7fe640 1 -- 172.21.15.76:0/2938431010 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0035020 con 0x7fd4dc071390 2024-03-08T06:01:45.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.129+0000 7fd4db7fe640 1 -- 172.21.15.76:0/2938431010 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d0031830 con 0x7fd4dc071390 2024-03-08T06:01:45.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.129+0000 7fd4db7fe640 1 -- 172.21.15.76:0/2938431010 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0031be0 con 0x7fd4dc071390 2024-03-08T06:01:45.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.130+0000 7fd4e2cff640 1 -- 172.21.15.76:0/2938431010 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4dc071390 msgr2=0x7fd4dc0cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:45.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.130+0000 7fd4e2cff640 1 --2- 172.21.15.76:0/2938431010 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4dc071390 0x7fd4dc0cf0e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00097a0 tx=0x7fd4d002b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.130+0000 7fd4e2cff640 1 -- 172.21.15.76:0/2938431010 shutdown_connections 2024-03-08T06:01:45.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.130+0000 7fd4e2cff640 1 --2- 172.21.15.76:0/2938431010 >> v2:172.21.15.81:3300/0 conn(0x7fd4dc0cd310 0x7fd4dc0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.130+0000 7fd4e2cff640 1 --2- 172.21.15.76:0/2938431010 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4dc071390 0x7fd4dc0cf0e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.130+0000 7fd4e2cff640 1 -- 172.21.15.76:0/2938431010 >> 172.21.15.76:0/2938431010 conn(0x7fd4dc0cf740 msgr2=0x7fd4dc0cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:45.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.130+0000 7fd4e2cff640 1 -- 172.21.15.76:0/2938431010 shutdown_connections 2024-03-08T06:01:45.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.130+0000 7fd4e2cff640 1 -- 172.21.15.76:0/2938431010 wait complete. 2024-03-08T06:01:45.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.131+0000 7fd4e2cff640 1 Processor -- start 2024-03-08T06:01:45.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.131+0000 7fd4e2cff640 1 -- start start 2024-03-08T06:01:45.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4e2cff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4dc071390 0x7fd4dc15dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:45.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4e2cff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4dc0cd310 0x7fd4dc1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:45.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4e2cff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4dc1502a0 0x7fd4dc146db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:45.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4e2cff640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc15c060 con 0x7fd4dc1502a0 2024-03-08T06:01:45.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4e2cff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc15bee0 con 0x7fd4dc0cd310 2024-03-08T06:01:45.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4e2cff640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc0cdd00 con 0x7fd4dc071390 2024-03-08T06:01:45.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4e129c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4dc1502a0 0x7fd4dc146db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:45.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4e129c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4dc1502a0 0x7fd4dc146db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45602/0 (socket says 172.21.15.76:45602) 2024-03-08T06:01:45.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4e129c640 1 -- 172.21.15.76:0/1838089556 learned_addr learned my addr 172.21.15.76:0/1838089556 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4e0a9b640 1 --2- 172.21.15.76:0/1838089556 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4dc071390 0x7fd4dc15dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4dbfff640 1 --2- 172.21.15.76:0/1838089556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4dc0cd310 0x7fd4dc1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4e129c640 1 -- 172.21.15.76:0/1838089556 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4dc071390 msgr2=0x7fd4dc15dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4e129c640 1 --2- 172.21.15.76:0/1838089556 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4dc071390 0x7fd4dc15dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4e129c640 1 -- 172.21.15.76:0/1838089556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4dc0cd310 msgr2=0x7fd4dc1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4e129c640 1 --2- 172.21.15.76:0/1838089556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4dc0cd310 0x7fd4dc1536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4e129c640 1 -- 172.21.15.76:0/1838089556 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc150b20 con 0x7fd4dc1502a0 2024-03-08T06:01:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4dbfff640 1 --2- 172.21.15.76:0/1838089556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4dc0cd310 0x7fd4dc1536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.132+0000 7fd4e129c640 1 --2- 172.21.15.76:0/1838089556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4dc1502a0 0x7fd4dc146db0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009d30 tx=0x7fd4d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.133+0000 7fd4d97fa640 1 -- 172.21.15.76:0/1838089556 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4018020 con 0x7fd4dc1502a0 2024-03-08T06:01:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.133+0000 7fd4d97fa640 1 -- 172.21.15.76:0/1838089556 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d4014480 con 0x7fd4dc1502a0 2024-03-08T06:01:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.133+0000 7fd4e2cff640 1 -- 172.21.15.76:0/1838089556 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4dc15e340 con 0x7fd4dc1502a0 2024-03-08T06:01:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.133+0000 7fd4d97fa640 1 -- 172.21.15.76:0/1838089556 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4014870 con 0x7fd4dc1502a0 2024-03-08T06:01:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.133+0000 7fd4e2cff640 1 -- 172.21.15.76:0/1838089556 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc15e900 con 0x7fd4dc1502a0 2024-03-08T06:01:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.134+0000 7fd4e2cff640 1 -- 172.21.15.76:0/1838089556 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4dc04f030 con 0x7fd4dc1502a0 2024-03-08T06:01:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.134+0000 7fd4d97fa640 1 -- 172.21.15.76:0/1838089556 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4d4005d10 con 0x7fd4dc1502a0 2024-03-08T06:01:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.135+0000 7fd4d97fa640 1 --2- 172.21.15.76:0/1838089556 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd4c006b4d0 0x7fd4c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.135+0000 7fd4e0a9b640 1 --2- 172.21.15.76:0/1838089556 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd4c006b4d0 0x7fd4c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.135+0000 7fd4d97fa640 1 -- 172.21.15.76:0/1838089556 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd4d408a9d0 con 0x7fd4dc1502a0 2024-03-08T06:01:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.135+0000 7fd4e0a9b640 1 --2- 172.21.15.76:0/1838089556 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd4c006b4d0 0x7fd4c006d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0036000 tx=0x7fd4d003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:45.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.139+0000 7fd4d97fa640 1 -- 172.21.15.76:0/1838089556 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4d4022070 con 0x7fd4dc1502a0 2024-03-08T06:01:45.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.311+0000 7fd4e2cff640 1 -- 172.21.15.76:0/1838089556 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd4dc15eb00 con 0x7fd4dc1502a0 2024-03-08T06:01:45.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.311+0000 7fd4d97fa640 1 -- 172.21.15.76:0/1838089556 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd4dc150b20 con 0x7fd4dc1502a0 2024-03-08T06:01:45.312 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:45.312 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2024-03-08T06:01:45.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:01:45.312 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2024-03-08T06:01:45.312 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2024-03-08T06:01:45.312 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:01:45.312 INFO:teuthology.orchestra.run.smithi076.stdout:] 2024-03-08T06:01:45.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.313+0000 7fd4e2cff640 1 -- 172.21.15.76:0/1838089556 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd4c006b4d0 msgr2=0x7fd4c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:45.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.313+0000 7fd4e2cff640 1 --2- 172.21.15.76:0/1838089556 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd4c006b4d0 0x7fd4c006d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0036000 tx=0x7fd4d003c040 comp rx=0 tx=0).stop 2024-03-08T06:01:45.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.313+0000 7fd4e2cff640 1 -- 172.21.15.76:0/1838089556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4dc1502a0 msgr2=0x7fd4dc146db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:45.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.313+0000 7fd4e2cff640 1 --2- 172.21.15.76:0/1838089556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4dc1502a0 0x7fd4dc146db0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009d30 tx=0x7fd4d400c490 comp rx=0 tx=0).stop 2024-03-08T06:01:45.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.313+0000 7fd4e2cff640 1 -- 172.21.15.76:0/1838089556 shutdown_connections 2024-03-08T06:01:45.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.313+0000 7fd4e2cff640 1 --2- 172.21.15.76:0/1838089556 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd4c006b4d0 0x7fd4c006d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.313+0000 7fd4e2cff640 1 --2- 172.21.15.76:0/1838089556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4dc1502a0 0x7fd4dc146db0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.313+0000 7fd4e2cff640 1 --2- 172.21.15.76:0/1838089556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4dc0cd310 0x7fd4dc1536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.313+0000 7fd4e2cff640 1 --2- 172.21.15.76:0/1838089556 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd4dc071390 0x7fd4dc15dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.313+0000 7fd4e2cff640 1 -- 172.21.15.76:0/1838089556 >> 172.21.15.76:0/1838089556 conn(0x7fd4dc0cf740 msgr2=0x7fd4dc0b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:45.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.313+0000 7fd4e2cff640 1 -- 172.21.15.76:0/1838089556 shutdown_connections 2024-03-08T06:01:45.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.313+0000 7fd4e2cff640 1 -- 172.21.15.76:0/1838089556 wait complete. 2024-03-08T06:01:45.324 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-08T06:01:45.325 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-08T06:01:45.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.429+0000 7f6055840640 1 Processor -- start 2024-03-08T06:01:45.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.429+0000 7f6055840640 1 -- start start 2024-03-08T06:01:45.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.430+0000 7f6055840640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f605006ff90 0x7f60500d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:45.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.430+0000 7f6055840640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f60500d20d0 0x7f60500cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:45.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.430+0000 7f6055840640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605015f7f0 con 0x7f605015bd90 2024-03-08T06:01:45.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.430+0000 7f6055840640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f605015fad0 con 0x7f60500d20d0 2024-03-08T06:01:45.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.430+0000 7f6055840640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f605015fdb0 con 0x7f605006ff90 2024-03-08T06:01:45.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.430+0000 7f604f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f60500d20d0 0x7f60500cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:45.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.430+0000 7f604f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f60500d20d0 0x7f60500cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45612/0 (socket says 172.21.15.76:45612) 2024-03-08T06:01:45.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.430+0000 7f604f7fe640 1 -- 172.21.15.76:0/3556764409 learned_addr learned my addr 172.21.15.76:0/3556764409 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:45.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.430+0000 7f604e7fc640 1 --2- 172.21.15.76:0/3556764409 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f605006ff90 0x7f60500d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:45.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.430+0000 7f604f7fe640 1 -- 172.21.15.76:0/3556764409 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f605006ff90 msgr2=0x7f60500d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:45.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.430+0000 7f604f7fe640 1 --2- 172.21.15.76:0/3556764409 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f605006ff90 0x7f60500d1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.430+0000 7f604f7fe640 1 -- 172.21.15.76:0/3556764409 >> v1:172.21.15.81:6789/0 conn(0x7f605015bd90 legacy=0x7f605006f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:45.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.430+0000 7f604f7fe640 1 -- 172.21.15.76:0/3556764409 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6050160090 con 0x7f60500d20d0 2024-03-08T06:01:45.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.431+0000 7f604f7fe640 1 --2- 172.21.15.76:0/3556764409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60500d20d0 0x7f60500cf9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6048009e50 tx=0x7f604802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bda7a95b9e3441e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:45.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.431+0000 7f604dffb640 1 -- 172.21.15.76:0/3556764409 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604802cb90 con 0x7f60500d20d0 2024-03-08T06:01:45.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.431+0000 7f604dffb640 1 -- 172.21.15.76:0/3556764409 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604802cd30 con 0x7f60500d20d0 2024-03-08T06:01:45.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.431+0000 7f604dffb640 1 -- 172.21.15.76:0/3556764409 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048034610 con 0x7f60500d20d0 2024-03-08T06:01:45.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.432+0000 7f6055840640 1 -- 172.21.15.76:0/3556764409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60500d20d0 msgr2=0x7f60500cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:45.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.432+0000 7f6055840640 1 --2- 172.21.15.76:0/3556764409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60500d20d0 0x7f60500cf9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6048009e50 tx=0x7f604802bff0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.432+0000 7f6055840640 1 -- 172.21.15.76:0/3556764409 shutdown_connections 2024-03-08T06:01:45.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.432+0000 7f6055840640 1 --2- 172.21.15.76:0/3556764409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60500d20d0 0x7f60500cf9c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.432+0000 7f6055840640 1 --2- 172.21.15.76:0/3556764409 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f605006ff90 0x7f60500d1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.432+0000 7f6055840640 1 -- 172.21.15.76:0/3556764409 >> 172.21.15.76:0/3556764409 conn(0x7f605006ae60 msgr2=0x7f605006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:45.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.432+0000 7f6055840640 1 -- 172.21.15.76:0/3556764409 shutdown_connections 2024-03-08T06:01:45.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.432+0000 7f6055840640 1 -- 172.21.15.76:0/3556764409 wait complete. 2024-03-08T06:01:45.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.433+0000 7f6055840640 1 Processor -- start 2024-03-08T06:01:45.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.433+0000 7f6055840640 1 -- start start 2024-03-08T06:01:45.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f6055840640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f605006ff90 0x7f605015ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:45.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f6055840640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60500d20d0 0x7f60500b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:45.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f6055840640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f605015bd90 0x7f60500b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:45.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f6055840640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60500cff00 con 0x7f605006ff90 2024-03-08T06:01:45.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f6055840640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60500d2760 con 0x7f60500d20d0 2024-03-08T06:01:45.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f6055840640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60500d0080 con 0x7f605015bd90 2024-03-08T06:01:45.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f604effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f605006ff90 0x7f605015ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:45.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f604effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f605006ff90 0x7f605015ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45624/0 (socket says 172.21.15.76:45624) 2024-03-08T06:01:45.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f604effd640 1 -- 172.21.15.76:0/2457981416 learned_addr learned my addr 172.21.15.76:0/2457981416 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:45.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f604f7fe640 1 --2- 172.21.15.76:0/2457981416 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f605015bd90 0x7f60500b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:45.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f604e7fc640 1 --2- 172.21.15.76:0/2457981416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60500d20d0 0x7f60500b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:45.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f604effd640 1 -- 172.21.15.76:0/2457981416 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f605015bd90 msgr2=0x7f60500b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:45.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f604effd640 1 --2- 172.21.15.76:0/2457981416 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f605015bd90 0x7f60500b5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f604effd640 1 -- 172.21.15.76:0/2457981416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60500d20d0 msgr2=0x7f60500b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:45.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f604effd640 1 --2- 172.21.15.76:0/2457981416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60500d20d0 0x7f60500b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f604effd640 1 -- 172.21.15.76:0/2457981416 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60501f4c40 con 0x7f605006ff90 2024-03-08T06:01:45.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f604f7fe640 1 --2- 172.21.15.76:0/2457981416 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f605015bd90 0x7f60500b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:45.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.434+0000 7f604effd640 1 --2- 172.21.15.76:0/2457981416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f605006ff90 0x7f605015ea70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f604000c3c0 tx=0x7f604000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:45.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.435+0000 7f6037fff640 1 -- 172.21.15.76:0/2457981416 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60400052d0 con 0x7f605006ff90 2024-03-08T06:01:45.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.435+0000 7f6037fff640 1 -- 172.21.15.76:0/2457981416 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6040005490 con 0x7f605006ff90 2024-03-08T06:01:45.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.435+0000 7f6055840640 1 -- 172.21.15.76:0/2457981416 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60501f76e0 con 0x7f605006ff90 2024-03-08T06:01:45.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.435+0000 7f6037fff640 1 -- 172.21.15.76:0/2457981416 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040007260 con 0x7f605006ff90 2024-03-08T06:01:45.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.435+0000 7f6055840640 1 -- 172.21.15.76:0/2457981416 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60501f7ca0 con 0x7f605006ff90 2024-03-08T06:01:45.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.436+0000 7f6055840640 1 -- 172.21.15.76:0/2457981416 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60501f7ea0 con 0x7f605006ff90 2024-03-08T06:01:45.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.436+0000 7f6037fff640 1 -- 172.21.15.76:0/2457981416 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6040007400 con 0x7f605006ff90 2024-03-08T06:01:45.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.436+0000 7f6037fff640 1 --2- 172.21.15.76:0/2457981416 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f603c06b4d0 0x7f603c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:45.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.437+0000 7f604e7fc640 1 --2- 172.21.15.76:0/2457981416 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f603c06b4d0 0x7f603c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:45.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.437+0000 7f6037fff640 1 -- 172.21.15.76:0/2457981416 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f60400894e0 con 0x7f605006ff90 2024-03-08T06:01:45.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.437+0000 7f604e7fc640 1 --2- 172.21.15.76:0/2457981416 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f603c06b4d0 0x7f603c06d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f60501f85b0 tx=0x7f6044008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:45.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.441+0000 7f6037fff640 1 -- 172.21.15.76:0/2457981416 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6040058520 con 0x7f605006ff90 2024-03-08T06:01:45.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.610+0000 7f6055840640 1 -- 172.21.15.76:0/2457981416 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f60501f74d0 con 0x7f605006ff90 2024-03-08T06:01:45.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.867+0000 7f6037fff640 1 -- 172.21.15.76:0/2457981416 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6040057e20 con 0x7f605006ff90 2024-03-08T06:01:45.868 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2024-03-08T06:01:45.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.870+0000 7f6055840640 1 -- 172.21.15.76:0/2457981416 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f603c06b4d0 msgr2=0x7f603c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:45.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.870+0000 7f6055840640 1 --2- 172.21.15.76:0/2457981416 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f603c06b4d0 0x7f603c06d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f60501f85b0 tx=0x7f6044008040 comp rx=0 tx=0).stop 2024-03-08T06:01:45.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.870+0000 7f6055840640 1 -- 172.21.15.76:0/2457981416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f605006ff90 msgr2=0x7f605015ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:45.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.870+0000 7f6055840640 1 --2- 172.21.15.76:0/2457981416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f605006ff90 0x7f605015ea70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f604000c3c0 tx=0x7f604000c880 comp rx=0 tx=0).stop 2024-03-08T06:01:45.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.870+0000 7f6055840640 1 -- 172.21.15.76:0/2457981416 shutdown_connections 2024-03-08T06:01:45.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.870+0000 7f6055840640 1 --2- 172.21.15.76:0/2457981416 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f603c06b4d0 0x7f603c06d990 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.870+0000 7f6055840640 1 --2- 172.21.15.76:0/2457981416 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f605015bd90 0x7f60500b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.870+0000 7f6055840640 1 --2- 172.21.15.76:0/2457981416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60500d20d0 0x7f60500b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.870+0000 7f6055840640 1 --2- 172.21.15.76:0/2457981416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f605006ff90 0x7f605015ea70 secure :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f604000c3c0 tx=0x7f604000c880 comp rx=0 tx=0).stop 2024-03-08T06:01:45.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.870+0000 7f6055840640 1 -- 172.21.15.76:0/2457981416 >> 172.21.15.76:0/2457981416 conn(0x7f605006ae60 msgr2=0x7f605015b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:45.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.870+0000 7f6055840640 1 -- 172.21.15.76:0/2457981416 shutdown_connections 2024-03-08T06:01:45.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.870+0000 7f6055840640 1 -- 172.21.15.76:0/2457981416 wait complete. 2024-03-08T06:01:45.885 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-08T06:01:45.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.992+0000 7f3be0b1e640 1 Processor -- start 2024-03-08T06:01:45.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.992+0000 7f3be0b1e640 1 -- start start 2024-03-08T06:01:45.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.993+0000 7f3be0b1e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3bdc162660 0x7f3bdc0acf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:45.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.993+0000 7f3be0b1e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bdc0b3210 con 0x7f3bdc0b41d0 2024-03-08T06:01:45.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.993+0000 7f3be0b1e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bdc0b34f0 con 0x7f3bdc0b3df0 2024-03-08T06:01:45.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.993+0000 7f3be0b1e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bdc0b37d0 con 0x7f3bdc162660 2024-03-08T06:01:45.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.993+0000 7f3bdad9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3bdc0b41d0 0x7f3bdc072460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48466/0 (socket says 172.21.15.76:48466) 2024-03-08T06:01:45.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.993+0000 7f3bdad9d640 1 -- 172.21.15.76:0/554400397 learned_addr learned my addr 172.21.15.76:0/554400397 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:45.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd9d9b640 1 --2- 172.21.15.76:0/554400397 >> v2:172.21.15.81:3300/0 conn(0x7f3bdc162660 0x7f3bdc0acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:45.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd959a640 1 -- 172.21.15.76:0/554400397 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617458629 0 0) 0x7f3bdc0b3210 con 0x7f3bdc0b41d0 2024-03-08T06:01:45.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd959a640 1 -- 172.21.15.76:0/554400397 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bc8003520 con 0x7f3bdc0b41d0 2024-03-08T06:01:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd959a640 1 -- 172.21.15.76:0/554400397 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3376928097 0 0) 0x7f3bc8003520 con 0x7f3bdc0b41d0 2024-03-08T06:01:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd959a640 1 -- 172.21.15.76:0/554400397 >> v2:172.21.15.81:3300/0 conn(0x7f3bdc162660 msgr2=0x7f3bdc0acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd959a640 1 --2- 172.21.15.76:0/554400397 >> v2:172.21.15.81:3300/0 conn(0x7f3bdc162660 0x7f3bdc0acf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd959a640 1 -- 172.21.15.76:0/554400397 >> v1:172.21.15.81:6789/0 conn(0x7f3bdc0b3df0 legacy=0x7f3bdc06bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd959a640 1 -- 172.21.15.76:0/554400397 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc0ad480 con 0x7f3bdc0b41d0 2024-03-08T06:01:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd959a640 1 -- 172.21.15.76:0/554400397 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f3bcc0019f0 con 0x7f3bdc0b41d0 2024-03-08T06:01:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd959a640 1 -- 172.21.15.76:0/554400397 >> v1:172.21.15.76:6789/0 conn(0x7f3bdc0b41d0 legacy=0x7f3bdc072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd959a640 1 --2- 172.21.15.76:0/554400397 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3bc80039b0 0x7f3bc8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd959a640 1 --2- 172.21.15.76:0/554400397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bc80043b0 0x7f3bc8004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd959a640 1 --2- 172.21.15.76:0/554400397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3bc8005120 0x7f3bc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:45.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd959a640 1 -- 172.21.15.76:0/554400397 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0b37d0 con 0x7f3bc80043b0 2024-03-08T06:01:45.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd959a640 1 -- 172.21.15.76:0/554400397 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc800c060 con 0x7f3bc8005120 2024-03-08T06:01:45.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd959a640 1 -- 172.21.15.76:0/554400397 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc800c260 con 0x7f3bc80039b0 2024-03-08T06:01:45.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd9d9b640 1 --2- 172.21.15.76:0/554400397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bc80043b0 0x7f3bc8004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:45.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bdad9d640 1 --2- 172.21.15.76:0/554400397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3bc8005120 0x7f3bc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:45.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bda59c640 1 --2- 172.21.15.76:0/554400397 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3bc80039b0 0x7f3bc8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:45.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd9d9b640 1 -- 172.21.15.76:0/554400397 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3bc80039b0 msgr2=0x7f3bc8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:45.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd9d9b640 1 --2- 172.21.15.76:0/554400397 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3bc80039b0 0x7f3bc8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd9d9b640 1 -- 172.21.15.76:0/554400397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3bc8005120 msgr2=0x7f3bc800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:45.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd9d9b640 1 --2- 172.21.15.76:0/554400397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3bc8005120 0x7f3bc800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:45.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bd9d9b640 1 -- 172.21.15.76:0/554400397 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8003520 con 0x7f3bc80043b0 2024-03-08T06:01:45.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.994+0000 7f3bdad9d640 1 -- 172.21.15.76:0/554400397 reap_dead start 2024-03-08T06:01:46.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.995+0000 7f3bd9d9b640 1 --2- 172.21.15.76:0/554400397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bc80043b0 0x7f3bc8004880 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009a90 tx=0x7f3bc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e8d89544248ef3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:46.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.995+0000 7f3bd959a640 1 -- 172.21.15.76:0/554400397 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4039070 con 0x7f3bc80043b0 2024-03-08T06:01:46.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.995+0000 7f3bd959a640 1 -- 172.21.15.76:0/554400397 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc402c680 con 0x7f3bc80043b0 2024-03-08T06:01:46.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.995+0000 7f3bd959a640 1 -- 172.21.15.76:0/554400397 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4009690 con 0x7f3bc80043b0 2024-03-08T06:01:46.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.996+0000 7f3be0b1e640 1 -- 172.21.15.76:0/554400397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bc80043b0 msgr2=0x7f3bc8004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:46.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.996+0000 7f3be0b1e640 1 --2- 172.21.15.76:0/554400397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bc80043b0 0x7f3bc8004880 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009a90 tx=0x7f3bc8005830 comp rx=0 tx=0).stop 2024-03-08T06:01:46.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.996+0000 7f3be0b1e640 1 -- 172.21.15.76:0/554400397 shutdown_connections 2024-03-08T06:01:46.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.996+0000 7f3be0b1e640 1 --2- 172.21.15.76:0/554400397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bc80043b0 0x7f3bc8004880 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:46.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.996+0000 7f3be0b1e640 1 -- 172.21.15.76:0/554400397 >> 172.21.15.76:0/554400397 conn(0x7f3bdc0d0100 msgr2=0x7f3bdc0d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:46.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.996+0000 7f3be0b1e640 1 -- 172.21.15.76:0/554400397 shutdown_connections 2024-03-08T06:01:46.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.996+0000 7f3be0b1e640 1 -- 172.21.15.76:0/554400397 wait complete. 2024-03-08T06:01:46.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.997+0000 7f3be0b1e640 1 Processor -- start 2024-03-08T06:01:46.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.997+0000 7f3be0b1e640 1 -- start start 2024-03-08T06:01:46.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3be0b1e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3bc80043b0 0x7f3bdc15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:46.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3be0b1e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3bdc14ef90 0x7f3bdc14e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:46.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3be0b1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bdc152720 0x7f3bdc152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:46.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3be0b1e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0ad2d0 con 0x7f3bdc152720 2024-03-08T06:01:46.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3be0b1e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0ad150 con 0x7f3bc80043b0 2024-03-08T06:01:46.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3be0b1e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc06bf10 con 0x7f3bdc14ef90 2024-03-08T06:01:46.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3bdad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bdc152720 0x7f3bdc152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:46.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3bdad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bdc152720 0x7f3bdc152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45632/0 (socket says 172.21.15.76:45632) 2024-03-08T06:01:46.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3bdad9d640 1 -- 172.21.15.76:0/2338671310 learned_addr learned my addr 172.21.15.76:0/2338671310 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:46.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3bd9d9b640 1 --2- 172.21.15.76:0/2338671310 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3bdc14ef90 0x7f3bdc14e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:46.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3bda59c640 1 --2- 172.21.15.76:0/2338671310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3bc80043b0 0x7f3bdc15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:46.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3bdad9d640 1 -- 172.21.15.76:0/2338671310 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3bdc14ef90 msgr2=0x7f3bdc14e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:46.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3bdad9d640 1 --2- 172.21.15.76:0/2338671310 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3bdc14ef90 0x7f3bdc14e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:46.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3bdad9d640 1 -- 172.21.15.76:0/2338671310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3bc80043b0 msgr2=0x7f3bdc15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:46.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3bdad9d640 1 --2- 172.21.15.76:0/2338671310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3bc80043b0 0x7f3bdc15d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:46.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3bdad9d640 1 -- 172.21.15.76:0/2338671310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc15dd80 con 0x7f3bdc152720 2024-03-08T06:01:46.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.998+0000 7f3bdad9d640 1 --2- 172.21.15.76:0/2338671310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bdc152720 0x7f3bdc152b80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc0aca80 tx=0x7f3bcc007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:46.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.999+0000 7f3bd2ffd640 1 -- 172.21.15.76:0/2338671310 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc00ab80 con 0x7f3bdc152720 2024-03-08T06:01:46.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.999+0000 7f3bd2ffd640 1 -- 172.21.15.76:0/2338671310 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc019070 con 0x7f3bdc152720 2024-03-08T06:01:46.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.999+0000 7f3be0b1e640 1 -- 172.21.15.76:0/2338671310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bdc15e060 con 0x7f3bdc152720 2024-03-08T06:01:46.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.999+0000 7f3bd2ffd640 1 -- 172.21.15.76:0/2338671310 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc0133f0 con 0x7f3bdc152720 2024-03-08T06:01:46.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:45.999+0000 7f3be0b1e640 1 -- 172.21.15.76:0/2338671310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bdc15e620 con 0x7f3bdc152720 2024-03-08T06:01:46.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.000+0000 7f3be0b1e640 1 -- 172.21.15.76:0/2338671310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bdc06a1d0 con 0x7f3bdc152720 2024-03-08T06:01:46.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.000+0000 7f3bd2ffd640 1 -- 172.21.15.76:0/2338671310 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3bcc013590 con 0x7f3bdc152720 2024-03-08T06:01:46.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.000+0000 7f3bd2ffd640 1 --2- 172.21.15.76:0/2338671310 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3bbc06b500 0x7f3bbc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:46.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.001+0000 7f3bd2ffd640 1 -- 172.21.15.76:0/2338671310 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f3bcc08a550 con 0x7f3bdc152720 2024-03-08T06:01:46.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.001+0000 7f3bda59c640 1 --2- 172.21.15.76:0/2338671310 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3bbc06b500 0x7f3bbc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:46.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.001+0000 7f3bda59c640 1 --2- 172.21.15.76:0/2338671310 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3bbc06b500 0x7f3bbc06d9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc1f70b0 tx=0x7f3bd4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:46.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.005+0000 7f3bd2ffd640 1 -- 172.21.15.76:0/2338671310 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3bcc058dd0 con 0x7f3bdc152720 2024-03-08T06:01:46.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.176+0000 7f3be0b1e640 1 -- 172.21.15.76:0/2338671310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3bdc066f90 con 0x7f3bdc152720 2024-03-08T06:01:46.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.872+0000 7f3bd2ffd640 1 -- 172.21.15.76:0/2338671310 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3bcc05ce70 con 0x7f3bdc152720 2024-03-08T06:01:46.873 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2024-03-08T06:01:46.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.875+0000 7f3be0b1e640 1 -- 172.21.15.76:0/2338671310 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3bbc06b500 msgr2=0x7f3bbc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:46.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.875+0000 7f3be0b1e640 1 --2- 172.21.15.76:0/2338671310 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3bbc06b500 0x7f3bbc06d9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc1f70b0 tx=0x7f3bd4009040 comp rx=0 tx=0).stop 2024-03-08T06:01:46.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.875+0000 7f3be0b1e640 1 -- 172.21.15.76:0/2338671310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bdc152720 msgr2=0x7f3bdc152b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:46.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.875+0000 7f3be0b1e640 1 --2- 172.21.15.76:0/2338671310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bdc152720 0x7f3bdc152b80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc0aca80 tx=0x7f3bcc007a90 comp rx=0 tx=0).stop 2024-03-08T06:01:46.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.875+0000 7f3be0b1e640 1 -- 172.21.15.76:0/2338671310 shutdown_connections 2024-03-08T06:01:46.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.875+0000 7f3be0b1e640 1 --2- 172.21.15.76:0/2338671310 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3bbc06b500 0x7f3bbc06d9c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:46.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.875+0000 7f3be0b1e640 1 --2- 172.21.15.76:0/2338671310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bdc152720 0x7f3bdc152b80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:46.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.875+0000 7f3be0b1e640 1 --2- 172.21.15.76:0/2338671310 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3bdc14ef90 0x7f3bdc14e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:46.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.875+0000 7f3be0b1e640 1 --2- 172.21.15.76:0/2338671310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3bc80043b0 0x7f3bdc15d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:46.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.875+0000 7f3be0b1e640 1 -- 172.21.15.76:0/2338671310 >> 172.21.15.76:0/2338671310 conn(0x7f3bdc0d0100 msgr2=0x7f3bdc0cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:46.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.875+0000 7f3be0b1e640 1 -- 172.21.15.76:0/2338671310 shutdown_connections 2024-03-08T06:01:46.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:46.875+0000 7f3be0b1e640 1 -- 172.21.15.76:0/2338671310 wait complete. 2024-03-08T06:01:46.898 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-08T06:01:47.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.003+0000 7f3afd4c0640 1 Processor -- start 2024-03-08T06:01:47.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.003+0000 7f3afd4c0640 1 -- start start 2024-03-08T06:01:47.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.004+0000 7f3afd4c0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3af8071e10 0x7f3af80cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:47.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.004+0000 7f3afd4c0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3af80b40d0 0x7f3af80b4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:47.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.004+0000 7f3afd4c0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af815c060 con 0x7f3af8145240 2024-03-08T06:01:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.004+0000 7f3afd4c0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af80b4a80 con 0x7f3af8071e10 2024-03-08T06:01:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.004+0000 7f3afd4c0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af815c390 con 0x7f3af80b40d0 2024-03-08T06:01:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.004+0000 7f3af6ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3af8145240 0x7f3af815c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48476/0 (socket says 172.21.15.76:48476) 2024-03-08T06:01:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.004+0000 7f3af6ffd640 1 -- 172.21.15.76:0/1237429399 learned_addr learned my addr 172.21.15.76:0/1237429399 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.005+0000 7f3af67fc640 1 --2- 172.21.15.76:0/1237429399 >> v2:172.21.15.81:3300/0 conn(0x7f3af8071e10 0x7f3af80cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.005+0000 7f3af77fe640 1 --2- 172.21.15.76:0/1237429399 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3af80b40d0 0x7f3af80b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.005+0000 7f3af5ffb640 1 -- 172.21.15.76:0/1237429399 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028878289 0 0) 0x7f3af815c060 con 0x7f3af8145240 2024-03-08T06:01:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.005+0000 7f3af5ffb640 1 -- 172.21.15.76:0/1237429399 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ae0003520 con 0x7f3af8145240 2024-03-08T06:01:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.005+0000 7f3af67fc640 1 -- 172.21.15.76:0/1237429399 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3af80b40d0 msgr2=0x7f3af80b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.005+0000 7f3af67fc640 1 --2- 172.21.15.76:0/1237429399 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3af80b40d0 0x7f3af80b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.005+0000 7f3af67fc640 1 -- 172.21.15.76:0/1237429399 >> v1:172.21.15.76:6789/0 conn(0x7f3af8145240 legacy=0x7f3af815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.005+0000 7f3af67fc640 1 -- 172.21.15.76:0/1237429399 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af8145620 con 0x7f3af8071e10 2024-03-08T06:01:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.005+0000 7f3af77fe640 1 --2- 172.21.15.76:0/1237429399 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3af80b40d0 0x7f3af80b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.006+0000 7f3af67fc640 1 --2- 172.21.15.76:0/1237429399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3af8071e10 0x7f3af80cc0f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3ae80097a0 tx=0x7f3ae802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fd37a41eab38c90 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.006+0000 7f3af5ffb640 1 -- 172.21.15.76:0/1237429399 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8035020 con 0x7f3af8071e10 2024-03-08T06:01:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.006+0000 7f3af5ffb640 1 -- 172.21.15.76:0/1237429399 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae8031830 con 0x7f3af8071e10 2024-03-08T06:01:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.006+0000 7f3af5ffb640 1 -- 172.21.15.76:0/1237429399 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8031c00 con 0x7f3af8071e10 2024-03-08T06:01:47.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.006+0000 7f3afd4c0640 1 -- 172.21.15.76:0/1237429399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3af8071e10 msgr2=0x7f3af80cc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:47.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.006+0000 7f3afd4c0640 1 --2- 172.21.15.76:0/1237429399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3af8071e10 0x7f3af80cc0f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3ae80097a0 tx=0x7f3ae802b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:47.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.007+0000 7f3afd4c0640 1 -- 172.21.15.76:0/1237429399 shutdown_connections 2024-03-08T06:01:47.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.007+0000 7f3afd4c0640 1 --2- 172.21.15.76:0/1237429399 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3af80b40d0 0x7f3af80b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:47.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.007+0000 7f3afd4c0640 1 --2- 172.21.15.76:0/1237429399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3af8071e10 0x7f3af80cc0f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:47.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.007+0000 7f3afd4c0640 1 -- 172.21.15.76:0/1237429399 >> 172.21.15.76:0/1237429399 conn(0x7f3af806ac80 msgr2=0x7f3af806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:47.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.007+0000 7f3afd4c0640 1 -- 172.21.15.76:0/1237429399 shutdown_connections 2024-03-08T06:01:47.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.007+0000 7f3afd4c0640 1 -- 172.21.15.76:0/1237429399 wait complete. 2024-03-08T06:01:47.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.008+0000 7f3afd4c0640 1 Processor -- start 2024-03-08T06:01:47.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.008+0000 7f3afd4c0640 1 -- start start 2024-03-08T06:01:47.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3afd4c0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3af8071e10 0x7f3af80adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:47.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3afd4c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af80b40d0 0x7f3af80a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:47.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3afd4c0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3af8145240 0x7f3af80a8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:47.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3afd4c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af80ccd40 con 0x7f3af80b40d0 2024-03-08T06:01:47.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3afd4c0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af80cca60 con 0x7f3af8071e10 2024-03-08T06:01:47.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3afd4c0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af80af0e0 con 0x7f3af8145240 2024-03-08T06:01:47.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3af67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af80b40d0 0x7f3af80a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3af67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af80b40d0 0x7f3af80a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45636/0 (socket says 172.21.15.76:45636) 2024-03-08T06:01:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3af67fc640 1 -- 172.21.15.76:0/2298147889 learned_addr learned my addr 172.21.15.76:0/2298147889 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3af6ffd640 1 --2- 172.21.15.76:0/2298147889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3af8071e10 0x7f3af80adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3af77fe640 1 --2- 172.21.15.76:0/2298147889 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3af8145240 0x7f3af80a8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3af67fc640 1 -- 172.21.15.76:0/2298147889 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3af8145240 msgr2=0x7f3af80a8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3af67fc640 1 --2- 172.21.15.76:0/2298147889 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3af8145240 0x7f3af80a8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3af67fc640 1 -- 172.21.15.76:0/2298147889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3af8071e10 msgr2=0x7f3af80adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3af67fc640 1 --2- 172.21.15.76:0/2298147889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3af8071e10 0x7f3af80adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3af67fc640 1 -- 172.21.15.76:0/2298147889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af80ae460 con 0x7f3af80b40d0 2024-03-08T06:01:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3af6ffd640 1 --2- 172.21.15.76:0/2298147889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3af8071e10 0x7f3af80adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3af67fc640 1 --2- 172.21.15.76:0/2298147889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af80b40d0 0x7f3af80a84c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8034040 tx=0x7f3ae80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3adf7fe640 1 -- 172.21.15.76:0/2298147889 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8040070 con 0x7f3af80b40d0 2024-03-08T06:01:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3adf7fe640 1 -- 172.21.15.76:0/2298147889 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae803a850 con 0x7f3af80b40d0 2024-03-08T06:01:47.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3adf7fe640 1 -- 172.21.15.76:0/2298147889 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8035040 con 0x7f3af80b40d0 2024-03-08T06:01:47.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.009+0000 7f3afd4c0640 1 -- 172.21.15.76:0/2298147889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af815f810 con 0x7f3af80b40d0 2024-03-08T06:01:47.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.010+0000 7f3afd4c0640 1 -- 172.21.15.76:0/2298147889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af81322e0 con 0x7f3af80b40d0 2024-03-08T06:01:47.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.010+0000 7f3afd4c0640 1 -- 172.21.15.76:0/2298147889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af804f030 con 0x7f3af80b40d0 2024-03-08T06:01:47.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.011+0000 7f3adf7fe640 1 -- 172.21.15.76:0/2298147889 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ae8038020 con 0x7f3af80b40d0 2024-03-08T06:01:47.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.011+0000 7f3adf7fe640 1 --2- 172.21.15.76:0/2298147889 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3ae406b4d0 0x7f3ae406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:47.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.012+0000 7f3af6ffd640 1 --2- 172.21.15.76:0/2298147889 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3ae406b4d0 0x7f3ae406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:47.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.012+0000 7f3adf7fe640 1 -- 172.21.15.76:0/2298147889 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f3ae80ab970 con 0x7f3af80b40d0 2024-03-08T06:01:47.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.012+0000 7f3af6ffd640 1 --2- 172.21.15.76:0/2298147889 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3ae406b4d0 0x7f3ae406d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3aec001200 tx=0x7f3aec000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:47.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.016+0000 7f3adf7fe640 1 -- 172.21.15.76:0/2298147889 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ae807a750 con 0x7f3af80b40d0 2024-03-08T06:01:47.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.230+0000 7f3afd4c0640 1 -- 172.21.15.76:0/2298147889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3af80b3e90 con 0x7f3af80b40d0 2024-03-08T06:01:47.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.874+0000 7f3adf7fe640 1 -- 172.21.15.76:0/2298147889 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3ae807a050 con 0x7f3af80b40d0 2024-03-08T06:01:47.875 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 5 and data pool 6 2024-03-08T06:01:47.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.876+0000 7f3afd4c0640 1 -- 172.21.15.76:0/2298147889 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3ae406b4d0 msgr2=0x7f3ae406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:47.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.876+0000 7f3afd4c0640 1 --2- 172.21.15.76:0/2298147889 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3ae406b4d0 0x7f3ae406d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3aec001200 tx=0x7f3aec000bc0 comp rx=0 tx=0).stop 2024-03-08T06:01:47.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.876+0000 7f3afd4c0640 1 -- 172.21.15.76:0/2298147889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af80b40d0 msgr2=0x7f3af80a84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:47.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.876+0000 7f3afd4c0640 1 --2- 172.21.15.76:0/2298147889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af80b40d0 0x7f3af80a84c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8034040 tx=0x7f3ae80046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:47.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.876+0000 7f3afd4c0640 1 -- 172.21.15.76:0/2298147889 shutdown_connections 2024-03-08T06:01:47.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.876+0000 7f3afd4c0640 1 --2- 172.21.15.76:0/2298147889 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f3ae406b4d0 0x7f3ae406d990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:47.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.876+0000 7f3afd4c0640 1 --2- 172.21.15.76:0/2298147889 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3af8145240 0x7f3af80a8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:47.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.876+0000 7f3afd4c0640 1 --2- 172.21.15.76:0/2298147889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af80b40d0 0x7f3af80a84c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:47.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.876+0000 7f3afd4c0640 1 --2- 172.21.15.76:0/2298147889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3af8071e10 0x7f3af80adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:47.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.876+0000 7f3afd4c0640 1 -- 172.21.15.76:0/2298147889 >> 172.21.15.76:0/2298147889 conn(0x7f3af806ac80 msgr2=0x7f3af806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:47.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.876+0000 7f3afd4c0640 1 -- 172.21.15.76:0/2298147889 shutdown_connections 2024-03-08T06:01:47.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:47.876+0000 7f3afd4c0640 1 -- 172.21.15.76:0/2298147889 wait complete. 2024-03-08T06:01:47.900 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:48.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.001+0000 7f884049d640 1 Processor -- start 2024-03-08T06:01:48.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.002+0000 7f884049d640 1 -- start start 2024-03-08T06:01:48.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.002+0000 7f884049d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f883815a9f0 0x7f8838071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.002+0000 7f884049d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f88380c98e0 0x7f88380c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.002+0000 7f884049d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88380722e0 con 0x7f88380ca2d0 2024-03-08T06:01:48.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.002+0000 7f884049d640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88380ce790 con 0x7f883815a9f0 2024-03-08T06:01:48.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.002+0000 7f884049d640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88380ae8d0 con 0x7f88380c98e0 2024-03-08T06:01:48.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.003+0000 7f883ea3a640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f88380ca2d0 0x7f88380cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48486/0 (socket says 172.21.15.76:48486) 2024-03-08T06:01:48.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.003+0000 7f883ea3a640 1 -- 172.21.15.76:0/2881537032 learned_addr learned my addr 172.21.15.76:0/2881537032 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:48.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.003+0000 7f883da38640 1 --2- 172.21.15.76:0/2881537032 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f88380c98e0 0x7f88380c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.003+0000 7f883e239640 1 --2- 172.21.15.76:0/2881537032 >> v2:172.21.15.81:3300/0 conn(0x7f883815a9f0 0x7f8838071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.003+0000 7f883d237640 1 -- 172.21.15.76:0/2881537032 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359730949 0 0) 0x7f88380722e0 con 0x7f88380ca2d0 2024-03-08T06:01:48.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.003+0000 7f883d237640 1 -- 172.21.15.76:0/2881537032 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8824005730 con 0x7f88380ca2d0 2024-03-08T06:01:48.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.003+0000 7f883d237640 1 -- 172.21.15.76:0/2881537032 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1977968073 0 0) 0x7f8824005730 con 0x7f88380ca2d0 2024-03-08T06:01:48.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.003+0000 7f883da38640 1 -- 172.21.15.76:0/2881537032 >> v2:172.21.15.81:3300/0 conn(0x7f883815a9f0 msgr2=0x7f8838071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:48.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.003+0000 7f883da38640 1 --2- 172.21.15.76:0/2881537032 >> v2:172.21.15.81:3300/0 conn(0x7f883815a9f0 0x7f8838071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.003+0000 7f883da38640 1 -- 172.21.15.76:0/2881537032 >> v1:172.21.15.76:6789/0 conn(0x7f88380ca2d0 legacy=0x7f88380cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.003+0000 7f883da38640 1 -- 172.21.15.76:0/2881537032 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88380aebb0 con 0x7f88380c98e0 2024-03-08T06:01:48.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.003+0000 7f883e239640 1 --2- 172.21.15.76:0/2881537032 >> v2:172.21.15.81:3300/0 conn(0x7f883815a9f0 0x7f8838071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:01:48.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.004+0000 7f883da38640 1 --2- 172.21.15.76:0/2881537032 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f88380c98e0 0x7f88380c9d90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f88280097a0 tx=0x7f882802b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=715a77578f3a024f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:48.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.004+0000 7f883d237640 1 -- 172.21.15.76:0/2881537032 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828037070 con 0x7f88380c98e0 2024-03-08T06:01:48.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.004+0000 7f883d237640 1 -- 172.21.15.76:0/2881537032 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8828030510 con 0x7f88380c98e0 2024-03-08T06:01:48.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.004+0000 7f883d237640 1 -- 172.21.15.76:0/2881537032 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834001670 con 0x7f88380c98e0 2024-03-08T06:01:48.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.005+0000 7f884049d640 1 -- 172.21.15.76:0/2881537032 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f88380c98e0 msgr2=0x7f88380c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.005+0000 7f884049d640 1 --2- 172.21.15.76:0/2881537032 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f88380c98e0 0x7f88380c9d90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f88280097a0 tx=0x7f882802b790 comp rx=0 tx=0).stop 2024-03-08T06:01:48.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.005+0000 7f884049d640 1 -- 172.21.15.76:0/2881537032 shutdown_connections 2024-03-08T06:01:48.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.005+0000 7f884049d640 1 --2- 172.21.15.76:0/2881537032 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f88380c98e0 0x7f88380c9d90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.005+0000 7f884049d640 1 --2- 172.21.15.76:0/2881537032 >> v2:172.21.15.81:3300/0 conn(0x7f883815a9f0 0x7f8838071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.005+0000 7f884049d640 1 -- 172.21.15.76:0/2881537032 >> 172.21.15.76:0/2881537032 conn(0x7f88380cf740 msgr2=0x7f88380cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:48.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.005+0000 7f884049d640 1 -- 172.21.15.76:0/2881537032 shutdown_connections 2024-03-08T06:01:48.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.005+0000 7f884049d640 1 -- 172.21.15.76:0/2881537032 wait complete. 2024-03-08T06:01:48.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.006+0000 7f884049d640 1 Processor -- start 2024-03-08T06:01:48.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.006+0000 7f884049d640 1 -- start start 2024-03-08T06:01:48.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f884049d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88380c98e0 0x7f883815da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f884049d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88380ca2d0 0x7f88381f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f884049d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f883815a9f0 0x7f88380c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f884049d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883815af50 con 0x7f88380c98e0 2024-03-08T06:01:48.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f884049d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883815add0 con 0x7f88380ca2d0 2024-03-08T06:01:48.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f884049d640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88380cf330 con 0x7f883815a9f0 2024-03-08T06:01:48.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f883e239640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88380c98e0 0x7f883815da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f883e239640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88380c98e0 0x7f883815da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45646/0 (socket says 172.21.15.76:45646) 2024-03-08T06:01:48.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f883e239640 1 -- 172.21.15.76:0/2463593120 learned_addr learned my addr 172.21.15.76:0/2463593120 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:48.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f883ea3a640 1 --2- 172.21.15.76:0/2463593120 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f883815a9f0 0x7f88380c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f883da38640 1 --2- 172.21.15.76:0/2463593120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88380ca2d0 0x7f88381f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f883e239640 1 -- 172.21.15.76:0/2463593120 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f883815a9f0 msgr2=0x7f88380c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f883e239640 1 --2- 172.21.15.76:0/2463593120 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f883815a9f0 0x7f88380c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f883e239640 1 -- 172.21.15.76:0/2463593120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88380ca2d0 msgr2=0x7f88381f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f883e239640 1 --2- 172.21.15.76:0/2463593120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88380ca2d0 0x7f88381f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f883e239640 1 -- 172.21.15.76:0/2463593120 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883815df60 con 0x7f88380c98e0 2024-03-08T06:01:48.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.007+0000 7f883e239640 1 --2- 172.21.15.76:0/2463593120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88380c98e0 0x7f883815da20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f883000efc0 tx=0x7f883000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:48.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.008+0000 7f882effd640 1 -- 172.21.15.76:0/2463593120 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830018020 con 0x7f88380c98e0 2024-03-08T06:01:48.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.008+0000 7f882effd640 1 -- 172.21.15.76:0/2463593120 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8830014680 con 0x7f88380c98e0 2024-03-08T06:01:48.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.008+0000 7f884049d640 1 -- 172.21.15.76:0/2463593120 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f883814a670 con 0x7f88380c98e0 2024-03-08T06:01:48.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.008+0000 7f882effd640 1 -- 172.21.15.76:0/2463593120 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830014a50 con 0x7f88380c98e0 2024-03-08T06:01:48.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.008+0000 7f884049d640 1 -- 172.21.15.76:0/2463593120 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883815cf40 con 0x7f88380c98e0 2024-03-08T06:01:48.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.008+0000 7f884049d640 1 -- 172.21.15.76:0/2463593120 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88380ca090 con 0x7f88380c98e0 2024-03-08T06:01:48.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.009+0000 7f882effd640 1 -- 172.21.15.76:0/2463593120 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8830005d10 con 0x7f88380c98e0 2024-03-08T06:01:48.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.009+0000 7f882effd640 1 --2- 172.21.15.76:0/2463593120 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f881c06b4d0 0x7f881c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.009+0000 7f882effd640 1 -- 172.21.15.76:0/2463593120 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f883008b130 con 0x7f88380c98e0 2024-03-08T06:01:48.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.009+0000 7f883da38640 1 --2- 172.21.15.76:0/2463593120 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f881c06b4d0 0x7f881c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.010+0000 7f883da38640 1 --2- 172.21.15.76:0/2463593120 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f881c06b4d0 0x7f881c06d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8828034000 tx=0x7f882802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:48.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.014+0000 7f882effd640 1 -- 172.21.15.76:0/2463593120 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88380ca090 con 0x7f88380c98e0 2024-03-08T06:01:48.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.188+0000 7f884049d640 1 -- 172.21.15.76:0/2463593120 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88380ca950 con 0x7f88380c98e0 2024-03-08T06:01:48.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.189+0000 7f882effd640 1 -- 172.21.15.76:0/2463593120 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16871 (secure 0 0 0) 0x7f88380ca950 con 0x7f88380c98e0 2024-03-08T06:01:48.190 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:48.190 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":39,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:01:47.231864+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:01:45.628991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-08T06:01:46.195100+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:48.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.191+0000 7f884049d640 1 -- 172.21.15.76:0/2463593120 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f881c06b4d0 msgr2=0x7f881c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.191+0000 7f884049d640 1 --2- 172.21.15.76:0/2463593120 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f881c06b4d0 0x7f881c06d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8828034000 tx=0x7f882802f000 comp rx=0 tx=0).stop 2024-03-08T06:01:48.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.191+0000 7f884049d640 1 -- 172.21.15.76:0/2463593120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88380c98e0 msgr2=0x7f883815da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.191+0000 7f884049d640 1 --2- 172.21.15.76:0/2463593120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88380c98e0 0x7f883815da20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f883000efc0 tx=0x7f883000c490 comp rx=0 tx=0).stop 2024-03-08T06:01:48.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.191+0000 7f884049d640 1 -- 172.21.15.76:0/2463593120 shutdown_connections 2024-03-08T06:01:48.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.191+0000 7f884049d640 1 --2- 172.21.15.76:0/2463593120 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f881c06b4d0 0x7f881c06d990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.191+0000 7f884049d640 1 --2- 172.21.15.76:0/2463593120 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f883815a9f0 0x7f88380c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.191+0000 7f884049d640 1 --2- 172.21.15.76:0/2463593120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88380ca2d0 0x7f88381f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.191+0000 7f884049d640 1 --2- 172.21.15.76:0/2463593120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88380c98e0 0x7f883815da20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.191+0000 7f884049d640 1 -- 172.21.15.76:0/2463593120 >> 172.21.15.76:0/2463593120 conn(0x7f88380cf740 msgr2=0x7f883806dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:48.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.191+0000 7f884049d640 1 -- 172.21.15.76:0/2463593120 shutdown_connections 2024-03-08T06:01:48.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.191+0000 7f884049d640 1 -- 172.21.15.76:0/2463593120 wait complete. 2024-03-08T06:01:48.203 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.302+0000 7fbc84a2a640 1 Processor -- start 2024-03-08T06:01:48.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.302+0000 7fbc84a2a640 1 -- start start 2024-03-08T06:01:48.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc84a2a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbc8015a9f0 0x7fbc80071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc84a2a640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc800c98e0 0x7fbc800c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc84a2a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbc800ca2d0 0x7fbc800cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc84a2a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc800ce790 con 0x7fbc8015a9f0 2024-03-08T06:01:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc84a2a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc800722e0 con 0x7fbc800ca2d0 2024-03-08T06:01:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc84a2a640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc800ae8d0 con 0x7fbc800c98e0 2024-03-08T06:01:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc7e59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbc8015a9f0 0x7fbc80071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc7e59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbc8015a9f0 0x7fbc80071da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45654/0 (socket says 172.21.15.76:45654) 2024-03-08T06:01:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc7e59c640 1 -- 172.21.15.76:0/4039397028 learned_addr learned my addr 172.21.15.76:0/4039397028 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc7dd9b640 1 --2- 172.21.15.76:0/4039397028 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc800c98e0 0x7fbc800c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc7ed9d640 1 --2- 172.21.15.76:0/4039397028 >> v2:172.21.15.81:3300/0 conn(0x7fbc800ca2d0 0x7fbc800cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc7e59c640 1 -- 172.21.15.76:0/4039397028 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc800c98e0 msgr2=0x7fbc800c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc7e59c640 1 --2- 172.21.15.76:0/4039397028 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc800c98e0 0x7fbc800c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc7e59c640 1 -- 172.21.15.76:0/4039397028 >> v2:172.21.15.81:3300/0 conn(0x7fbc800ca2d0 msgr2=0x7fbc800cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc7e59c640 1 --2- 172.21.15.76:0/4039397028 >> v2:172.21.15.81:3300/0 conn(0x7fbc800ca2d0 0x7fbc800cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc7e59c640 1 -- 172.21.15.76:0/4039397028 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc800aebb0 con 0x7fbc8015a9f0 2024-03-08T06:01:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.303+0000 7fbc7dd9b640 1 --2- 172.21.15.76:0/4039397028 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc800c98e0 0x7fbc800c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.304+0000 7fbc7e59c640 1 --2- 172.21.15.76:0/4039397028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc8015a9f0 0x7fbc80071da0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbc80072d10 tx=0x7fbc7002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c57bd4fb773342 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.304+0000 7fbc7d59a640 1 -- 172.21.15.76:0/4039397028 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70035020 con 0x7fbc8015a9f0 2024-03-08T06:01:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.304+0000 7fbc7d59a640 1 -- 172.21.15.76:0/4039397028 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc70031870 con 0x7fbc8015a9f0 2024-03-08T06:01:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.304+0000 7fbc7d59a640 1 -- 172.21.15.76:0/4039397028 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70031c40 con 0x7fbc8015a9f0 2024-03-08T06:01:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.304+0000 7fbc84a2a640 1 -- 172.21.15.76:0/4039397028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc8015a9f0 msgr2=0x7fbc80071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.304+0000 7fbc84a2a640 1 --2- 172.21.15.76:0/4039397028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc8015a9f0 0x7fbc80071da0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbc80072d10 tx=0x7fbc7002b910 comp rx=0 tx=0).stop 2024-03-08T06:01:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.305+0000 7fbc84a2a640 1 -- 172.21.15.76:0/4039397028 shutdown_connections 2024-03-08T06:01:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.305+0000 7fbc84a2a640 1 --2- 172.21.15.76:0/4039397028 >> v2:172.21.15.81:3300/0 conn(0x7fbc800ca2d0 0x7fbc800cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.305+0000 7fbc84a2a640 1 --2- 172.21.15.76:0/4039397028 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc800c98e0 0x7fbc800c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.305+0000 7fbc84a2a640 1 --2- 172.21.15.76:0/4039397028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc8015a9f0 0x7fbc80071da0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.305+0000 7fbc84a2a640 1 -- 172.21.15.76:0/4039397028 >> 172.21.15.76:0/4039397028 conn(0x7fbc800cf740 msgr2=0x7fbc800cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.305+0000 7fbc84a2a640 1 -- 172.21.15.76:0/4039397028 shutdown_connections 2024-03-08T06:01:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.305+0000 7fbc84a2a640 1 -- 172.21.15.76:0/4039397028 wait complete. 2024-03-08T06:01:48.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.306+0000 7fbc84a2a640 1 Processor -- start 2024-03-08T06:01:48.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.306+0000 7fbc84a2a640 1 -- start start 2024-03-08T06:01:48.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc84a2a640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc800c98e0 0x7fbc800b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc84a2a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc800ca2d0 0x7fbc8015d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc84a2a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc8015a9f0 0x7fbc800cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc84a2a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8015af50 con 0x7fbc800ca2d0 2024-03-08T06:01:48.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc84a2a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8015add0 con 0x7fbc8015a9f0 2024-03-08T06:01:48.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc84a2a640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc800cf160 con 0x7fbc800c98e0 2024-03-08T06:01:48.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc7dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc800ca2d0 0x7fbc8015d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc7dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc800ca2d0 0x7fbc8015d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45656/0 (socket says 172.21.15.76:45656) 2024-03-08T06:01:48.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc7dd9b640 1 -- 172.21.15.76:0/1558889518 learned_addr learned my addr 172.21.15.76:0/1558889518 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:48.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc7ed9d640 1 --2- 172.21.15.76:0/1558889518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc8015a9f0 0x7fbc800cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc7e59c640 1 --2- 172.21.15.76:0/1558889518 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc800c98e0 0x7fbc800b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc7dd9b640 1 -- 172.21.15.76:0/1558889518 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc800c98e0 msgr2=0x7fbc800b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc7dd9b640 1 --2- 172.21.15.76:0/1558889518 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc800c98e0 0x7fbc800b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc7dd9b640 1 -- 172.21.15.76:0/1558889518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc8015a9f0 msgr2=0x7fbc800cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc7dd9b640 1 --2- 172.21.15.76:0/1558889518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc8015a9f0 0x7fbc800cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc7dd9b640 1 -- 172.21.15.76:0/1558889518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc800c8a90 con 0x7fbc800ca2d0 2024-03-08T06:01:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc7dd9b640 1 --2- 172.21.15.76:0/1558889518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc800ca2d0 0x7fbc8015d870 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400e7e0 tx=0x7fbc7400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc6effd640 1 -- 172.21.15.76:0/1558889518 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7400f040 con 0x7fbc800ca2d0 2024-03-08T06:01:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc6effd640 1 -- 172.21.15.76:0/1558889518 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc74004030 con 0x7fbc800ca2d0 2024-03-08T06:01:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc6effd640 1 -- 172.21.15.76:0/1558889518 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc740043e0 con 0x7fbc800ca2d0 2024-03-08T06:01:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc84a2a640 1 -- 172.21.15.76:0/1558889518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc800c8d30 con 0x7fbc800ca2d0 2024-03-08T06:01:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.307+0000 7fbc84a2a640 1 -- 172.21.15.76:0/1558889518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc800c92f0 con 0x7fbc800ca2d0 2024-03-08T06:01:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.308+0000 7fbc84a2a640 1 -- 172.21.15.76:0/1558889518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc800cab80 con 0x7fbc800ca2d0 2024-03-08T06:01:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.309+0000 7fbc6effd640 1 -- 172.21.15.76:0/1558889518 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc7400c6d0 con 0x7fbc800ca2d0 2024-03-08T06:01:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.309+0000 7fbc6effd640 1 --2- 172.21.15.76:0/1558889518 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbc6006b4d0 0x7fbc6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.309+0000 7fbc7e59c640 1 --2- 172.21.15.76:0/1558889518 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbc6006b4d0 0x7fbc6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.309+0000 7fbc6effd640 1 -- 172.21.15.76:0/1558889518 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc74089e80 con 0x7fbc800ca2d0 2024-03-08T06:01:48.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.310+0000 7fbc7e59c640 1 --2- 172.21.15.76:0/1558889518 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbc6006b4d0 0x7fbc6006d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbc70036000 tx=0x7fbc7003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:48.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.314+0000 7fbc6effd640 1 -- 172.21.15.76:0/1558889518 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc800cab80 con 0x7fbc800ca2d0 2024-03-08T06:01:48.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.488+0000 7fbc84a2a640 1 -- 172.21.15.76:0/1558889518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc800725a0 con 0x7fbc800ca2d0 2024-03-08T06:01:48.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.489+0000 7fbc6effd640 1 -- 172.21.15.76:0/1558889518 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16871 (secure 0 0 0) 0x7fbc74017020 con 0x7fbc800ca2d0 2024-03-08T06:01:48.490 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:48.490 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":39,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:01:47.231864+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:01:45.628991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-08T06:01:46.195100+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:48.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.491+0000 7fbc84a2a640 1 -- 172.21.15.76:0/1558889518 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbc6006b4d0 msgr2=0x7fbc6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.491+0000 7fbc84a2a640 1 --2- 172.21.15.76:0/1558889518 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbc6006b4d0 0x7fbc6006d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbc70036000 tx=0x7fbc7003c040 comp rx=0 tx=0).stop 2024-03-08T06:01:48.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.491+0000 7fbc84a2a640 1 -- 172.21.15.76:0/1558889518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc800ca2d0 msgr2=0x7fbc8015d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.491+0000 7fbc84a2a640 1 --2- 172.21.15.76:0/1558889518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc800ca2d0 0x7fbc8015d870 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400e7e0 tx=0x7fbc7400eca0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.491+0000 7fbc84a2a640 1 -- 172.21.15.76:0/1558889518 shutdown_connections 2024-03-08T06:01:48.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.491+0000 7fbc84a2a640 1 --2- 172.21.15.76:0/1558889518 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbc6006b4d0 0x7fbc6006d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.491+0000 7fbc84a2a640 1 --2- 172.21.15.76:0/1558889518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc8015a9f0 0x7fbc800cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.491+0000 7fbc84a2a640 1 --2- 172.21.15.76:0/1558889518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc800ca2d0 0x7fbc8015d870 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.491+0000 7fbc84a2a640 1 --2- 172.21.15.76:0/1558889518 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc800c98e0 0x7fbc800b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.492+0000 7fbc84a2a640 1 -- 172.21.15.76:0/1558889518 >> 172.21.15.76:0/1558889518 conn(0x7fbc800cf740 msgr2=0x7fbc800b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:48.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.492+0000 7fbc84a2a640 1 -- 172.21.15.76:0/1558889518 shutdown_connections 2024-03-08T06:01:48.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.492+0000 7fbc84a2a640 1 -- 172.21.15.76:0/1558889518 wait complete. 2024-03-08T06:01:48.503 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-08T06:01:48.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.606+0000 7f5c30e15640 1 Processor -- start 2024-03-08T06:01:48.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.606+0000 7f5c30e15640 1 -- start start 2024-03-08T06:01:48.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.606+0000 7f5c30e15640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5c2c15dbc0 0x7f5c2c0d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.606+0000 7f5c30e15640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c2c0b9840 0x7f5c2c0b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.606+0000 7f5c30e15640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c2c0d27d0 con 0x7f5c2c0b5b20 2024-03-08T06:01:48.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.606+0000 7f5c30e15640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c2c0b5f90 con 0x7f5c2c15dbc0 2024-03-08T06:01:48.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.606+0000 7f5c30e15640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0d2ab0 con 0x7f5c2c0b9840 2024-03-08T06:01:48.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.607+0000 7f5c23fff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c2c0b9840 0x7f5c2c0b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.607+0000 7f5c2a59c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5c2c15dbc0 0x7f5c2c0d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.607+0000 7f5c2ad9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5c2c0b5b20 0x7f5c2c06c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:45808/0 (socket says 172.21.15.76:45808) 2024-03-08T06:01:48.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.607+0000 7f5c23fff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c2c0b9840 0x7f5c2c0b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:35426/0 (socket says 172.21.15.76:35426) 2024-03-08T06:01:48.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.607+0000 7f5c2ad9d640 1 -- 172.21.15.76:0/2131611479 learned_addr learned my addr 172.21.15.76:0/2131611479 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:48.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.607+0000 7f5c2a59c640 1 -- 172.21.15.76:0/2131611479 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c2c0b9840 msgr2=0x7f5c2c0b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:48.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.607+0000 7f5c2a59c640 1 --2- 172.21.15.76:0/2131611479 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c2c0b9840 0x7f5c2c0b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.607+0000 7f5c2a59c640 1 -- 172.21.15.76:0/2131611479 >> v1:172.21.15.81:6789/0 conn(0x7f5c2c0b5b20 legacy=0x7f5c2c06c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.607+0000 7f5c2a59c640 1 -- 172.21.15.76:0/2131611479 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2c0d2d90 con 0x7f5c2c15dbc0 2024-03-08T06:01:48.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.608+0000 7f5c2a59c640 1 --2- 172.21.15.76:0/2131611479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2c15dbc0 0x7f5c2c0d1ef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0097a0 tx=0x7f5c1c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc7810c36b4cb81e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:48.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.608+0000 7f5c29d9b640 1 -- 172.21.15.76:0/2131611479 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c035020 con 0x7f5c2c15dbc0 2024-03-08T06:01:48.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.608+0000 7f5c29d9b640 1 -- 172.21.15.76:0/2131611479 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c031830 con 0x7f5c2c15dbc0 2024-03-08T06:01:48.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.608+0000 7f5c29d9b640 1 -- 172.21.15.76:0/2131611479 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c031c00 con 0x7f5c2c15dbc0 2024-03-08T06:01:48.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.609+0000 7f5c30e15640 1 -- 172.21.15.76:0/2131611479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2c15dbc0 msgr2=0x7f5c2c0d1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.609+0000 7f5c30e15640 1 --2- 172.21.15.76:0/2131611479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2c15dbc0 0x7f5c2c0d1ef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0097a0 tx=0x7f5c1c02b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.609+0000 7f5c30e15640 1 -- 172.21.15.76:0/2131611479 shutdown_connections 2024-03-08T06:01:48.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.609+0000 7f5c30e15640 1 --2- 172.21.15.76:0/2131611479 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c2c0b9840 0x7f5c2c0b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.609+0000 7f5c30e15640 1 --2- 172.21.15.76:0/2131611479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2c15dbc0 0x7f5c2c0d1ef0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.609+0000 7f5c30e15640 1 -- 172.21.15.76:0/2131611479 >> 172.21.15.76:0/2131611479 conn(0x7f5c2c0670d0 msgr2=0x7f5c2c0d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:48.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.609+0000 7f5c30e15640 1 -- 172.21.15.76:0/2131611479 shutdown_connections 2024-03-08T06:01:48.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.609+0000 7f5c30e15640 1 -- 172.21.15.76:0/2131611479 wait complete. 2024-03-08T06:01:48.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.610+0000 7f5c30e15640 1 Processor -- start 2024-03-08T06:01:48.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.610+0000 7f5c30e15640 1 -- start start 2024-03-08T06:01:48.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c30e15640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c2c0b5b20 0x7f5c2c14a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c30e15640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c2c0b9840 0x7f5c2c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c30e15640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2c15dbc0 0x7f5c2c1f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c30e15640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0d2430 con 0x7f5c2c0b5b20 2024-03-08T06:01:48.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c30e15640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0d3870 con 0x7f5c2c15dbc0 2024-03-08T06:01:48.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c30e15640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0d25b0 con 0x7f5c2c0b9840 2024-03-08T06:01:48.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c2a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c2c0b5b20 0x7f5c2c14a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c2a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c2c0b5b20 0x7f5c2c14a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51222/0 (socket says 172.21.15.76:51222) 2024-03-08T06:01:48.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c2a59c640 1 -- 172.21.15.76:0/1798224842 learned_addr learned my addr 172.21.15.76:0/1798224842 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:48.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c23fff640 1 --2- 172.21.15.76:0/1798224842 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c2c0b9840 0x7f5c2c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c2ad9d640 1 --2- 172.21.15.76:0/1798224842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2c15dbc0 0x7f5c2c1f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c2a59c640 1 -- 172.21.15.76:0/1798224842 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c2c0b9840 msgr2=0x7f5c2c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c2a59c640 1 --2- 172.21.15.76:0/1798224842 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c2c0b9840 0x7f5c2c0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c2a59c640 1 -- 172.21.15.76:0/1798224842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2c15dbc0 msgr2=0x7f5c2c1f6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c2a59c640 1 --2- 172.21.15.76:0/1798224842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2c15dbc0 0x7f5c2c1f6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c2a59c640 1 -- 172.21.15.76:0/1798224842 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2c159750 con 0x7f5c2c0b5b20 2024-03-08T06:01:48.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c2a59c640 1 --2- 172.21.15.76:0/1798224842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c2c0b5b20 0x7f5c2c14a210 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c02b910 tx=0x7f5c1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:48.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c22ffd640 1 -- 172.21.15.76:0/1798224842 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c040070 con 0x7f5c2c0b5b20 2024-03-08T06:01:48.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c22ffd640 1 -- 172.21.15.76:0/1798224842 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c03a930 con 0x7f5c2c0b5b20 2024-03-08T06:01:48.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.611+0000 7f5c30e15640 1 -- 172.21.15.76:0/1798224842 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2c158bd0 con 0x7f5c2c0b5b20 2024-03-08T06:01:48.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.612+0000 7f5c22ffd640 1 -- 172.21.15.76:0/1798224842 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c035040 con 0x7f5c2c0b5b20 2024-03-08T06:01:48.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.612+0000 7f5c30e15640 1 -- 172.21.15.76:0/1798224842 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2c159190 con 0x7f5c2c0b5b20 2024-03-08T06:01:48.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.613+0000 7f5c30e15640 1 -- 172.21.15.76:0/1798224842 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c2c04f030 con 0x7f5c2c0b5b20 2024-03-08T06:01:48.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.613+0000 7f5c22ffd640 1 -- 172.21.15.76:0/1798224842 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c1c04b060 con 0x7f5c2c0b5b20 2024-03-08T06:01:48.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.613+0000 7f5c22ffd640 1 --2- 172.21.15.76:0/1798224842 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5c0c06b500 0x7f5c0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:48.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.613+0000 7f5c22ffd640 1 -- 172.21.15.76:0/1798224842 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5c1c0aba00 con 0x7f5c2c0b5b20 2024-03-08T06:01:48.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.613+0000 7f5c23fff640 1 --2- 172.21.15.76:0/1798224842 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5c0c06b500 0x7f5c0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:48.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.614+0000 7f5c23fff640 1 --2- 172.21.15.76:0/1798224842 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5c0c06b500 0x7f5c0c06d9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c15bfa0 tx=0x7f5c14009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:48.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.618+0000 7f5c22ffd640 1 -- 172.21.15.76:0/1798224842 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c1c031800 con 0x7f5c2c0b5b20 2024-03-08T06:01:48.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.824+0000 7f5c30e15640 1 -- 172.21.15.76:0/1798224842 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5c2c0c8550 con 0x7f5c2c0b5b20 2024-03-08T06:01:48.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.892+0000 7f5c22ffd640 1 -- 172.21.15.76:0/1798224842 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5c1c07a0b0 con 0x7f5c2c0b5b20 2024-03-08T06:01:48.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.894+0000 7f5c30e15640 1 -- 172.21.15.76:0/1798224842 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5c0c06b500 msgr2=0x7f5c0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.894+0000 7f5c30e15640 1 --2- 172.21.15.76:0/1798224842 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5c0c06b500 0x7f5c0c06d9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c15bfa0 tx=0x7f5c14009190 comp rx=0 tx=0).stop 2024-03-08T06:01:48.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.894+0000 7f5c30e15640 1 -- 172.21.15.76:0/1798224842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c2c0b5b20 msgr2=0x7f5c2c14a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:48.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.894+0000 7f5c30e15640 1 --2- 172.21.15.76:0/1798224842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c2c0b5b20 0x7f5c2c14a210 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c02b910 tx=0x7f5c1c0046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.894+0000 7f5c30e15640 1 -- 172.21.15.76:0/1798224842 shutdown_connections 2024-03-08T06:01:48.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.894+0000 7f5c30e15640 1 --2- 172.21.15.76:0/1798224842 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5c0c06b500 0x7f5c0c06d9c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.894+0000 7f5c30e15640 1 --2- 172.21.15.76:0/1798224842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2c15dbc0 0x7f5c2c1f6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.894+0000 7f5c30e15640 1 --2- 172.21.15.76:0/1798224842 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c2c0b9840 0x7f5c2c0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.894+0000 7f5c30e15640 1 --2- 172.21.15.76:0/1798224842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c2c0b5b20 0x7f5c2c14a210 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:48.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.894+0000 7f5c30e15640 1 -- 172.21.15.76:0/1798224842 >> 172.21.15.76:0/1798224842 conn(0x7f5c2c0670d0 msgr2=0x7f5c2c0cde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:48.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.894+0000 7f5c30e15640 1 -- 172.21.15.76:0/1798224842 shutdown_connections 2024-03-08T06:01:48.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:48.894+0000 7f5c30e15640 1 -- 172.21.15.76:0/1798224842 wait complete. 2024-03-08T06:01:48.906 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-08T06:01:48.907 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-08T06:01:49.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.009+0000 7f778570d640 1 Processor -- start 2024-03-08T06:01:49.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.009+0000 7f778570d640 1 -- start start 2024-03-08T06:01:49.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.010+0000 7f778570d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f778006c7a0 0x7f7780099bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:49.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.010+0000 7f778570d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f778006bfa0 0x7f77800cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:49.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.010+0000 7f778570d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778009a400 con 0x7f778015e0c0 2024-03-08T06:01:49.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.010+0000 7f778570d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f778009a9c0 con 0x7f778006c7a0 2024-03-08T06:01:49.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.010+0000 7f778570d640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77800cc4c0 con 0x7f778006bfa0 2024-03-08T06:01:49.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.010+0000 7f777effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f778006c7a0 0x7f7780099bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:49.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.010+0000 7f777effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f778006c7a0 0x7f7780099bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51236/0 (socket says 172.21.15.76:51236) 2024-03-08T06:01:49.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.010+0000 7f777effd640 1 -- 172.21.15.76:0/3462545873 learned_addr learned my addr 172.21.15.76:0/3462545873 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:49.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.010+0000 7f777e7fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f778015e0c0 0x7f778006cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39228/0 (socket says 172.21.15.76:39228) 2024-03-08T06:01:49.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.011+0000 7f777f7fe640 1 --2- 172.21.15.76:0/3462545873 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f778006bfa0 0x7f77800cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:49.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.011+0000 7f777effd640 1 -- 172.21.15.76:0/3462545873 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f778006bfa0 msgr2=0x7f77800cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:49.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.011+0000 7f777effd640 1 --2- 172.21.15.76:0/3462545873 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f778006bfa0 0x7f77800cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:49.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.011+0000 7f777effd640 1 -- 172.21.15.76:0/3462545873 >> v1:172.21.15.76:6789/0 conn(0x7f778015e0c0 legacy=0x7f778006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:49.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.011+0000 7f777dffb640 1 -- 172.21.15.76:0/3462545873 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802271088 0 0) 0x7f778009a400 con 0x7f778015e0c0 2024-03-08T06:01:49.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.011+0000 7f777effd640 1 -- 172.21.15.76:0/3462545873 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778009a6e0 con 0x7f778006c7a0 2024-03-08T06:01:49.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.011+0000 7f777effd640 1 --2- 172.21.15.76:0/3462545873 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f778006c7a0 0x7f7780099bb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7770009870 tx=0x7f777002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34029ae35e03e8e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:49.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.012+0000 7f777dffb640 1 -- 172.21.15.76:0/3462545873 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77700317c0 con 0x7f778006c7a0 2024-03-08T06:01:49.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.012+0000 7f777dffb640 1 -- 172.21.15.76:0/3462545873 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7770037070 con 0x7f778006c7a0 2024-03-08T06:01:49.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.012+0000 7f777dffb640 1 -- 172.21.15.76:0/3462545873 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770031b80 con 0x7f778006c7a0 2024-03-08T06:01:49.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.012+0000 7f778570d640 1 -- 172.21.15.76:0/3462545873 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f778006c7a0 msgr2=0x7f7780099bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:49.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.012+0000 7f778570d640 1 --2- 172.21.15.76:0/3462545873 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f778006c7a0 0x7f7780099bb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7770009870 tx=0x7f777002b860 comp rx=0 tx=0).stop 2024-03-08T06:01:49.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.012+0000 7f778570d640 1 -- 172.21.15.76:0/3462545873 shutdown_connections 2024-03-08T06:01:49.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.012+0000 7f778570d640 1 --2- 172.21.15.76:0/3462545873 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f778006bfa0 0x7f77800cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:49.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.012+0000 7f778570d640 1 --2- 172.21.15.76:0/3462545873 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f778006c7a0 0x7f7780099bb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:49.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.012+0000 7f778570d640 1 -- 172.21.15.76:0/3462545873 >> 172.21.15.76:0/3462545873 conn(0x7f77800b34c0 msgr2=0x7f77800b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:49.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.012+0000 7f778570d640 1 -- 172.21.15.76:0/3462545873 shutdown_connections 2024-03-08T06:01:49.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.012+0000 7f778570d640 1 -- 172.21.15.76:0/3462545873 wait complete. 2024-03-08T06:01:49.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.013+0000 7f778570d640 1 Processor -- start 2024-03-08T06:01:49.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.014+0000 7f778570d640 1 -- start start 2024-03-08T06:01:49.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.014+0000 7f778570d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f778006bfa0 0x7f77801f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:49.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.014+0000 7f778570d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f778006c7a0 0x7f77801f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:49.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.014+0000 7f778570d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77800b4820 0x7f778015e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:49.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.014+0000 7f778570d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77800cce90 con 0x7f77800b4820 2024-03-08T06:01:49.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.014+0000 7f778570d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778009a0f0 con 0x7f778006bfa0 2024-03-08T06:01:49.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.014+0000 7f778570d640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77800cd010 con 0x7f778006c7a0 2024-03-08T06:01:49.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.014+0000 7f777f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77800b4820 0x7f778015e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:49.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.014+0000 7f777f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77800b4820 0x7f778015e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51240/0 (socket says 172.21.15.76:51240) 2024-03-08T06:01:49.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.014+0000 7f777f7fe640 1 -- 172.21.15.76:0/1538525231 learned_addr learned my addr 172.21.15.76:0/1538525231 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:49.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.015+0000 7f777effd640 1 --2- 172.21.15.76:0/1538525231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f778006bfa0 0x7f77801f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:49.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.015+0000 7f777e7fc640 1 --2- 172.21.15.76:0/1538525231 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f778006c7a0 0x7f77801f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:49.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.015+0000 7f777f7fe640 1 -- 172.21.15.76:0/1538525231 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f778006c7a0 msgr2=0x7f77801f2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:49.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.015+0000 7f777f7fe640 1 --2- 172.21.15.76:0/1538525231 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f778006c7a0 0x7f77801f2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:49.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.015+0000 7f777f7fe640 1 -- 172.21.15.76:0/1538525231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f778006bfa0 msgr2=0x7f77801f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:49.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.015+0000 7f777f7fe640 1 --2- 172.21.15.76:0/1538525231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f778006bfa0 0x7f77801f3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:49.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.015+0000 7f777f7fe640 1 -- 172.21.15.76:0/1538525231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778015f930 con 0x7f77800b4820 2024-03-08T06:01:49.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.015+0000 7f777e7fc640 1 --2- 172.21.15.76:0/1538525231 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f778006c7a0 0x7f77801f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:49.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.015+0000 7f777f7fe640 1 --2- 172.21.15.76:0/1538525231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77800b4820 0x7f778015e4b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7778007e70 tx=0x7f777800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:49.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.015+0000 7f77677fe640 1 -- 172.21.15.76:0/1538525231 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7778014620 con 0x7f77800b4820 2024-03-08T06:01:49.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.015+0000 7f77677fe640 1 -- 172.21.15.76:0/1538525231 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7778019070 con 0x7f77800b4820 2024-03-08T06:01:49.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.015+0000 7f778570d640 1 -- 172.21.15.76:0/1538525231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77800b4d00 con 0x7f77800b4820 2024-03-08T06:01:49.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.015+0000 7f77677fe640 1 -- 172.21.15.76:0/1538525231 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7778014a20 con 0x7f77800b4820 2024-03-08T06:01:49.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.015+0000 7f778570d640 1 -- 172.21.15.76:0/1538525231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77800b52c0 con 0x7f77800b4820 2024-03-08T06:01:49.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.016+0000 7f778570d640 1 -- 172.21.15.76:0/1538525231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f778004f030 con 0x7f77800b4820 2024-03-08T06:01:49.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.016+0000 7f77677fe640 1 -- 172.21.15.76:0/1538525231 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7778014bc0 con 0x7f77800b4820 2024-03-08T06:01:49.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.017+0000 7f77677fe640 1 --2- 172.21.15.76:0/1538525231 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f776c06b4d0 0x7f776c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:49.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.017+0000 7f77677fe640 1 -- 172.21.15.76:0/1538525231 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f777808abb0 con 0x7f77800b4820 2024-03-08T06:01:49.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.017+0000 7f777effd640 1 --2- 172.21.15.76:0/1538525231 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f776c06b4d0 0x7f776c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:49.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.017+0000 7f777effd640 1 --2- 172.21.15.76:0/1538525231 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f776c06b4d0 0x7f776c06d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7770036000 tx=0x7f777003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:49.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.021+0000 7f77677fe640 1 -- 172.21.15.76:0/1538525231 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7778059930 con 0x7f77800b4820 2024-03-08T06:01:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.225+0000 7f778570d640 1 -- 172.21.15.76:0/1538525231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f778006e5d0 con 0x7f77800b4820 2024-03-08T06:01:49.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.901+0000 7f77677fe640 1 -- 172.21.15.76:0/1538525231 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7778059230 con 0x7f77800b4820 2024-03-08T06:01:49.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.903+0000 7f778570d640 1 -- 172.21.15.76:0/1538525231 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f776c06b4d0 msgr2=0x7f776c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:49.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.903+0000 7f778570d640 1 --2- 172.21.15.76:0/1538525231 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f776c06b4d0 0x7f776c06d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7770036000 tx=0x7f777003d040 comp rx=0 tx=0).stop 2024-03-08T06:01:49.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.904+0000 7f778570d640 1 -- 172.21.15.76:0/1538525231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77800b4820 msgr2=0x7f778015e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:49.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.904+0000 7f778570d640 1 --2- 172.21.15.76:0/1538525231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77800b4820 0x7f778015e4b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7778007e70 tx=0x7f777800e4b0 comp rx=0 tx=0).stop 2024-03-08T06:01:49.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.904+0000 7f778570d640 1 -- 172.21.15.76:0/1538525231 shutdown_connections 2024-03-08T06:01:49.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.904+0000 7f778570d640 1 --2- 172.21.15.76:0/1538525231 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f776c06b4d0 0x7f776c06d990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:49.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.904+0000 7f778570d640 1 --2- 172.21.15.76:0/1538525231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77800b4820 0x7f778015e4b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:49.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.904+0000 7f778570d640 1 --2- 172.21.15.76:0/1538525231 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f778006c7a0 0x7f77801f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:49.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.904+0000 7f778570d640 1 --2- 172.21.15.76:0/1538525231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f778006bfa0 0x7f77801f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:49.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.904+0000 7f778570d640 1 -- 172.21.15.76:0/1538525231 >> 172.21.15.76:0/1538525231 conn(0x7f77800b34c0 msgr2=0x7f77800b38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:49.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.904+0000 7f778570d640 1 -- 172.21.15.76:0/1538525231 shutdown_connections 2024-03-08T06:01:49.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:49.904+0000 7f778570d640 1 -- 172.21.15.76:0/1538525231 wait complete. 2024-03-08T06:01:49.917 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-08T06:01:49.917 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:01:50.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.023+0000 7f8b8fd56640 1 Processor -- start 2024-03-08T06:01:50.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.023+0000 7f8b8fd56640 1 -- start start 2024-03-08T06:01:50.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.024+0000 7f8b8fd56640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8b880cec20 0x7f8b880cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.024+0000 7f8b8fd56640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b880ae8d0 con 0x7f8b8815a9f0 2024-03-08T06:01:50.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.024+0000 7f8b8fd56640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b880aebb0 con 0x7f8b880c98e0 2024-03-08T06:01:50.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.024+0000 7f8b8fd56640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b880ce790 con 0x7f8b880cec20 2024-03-08T06:01:50.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.024+0000 7f8b8e2f3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8b880cec20 0x7f8b880cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.024+0000 7f8b8daf2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8b8815a9f0 0x7f8b88071da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39244/0 (socket says 172.21.15.76:39244) 2024-03-08T06:01:50.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.024+0000 7f8b8daf2640 1 -- 172.21.15.76:0/2994162732 learned_addr learned my addr 172.21.15.76:0/2994162732 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:50.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.025+0000 7f8b8e2f3640 1 -- 172.21.15.76:0/2994162732 >> v1:172.21.15.81:6789/0 conn(0x7f8b880c98e0 legacy=0x7f8b880c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:50.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.025+0000 7f8b8e2f3640 1 -- 172.21.15.76:0/2994162732 >> v1:172.21.15.76:6789/0 conn(0x7f8b8815a9f0 legacy=0x7f8b88071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.025+0000 7f8b8caf0640 1 -- 172.21.15.76:0/2994162732 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2066540955 0 0) 0x7f8b880ae8d0 con 0x7f8b8815a9f0 2024-03-08T06:01:50.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.025+0000 7f8b8e2f3640 1 -- 172.21.15.76:0/2994162732 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b880aee90 con 0x7f8b880cec20 2024-03-08T06:01:50.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.025+0000 7f8b8e2f3640 1 --2- 172.21.15.76:0/2994162732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b880cec20 0x7f8b880cf070 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8b800097a0 tx=0x7f8b8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b77adbd4ea2bad server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:50.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.026+0000 7f8b8caf0640 1 -- 172.21.15.76:0/2994162732 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80035020 con 0x7f8b880cec20 2024-03-08T06:01:50.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.026+0000 7f8b8caf0640 1 -- 172.21.15.76:0/2994162732 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b80031830 con 0x7f8b880cec20 2024-03-08T06:01:50.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.026+0000 7f8b8caf0640 1 -- 172.21.15.76:0/2994162732 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80031c00 con 0x7f8b880cec20 2024-03-08T06:01:50.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.026+0000 7f8b8fd56640 1 -- 172.21.15.76:0/2994162732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b880cec20 msgr2=0x7f8b880cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.026+0000 7f8b8fd56640 1 --2- 172.21.15.76:0/2994162732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b880cec20 0x7f8b880cf070 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8b800097a0 tx=0x7f8b8002b8d0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.026+0000 7f8b8fd56640 1 -- 172.21.15.76:0/2994162732 shutdown_connections 2024-03-08T06:01:50.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.026+0000 7f8b8fd56640 1 --2- 172.21.15.76:0/2994162732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b880cec20 0x7f8b880cf070 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.026+0000 7f8b8fd56640 1 -- 172.21.15.76:0/2994162732 >> 172.21.15.76:0/2994162732 conn(0x7f8b880cf740 msgr2=0x7f8b880cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:50.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.026+0000 7f8b8fd56640 1 -- 172.21.15.76:0/2994162732 shutdown_connections 2024-03-08T06:01:50.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.026+0000 7f8b8fd56640 1 -- 172.21.15.76:0/2994162732 wait complete. 2024-03-08T06:01:50.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.027+0000 7f8b8fd56640 1 Processor -- start 2024-03-08T06:01:50.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.027+0000 7f8b8fd56640 1 -- start start 2024-03-08T06:01:50.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.028+0000 7f8b8fd56640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b880c98e0 0x7f8b8814ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.028+0000 7f8b8fd56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b880cec20 0x7f8b8814a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.028+0000 7f8b8fd56640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8b8815a9f0 0x7f8b880cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.028+0000 7f8b8fd56640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880ca620 con 0x7f8b880cec20 2024-03-08T06:01:50.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.028+0000 7f8b8fd56640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880ca4a0 con 0x7f8b880c98e0 2024-03-08T06:01:50.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.028+0000 7f8b8fd56640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8815add0 con 0x7f8b8815a9f0 2024-03-08T06:01:50.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.028+0000 7f8b8d2f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b880cec20 0x7f8b8814a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.028+0000 7f8b8d2f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b880cec20 0x7f8b8814a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51264/0 (socket says 172.21.15.76:51264) 2024-03-08T06:01:50.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.028+0000 7f8b8d2f1640 1 -- 172.21.15.76:0/3854302871 learned_addr learned my addr 172.21.15.76:0/3854302871 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:50.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.028+0000 7f8b8d2f1640 1 -- 172.21.15.76:0/3854302871 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8b8815a9f0 msgr2=0x7f8b880cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.028+0000 7f8b8d2f1640 1 --2- 172.21.15.76:0/3854302871 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8b8815a9f0 0x7f8b880cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.028+0000 7f8b8d2f1640 1 -- 172.21.15.76:0/3854302871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b880c98e0 msgr2=0x7f8b8814ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.028+0000 7f8b8daf2640 1 --2- 172.21.15.76:0/3854302871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b880c98e0 0x7f8b8814ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.029+0000 7f8b8d2f1640 1 --2- 172.21.15.76:0/3854302871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b880c98e0 0x7f8b8814ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.029+0000 7f8b8d2f1640 1 -- 172.21.15.76:0/3854302871 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b88156800 con 0x7f8b880cec20 2024-03-08T06:01:50.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.029+0000 7f8b8d2f1640 1 --2- 172.21.15.76:0/3854302871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b880cec20 0x7f8b8814a070 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800c400 tx=0x7f8b7800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:50.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.029+0000 7f8b7e7fc640 1 -- 172.21.15.76:0/3854302871 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78017070 con 0x7f8b880cec20 2024-03-08T06:01:50.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.029+0000 7f8b7e7fc640 1 -- 172.21.15.76:0/3854302871 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b780035f0 con 0x7f8b880cec20 2024-03-08T06:01:50.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.029+0000 7f8b8fd56640 1 -- 172.21.15.76:0/3854302871 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b88156ae0 con 0x7f8b880cec20 2024-03-08T06:01:50.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.029+0000 7f8b7e7fc640 1 -- 172.21.15.76:0/3854302871 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78003a20 con 0x7f8b880cec20 2024-03-08T06:01:50.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.029+0000 7f8b8fd56640 1 -- 172.21.15.76:0/3854302871 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b881570a0 con 0x7f8b880cec20 2024-03-08T06:01:50.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.030+0000 7f8b8fd56640 1 -- 172.21.15.76:0/3854302871 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8804f030 con 0x7f8b880cec20 2024-03-08T06:01:50.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.030+0000 7f8b7e7fc640 1 -- 172.21.15.76:0/3854302871 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b7801a070 con 0x7f8b880cec20 2024-03-08T06:01:50.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.031+0000 7f8b7e7fc640 1 --2- 172.21.15.76:0/3854302871 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8b6c06b500 0x7f8b6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.031+0000 7f8b7e7fc640 1 -- 172.21.15.76:0/3854302871 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8b78089690 con 0x7f8b880cec20 2024-03-08T06:01:50.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.031+0000 7f8b8daf2640 1 --2- 172.21.15.76:0/3854302871 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8b6c06b500 0x7f8b6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.031+0000 7f8b8daf2640 1 --2- 172.21.15.76:0/3854302871 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8b6c06b500 0x7f8b6c06d9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8b840012e0 tx=0x7f8b84009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:50.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.035+0000 7f8b7e7fc640 1 -- 172.21.15.76:0/3854302871 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b78057d30 con 0x7f8b880cec20 2024-03-08T06:01:50.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.240+0000 7f8b8fd56640 1 -- 172.21.15.76:0/3854302871 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b880725a0 con 0x7f8b880cec20 2024-03-08T06:01:50.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.241+0000 7f8b7e7fc640 1 -- 172.21.15.76:0/3854302871 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f8b7805bdd0 con 0x7f8b880cec20 2024-03-08T06:01:50.242 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:50.242 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/606148582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":606148582},{"type":"v1","addr":"172.21.15.81:6835","nonce":606148582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/278988477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":278988477},{"type":"v1","addr":"172.21.15.76:6836","nonce":278988477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2526423312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":2526423312},{"type":"v1","addr":"172.21.15.76:6837","nonce":2526423312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:01:47.232184+0000","modified":"2024-03-08T06:01:49.227002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5358":{"gid":5358,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6834/1503895376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1503895376},{"type":"v1","addr":"172.21.15.81:6834","nonce":1503895376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T06:01:50.242 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 15 2024-03-08T06:01:50.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.243+0000 7f8b8fd56640 1 -- 172.21.15.76:0/3854302871 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8b6c06b500 msgr2=0x7f8b6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.243+0000 7f8b8fd56640 1 --2- 172.21.15.76:0/3854302871 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8b6c06b500 0x7f8b6c06d9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8b840012e0 tx=0x7f8b84009040 comp rx=0 tx=0).stop 2024-03-08T06:01:50.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.243+0000 7f8b8fd56640 1 -- 172.21.15.76:0/3854302871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b880cec20 msgr2=0x7f8b8814a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.243+0000 7f8b8fd56640 1 --2- 172.21.15.76:0/3854302871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b880cec20 0x7f8b8814a070 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800c400 tx=0x7f8b7800c8c0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.243+0000 7f8b8fd56640 1 -- 172.21.15.76:0/3854302871 shutdown_connections 2024-03-08T06:01:50.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.243+0000 7f8b8fd56640 1 --2- 172.21.15.76:0/3854302871 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f8b6c06b500 0x7f8b6c06d9c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.243+0000 7f8b8fd56640 1 --2- 172.21.15.76:0/3854302871 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8b8815a9f0 0x7f8b880cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.243+0000 7f8b8fd56640 1 --2- 172.21.15.76:0/3854302871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b880cec20 0x7f8b8814a070 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.243+0000 7f8b8fd56640 1 --2- 172.21.15.76:0/3854302871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b880c98e0 0x7f8b8814ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.243+0000 7f8b8fd56640 1 -- 172.21.15.76:0/3854302871 >> 172.21.15.76:0/3854302871 conn(0x7f8b880cf740 msgr2=0x7f8b88155d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:50.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.243+0000 7f8b8fd56640 1 -- 172.21.15.76:0/3854302871 shutdown_connections 2024-03-08T06:01:50.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.243+0000 7f8b8fd56640 1 -- 172.21.15.76:0/3854302871 wait complete. 2024-03-08T06:01:50.254 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:01:50.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.353+0000 7fc78d2a7640 1 Processor -- start 2024-03-08T06:01:50.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.353+0000 7fc78d2a7640 1 -- start start 2024-03-08T06:01:50.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.354+0000 7fc78d2a7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc78815dfc0 0x7fc78815e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.354+0000 7fc78d2a7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc78815e8e0 0x7fc78806c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.354+0000 7fc78d2a7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc78815ef30 con 0x7fc78806c750 2024-03-08T06:01:50.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.354+0000 7fc78d2a7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7880b6c10 con 0x7fc78815e8e0 2024-03-08T06:01:50.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.354+0000 7fc78d2a7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78815f210 con 0x7fc78815dfc0 2024-03-08T06:01:50.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.354+0000 7fc7867fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc78815e8e0 0x7fc78806c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.354+0000 7fc7877fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc78806c750 0x7fc7880b5630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39246/0 (socket says 172.21.15.76:39246) 2024-03-08T06:01:50.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.354+0000 7fc7877fe640 1 -- 172.21.15.76:0/1468615422 learned_addr learned my addr 172.21.15.76:0/1468615422 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:50.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.354+0000 7fc786ffd640 1 --2- 172.21.15.76:0/1468615422 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc78815dfc0 0x7fc78815e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.354+0000 7fc7867fc640 1 -- 172.21.15.76:0/1468615422 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc78815dfc0 msgr2=0x7fc78815e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:50.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.354+0000 7fc7867fc640 1 --2- 172.21.15.76:0/1468615422 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc78815dfc0 0x7fc78815e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.354+0000 7fc7867fc640 1 -- 172.21.15.76:0/1468615422 >> v1:172.21.15.76:6789/0 conn(0x7fc78806c750 legacy=0x7fc7880b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.354+0000 7fc785ffb640 1 -- 172.21.15.76:0/1468615422 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725786036 0 0) 0x7fc78815ef30 con 0x7fc78806c750 2024-03-08T06:01:50.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.354+0000 7fc7867fc640 1 -- 172.21.15.76:0/1468615422 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7880d1f00 con 0x7fc78815e8e0 2024-03-08T06:01:50.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.355+0000 7fc7867fc640 1 --2- 172.21.15.76:0/1468615422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78815e8e0 0x7fc78806c210 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009790 tx=0x7fc77c02b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8624b4f78f394b2f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:50.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.355+0000 7fc785ffb640 1 -- 172.21.15.76:0/1468615422 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c035020 con 0x7fc78815e8e0 2024-03-08T06:01:50.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.355+0000 7fc785ffb640 1 -- 172.21.15.76:0/1468615422 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c031890 con 0x7fc78815e8e0 2024-03-08T06:01:50.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.355+0000 7fc785ffb640 1 -- 172.21.15.76:0/1468615422 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c031c60 con 0x7fc78815e8e0 2024-03-08T06:01:50.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.356+0000 7fc78d2a7640 1 -- 172.21.15.76:0/1468615422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78815e8e0 msgr2=0x7fc78806c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.356+0000 7fc78d2a7640 1 --2- 172.21.15.76:0/1468615422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78815e8e0 0x7fc78806c210 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009790 tx=0x7fc77c02b930 comp rx=0 tx=0).stop 2024-03-08T06:01:50.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.356+0000 7fc78d2a7640 1 -- 172.21.15.76:0/1468615422 shutdown_connections 2024-03-08T06:01:50.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.356+0000 7fc78d2a7640 1 --2- 172.21.15.76:0/1468615422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78815e8e0 0x7fc78806c210 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.356+0000 7fc78d2a7640 1 --2- 172.21.15.76:0/1468615422 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc78815dfc0 0x7fc78815e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.356+0000 7fc78d2a7640 1 -- 172.21.15.76:0/1468615422 >> 172.21.15.76:0/1468615422 conn(0x7fc788066de0 msgr2=0x7fc7880671f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:50.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.356+0000 7fc78d2a7640 1 -- 172.21.15.76:0/1468615422 shutdown_connections 2024-03-08T06:01:50.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.356+0000 7fc78d2a7640 1 -- 172.21.15.76:0/1468615422 wait complete. 2024-03-08T06:01:50.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.357+0000 7fc78d2a7640 1 Processor -- start 2024-03-08T06:01:50.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.357+0000 7fc78d2a7640 1 -- start start 2024-03-08T06:01:50.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.358+0000 7fc78d2a7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc78806c750 0x7fc7881ed990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.358+0000 7fc78d2a7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78815dfc0 0x7fc7880cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.358+0000 7fc78d2a7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc78815e8e0 0x7fc7881f3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.358+0000 7fc78d2a7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7880d3710 con 0x7fc78815dfc0 2024-03-08T06:01:50.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.358+0000 7fc78d2a7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7880b9720 con 0x7fc78806c750 2024-03-08T06:01:50.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.358+0000 7fc78d2a7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7880d3890 con 0x7fc78815e8e0 2024-03-08T06:01:50.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.358+0000 7fc7867fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78815dfc0 0x7fc7880cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.358+0000 7fc7867fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78815dfc0 0x7fc7880cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51270/0 (socket says 172.21.15.76:51270) 2024-03-08T06:01:50.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.358+0000 7fc7867fc640 1 -- 172.21.15.76:0/1010340437 learned_addr learned my addr 172.21.15.76:0/1010340437 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:50.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.358+0000 7fc7867fc640 1 -- 172.21.15.76:0/1010340437 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc78815e8e0 msgr2=0x7fc7881f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.358+0000 7fc7867fc640 1 --2- 172.21.15.76:0/1010340437 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc78815e8e0 0x7fc7881f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.358+0000 7fc7867fc640 1 -- 172.21.15.76:0/1010340437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc78806c750 msgr2=0x7fc7881ed990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.358+0000 7fc7867fc640 1 --2- 172.21.15.76:0/1010340437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc78806c750 0x7fc7881ed990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.358+0000 7fc7867fc640 1 -- 172.21.15.76:0/1010340437 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7881f3e60 con 0x7fc78815dfc0 2024-03-08T06:01:50.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.359+0000 7fc7867fc640 1 --2- 172.21.15.76:0/1010340437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78815dfc0 0x7fc7880cd500 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0098b0 tx=0x7fc77c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:50.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.359+0000 7fc76f7fe640 1 -- 172.21.15.76:0/1010340437 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c035070 con 0x7fc78815dfc0 2024-03-08T06:01:50.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.359+0000 7fc76f7fe640 1 -- 172.21.15.76:0/1010340437 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c03bd70 con 0x7fc78815dfc0 2024-03-08T06:01:50.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.359+0000 7fc78d2a7640 1 -- 172.21.15.76:0/1010340437 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7881f40e0 con 0x7fc78815dfc0 2024-03-08T06:01:50.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.359+0000 7fc76f7fe640 1 -- 172.21.15.76:0/1010340437 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c004230 con 0x7fc78815dfc0 2024-03-08T06:01:50.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.359+0000 7fc78d2a7640 1 -- 172.21.15.76:0/1010340437 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7881f46a0 con 0x7fc78815dfc0 2024-03-08T06:01:50.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.360+0000 7fc78d2a7640 1 -- 172.21.15.76:0/1010340437 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7880b4640 con 0x7fc78815dfc0 2024-03-08T06:01:50.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.360+0000 7fc76f7fe640 1 -- 172.21.15.76:0/1010340437 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc77c038020 con 0x7fc78815dfc0 2024-03-08T06:01:50.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.360+0000 7fc76f7fe640 1 --2- 172.21.15.76:0/1010340437 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc77406b500 0x7fc77406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.361+0000 7fc76f7fe640 1 -- 172.21.15.76:0/1010340437 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc77c0ac6f0 con 0x7fc78815dfc0 2024-03-08T06:01:50.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.361+0000 7fc786ffd640 1 --2- 172.21.15.76:0/1010340437 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc77406b500 0x7fc77406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.361+0000 7fc786ffd640 1 --2- 172.21.15.76:0/1010340437 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc77406b500 0x7fc77406d9c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc7780070b0 tx=0x7fc778007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:50.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.365+0000 7fc76f7fe640 1 -- 172.21.15.76:0/1010340437 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7881f3e60 con 0x7fc78815dfc0 2024-03-08T06:01:50.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.536+0000 7fc78d2a7640 1 -- 172.21.15.76:0/1010340437 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc78804f030 con 0x7fc78815dfc0 2024-03-08T06:01:50.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.537+0000 7fc76f7fe640 1 -- 172.21.15.76:0/1010340437 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16858 (secure 0 0 0) 0x7fc77c040070 con 0x7fc78815dfc0 2024-03-08T06:01:50.538 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:50.538 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":40,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:01:48.875364+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:01:45.628991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-08T06:01:46.195100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:01:50.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.539+0000 7fc78d2a7640 1 -- 172.21.15.76:0/1010340437 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc77406b500 msgr2=0x7fc77406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.539+0000 7fc78d2a7640 1 --2- 172.21.15.76:0/1010340437 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc77406b500 0x7fc77406d9c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc7780070b0 tx=0x7fc778007040 comp rx=0 tx=0).stop 2024-03-08T06:01:50.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.539+0000 7fc78d2a7640 1 -- 172.21.15.76:0/1010340437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78815dfc0 msgr2=0x7fc7880cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.539+0000 7fc78d2a7640 1 --2- 172.21.15.76:0/1010340437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78815dfc0 0x7fc7880cd500 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0098b0 tx=0x7fc77c0046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.539+0000 7fc78d2a7640 1 -- 172.21.15.76:0/1010340437 shutdown_connections 2024-03-08T06:01:50.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.539+0000 7fc78d2a7640 1 --2- 172.21.15.76:0/1010340437 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc77406b500 0x7fc77406d9c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.539+0000 7fc78d2a7640 1 --2- 172.21.15.76:0/1010340437 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc78815e8e0 0x7fc7881f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.539+0000 7fc78d2a7640 1 --2- 172.21.15.76:0/1010340437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78815dfc0 0x7fc7880cd500 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.539+0000 7fc78d2a7640 1 --2- 172.21.15.76:0/1010340437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc78806c750 0x7fc7881ed990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.539+0000 7fc78d2a7640 1 -- 172.21.15.76:0/1010340437 >> 172.21.15.76:0/1010340437 conn(0x7fc788066de0 msgr2=0x7fc7880d43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:50.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.539+0000 7fc78d2a7640 1 -- 172.21.15.76:0/1010340437 shutdown_connections 2024-03-08T06:01:50.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.539+0000 7fc78d2a7640 1 -- 172.21.15.76:0/1010340437 wait complete. 2024-03-08T06:01:50.551 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-08T06:01:50.551 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:01:50.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.651+0000 7fd3c7a5c640 1 Processor -- start 2024-03-08T06:01:50.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.651+0000 7fd3c7a5c640 1 -- start start 2024-03-08T06:01:50.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.652+0000 7fd3c7a5c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd3c015bcd0 0x7fd3c00cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.652+0000 7fd3c7a5c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd3c009a250 0x7fd3c00cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.652+0000 7fd3c7a5c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c00ccbb0 con 0x7fd3c00cf730 2024-03-08T06:01:50.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.652+0000 7fd3c7a5c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c00cce90 con 0x7fd3c009a250 2024-03-08T06:01:50.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.652+0000 7fd3c7a5c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c00cd170 con 0x7fd3c015bcd0 2024-03-08T06:01:50.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.652+0000 7fd3c5ff9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd3c009a250 0x7fd3c00cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.652+0000 7fd3c4ff7640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd3c00cf730 0x7fd3c0099b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39250/0 (socket says 172.21.15.76:39250) 2024-03-08T06:01:50.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.652+0000 7fd3c5ff9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd3c009a250 0x7fd3c00cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51282/0 (socket says 172.21.15.76:51282) 2024-03-08T06:01:50.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.652+0000 7fd3c4ff7640 1 -- 172.21.15.76:0/3381003287 learned_addr learned my addr 172.21.15.76:0/3381003287 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:50.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.652+0000 7fd3c57f8640 1 --2- 172.21.15.76:0/3381003287 >> v2:172.21.15.81:3300/0 conn(0x7fd3c015bcd0 0x7fd3c00cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.652+0000 7fd3c5ff9640 1 -- 172.21.15.76:0/3381003287 >> v2:172.21.15.81:3300/0 conn(0x7fd3c015bcd0 msgr2=0x7fd3c00cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:50.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.652+0000 7fd3c5ff9640 1 --2- 172.21.15.76:0/3381003287 >> v2:172.21.15.81:3300/0 conn(0x7fd3c015bcd0 0x7fd3c00cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.652+0000 7fd3c5ff9640 1 -- 172.21.15.76:0/3381003287 >> v1:172.21.15.76:6789/0 conn(0x7fd3c00cf730 legacy=0x7fd3c0099b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.652+0000 7fd3c5ff9640 1 -- 172.21.15.76:0/3381003287 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c00cd450 con 0x7fd3c009a250 2024-03-08T06:01:50.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.652+0000 7fd3b7fff640 1 -- 172.21.15.76:0/3381003287 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547097812 0 0) 0x7fd3c00ccbb0 con 0x7fd3c00cf730 2024-03-08T06:01:50.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.653+0000 7fd3c5ff9640 1 --2- 172.21.15.76:0/3381003287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3c009a250 0x7fd3c00cdc50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0072cb0 tx=0x7fd3b802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c44c0851a9c1c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:50.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.653+0000 7fd3b7fff640 1 -- 172.21.15.76:0/3381003287 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8039070 con 0x7fd3c009a250 2024-03-08T06:01:50.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.653+0000 7fd3b7fff640 1 -- 172.21.15.76:0/3381003287 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b802cd00 con 0x7fd3c009a250 2024-03-08T06:01:50.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.653+0000 7fd3b7fff640 1 -- 172.21.15.76:0/3381003287 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8034660 con 0x7fd3c009a250 2024-03-08T06:01:50.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.654+0000 7fd3c7a5c640 1 -- 172.21.15.76:0/3381003287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3c009a250 msgr2=0x7fd3c00cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.654+0000 7fd3c7a5c640 1 --2- 172.21.15.76:0/3381003287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3c009a250 0x7fd3c00cdc50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0072cb0 tx=0x7fd3b802bf50 comp rx=0 tx=0).stop 2024-03-08T06:01:50.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.654+0000 7fd3c7a5c640 1 -- 172.21.15.76:0/3381003287 shutdown_connections 2024-03-08T06:01:50.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.654+0000 7fd3c7a5c640 1 --2- 172.21.15.76:0/3381003287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3c009a250 0x7fd3c00cdc50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.654+0000 7fd3c7a5c640 1 --2- 172.21.15.76:0/3381003287 >> v2:172.21.15.81:3300/0 conn(0x7fd3c015bcd0 0x7fd3c00cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.654+0000 7fd3c7a5c640 1 -- 172.21.15.76:0/3381003287 >> 172.21.15.76:0/3381003287 conn(0x7fd3c00b3440 msgr2=0x7fd3c00b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:50.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.654+0000 7fd3c7a5c640 1 -- 172.21.15.76:0/3381003287 shutdown_connections 2024-03-08T06:01:50.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.654+0000 7fd3c7a5c640 1 -- 172.21.15.76:0/3381003287 wait complete. 2024-03-08T06:01:50.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.655+0000 7fd3c7a5c640 1 Processor -- start 2024-03-08T06:01:50.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.655+0000 7fd3c7a5c640 1 -- start start 2024-03-08T06:01:50.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c7a5c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3c009a250 0x7fd3c015d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c7a5c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3c00cf730 0x7fd3c00c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c7a5c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd3c015bcd0 0x7fd3c00c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c7a5c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c009b160 con 0x7fd3c009a250 2024-03-08T06:01:50.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c7a5c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c015c0b0 con 0x7fd3c00cf730 2024-03-08T06:01:50.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c7a5c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c009b2e0 con 0x7fd3c015bcd0 2024-03-08T06:01:50.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c57f8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3c009a250 0x7fd3c015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c57f8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3c009a250 0x7fd3c015d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51284/0 (socket says 172.21.15.76:51284) 2024-03-08T06:01:50.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c57f8640 1 -- 172.21.15.76:0/1989463742 learned_addr learned my addr 172.21.15.76:0/1989463742 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:50.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c5ff9640 1 --2- 172.21.15.76:0/1989463742 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd3c015bcd0 0x7fd3c00c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c4ff7640 1 --2- 172.21.15.76:0/1989463742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3c00cf730 0x7fd3c00c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c57f8640 1 -- 172.21.15.76:0/1989463742 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd3c015bcd0 msgr2=0x7fd3c00c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c57f8640 1 --2- 172.21.15.76:0/1989463742 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd3c015bcd0 0x7fd3c00c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c57f8640 1 -- 172.21.15.76:0/1989463742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3c00cf730 msgr2=0x7fd3c00c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c57f8640 1 --2- 172.21.15.76:0/1989463742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3c00cf730 0x7fd3c00c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c57f8640 1 -- 172.21.15.76:0/1989463742 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c015d8f0 con 0x7fd3c009a250 2024-03-08T06:01:50.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c5ff9640 1 --2- 172.21.15.76:0/1989463742 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd3c015bcd0 0x7fd3c00c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:50.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c4ff7640 1 --2- 172.21.15.76:0/1989463742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3c00cf730 0x7fd3c00c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:50.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c57f8640 1 --2- 172.21.15.76:0/1989463742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3c009a250 0x7fd3c015d280 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00d800 tx=0x7fd3bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:50.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3b5ffb640 1 -- 172.21.15.76:0/1989463742 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc00b6c0 con 0x7fd3c009a250 2024-03-08T06:01:50.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3b5ffb640 1 -- 172.21.15.76:0/1989463742 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc019070 con 0x7fd3c009a250 2024-03-08T06:01:50.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3c7a5c640 1 -- 172.21.15.76:0/1989463742 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c015dbd0 con 0x7fd3c009a250 2024-03-08T06:01:50.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.656+0000 7fd3b5ffb640 1 -- 172.21.15.76:0/1989463742 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc00ba80 con 0x7fd3c009a250 2024-03-08T06:01:50.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.657+0000 7fd3c7a5c640 1 -- 172.21.15.76:0/1989463742 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c015e190 con 0x7fd3c009a250 2024-03-08T06:01:50.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.658+0000 7fd3b5ffb640 1 -- 172.21.15.76:0/1989463742 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd3bc00bc20 con 0x7fd3c009a250 2024-03-08T06:01:50.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.658+0000 7fd3c7a5c640 1 -- 172.21.15.76:0/1989463742 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c004f030 con 0x7fd3c009a250 2024-03-08T06:01:50.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.658+0000 7fd3b5ffb640 1 --2- 172.21.15.76:0/1989463742 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd3a406b500 0x7fd3a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.658+0000 7fd3b5ffb640 1 -- 172.21.15.76:0/1989463742 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd3bc08a7b0 con 0x7fd3c009a250 2024-03-08T06:01:50.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.658+0000 7fd3c4ff7640 1 --2- 172.21.15.76:0/1989463742 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd3a406b500 0x7fd3a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.658+0000 7fd3c4ff7640 1 --2- 172.21.15.76:0/1989463742 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd3a406b500 0x7fd3a406d9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00b2630 tx=0x7fd3b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:50.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.662+0000 7fd3b5ffb640 1 -- 172.21.15.76:0/1989463742 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd3bc059530 con 0x7fd3c009a250 2024-03-08T06:01:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.828+0000 7fd3c7a5c640 1 -- 172.21.15.76:0/1989463742 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd3c009a670 con 0x7fd3a406b500 2024-03-08T06:01:50.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.834+0000 7fd3b5ffb640 1 -- 172.21.15.76:0/1989463742 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251903 (secure 0 0 0) 0x7fd3c009a670 con 0x7fd3a406b500 2024-03-08T06:01:50.838 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:50.839 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-08T06:01:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.837+0000 7fd3c7a5c640 1 -- 172.21.15.76:0/1989463742 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd3a406b500 msgr2=0x7fd3a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.837+0000 7fd3c7a5c640 1 --2- 172.21.15.76:0/1989463742 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd3a406b500 0x7fd3a406d9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00b2630 tx=0x7fd3b0009040 comp rx=0 tx=0).stop 2024-03-08T06:01:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.837+0000 7fd3c7a5c640 1 -- 172.21.15.76:0/1989463742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3c009a250 msgr2=0x7fd3c015d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.837+0000 7fd3c7a5c640 1 --2- 172.21.15.76:0/1989463742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3c009a250 0x7fd3c015d280 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00d800 tx=0x7fd3bc00dcc0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.837+0000 7fd3c7a5c640 1 -- 172.21.15.76:0/1989463742 shutdown_connections 2024-03-08T06:01:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.838+0000 7fd3c7a5c640 1 --2- 172.21.15.76:0/1989463742 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd3a406b500 0x7fd3a406d9c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.838+0000 7fd3c7a5c640 1 --2- 172.21.15.76:0/1989463742 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd3c015bcd0 0x7fd3c00c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.838+0000 7fd3c7a5c640 1 --2- 172.21.15.76:0/1989463742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3c00cf730 0x7fd3c00c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.838+0000 7fd3c7a5c640 1 --2- 172.21.15.76:0/1989463742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3c009a250 0x7fd3c015d280 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.838+0000 7fd3c7a5c640 1 -- 172.21.15.76:0/1989463742 >> 172.21.15.76:0/1989463742 conn(0x7fd3c00b3440 msgr2=0x7fd3c00c7490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.838+0000 7fd3c7a5c640 1 -- 172.21.15.76:0/1989463742 shutdown_connections 2024-03-08T06:01:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.838+0000 7fd3c7a5c640 1 -- 172.21.15.76:0/1989463742 wait complete. 2024-03-08T06:01:50.849 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-03-08T06:01:49.935494+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220960,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753760,"statfs":{"total":773094113280,"available":772867850240,"internally_reserved":0,"allocated":4857856,"data_stored":2749732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.066288"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879211+0000","last_change":"2024-03-08T06:01:47.898935+0000","last_active":"2024-03-08T06:01:48.879211+0000","last_peered":"2024-03-08T06:01:48.879211+0000","last_clean":"2024-03-08T06:01:48.879211+0000","last_became_active":"2024-03-08T06:01:47.898668+0000","last_became_peered":"2024-03-08T06:01:47.898668+0000","last_unstale":"2024-03-08T06:01:48.879211+0000","last_undegraded":"2024-03-08T06:01:48.879211+0000","last_fullsized":"2024-03-08T06:01:48.879211+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:06:30.869091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879054+0000","last_change":"2024-03-08T06:01:47.896934+0000","last_active":"2024-03-08T06:01:48.879054+0000","last_peered":"2024-03-08T06:01:48.879054+0000","last_clean":"2024-03-08T06:01:48.879054+0000","last_became_active":"2024-03-08T06:01:47.896015+0000","last_became_peered":"2024-03-08T06:01:47.896015+0000","last_unstale":"2024-03-08T06:01:48.879054+0000","last_undegraded":"2024-03-08T06:01:48.879054+0000","last_fullsized":"2024-03-08T06:01:48.879054+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:16:48.835489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.906214+0000","last_change":"2024-03-08T06:01:47.900201+0000","last_active":"2024-03-08T06:01:47.906214+0000","last_peered":"2024-03-08T06:01:47.906214+0000","last_clean":"2024-03-08T06:01:47.906214+0000","last_became_active":"2024-03-08T06:01:47.899953+0000","last_became_peered":"2024-03-08T06:01:47.899953+0000","last_unstale":"2024-03-08T06:01:47.906214+0000","last_undegraded":"2024-03-08T06:01:47.906214+0000","last_fullsized":"2024-03-08T06:01:47.906214+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:33:00.231357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.903790+0000","last_change":"2024-03-08T06:01:47.897290+0000","last_active":"2024-03-08T06:01:47.903790+0000","last_peered":"2024-03-08T06:01:47.903790+0000","last_clean":"2024-03-08T06:01:47.903790+0000","last_became_active":"2024-03-08T06:01:47.897039+0000","last_became_peered":"2024-03-08T06:01:47.897039+0000","last_unstale":"2024-03-08T06:01:47.903790+0000","last_undegraded":"2024-03-08T06:01:47.903790+0000","last_fullsized":"2024-03-08T06:01:47.903790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:58:42.629432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.903794+0000","last_change":"2024-03-08T06:01:47.896786+0000","last_active":"2024-03-08T06:01:47.903794+0000","last_peered":"2024-03-08T06:01:47.903794+0000","last_clean":"2024-03-08T06:01:47.903794+0000","last_became_active":"2024-03-08T06:01:47.896623+0000","last_became_peered":"2024-03-08T06:01:47.896623+0000","last_unstale":"2024-03-08T06:01:47.903794+0000","last_undegraded":"2024-03-08T06:01:47.903794+0000","last_fullsized":"2024-03-08T06:01:47.903794+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:17:26.545703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.894260+0000","last_change":"2024-03-08T06:01:47.890376+0000","last_active":"2024-03-08T06:01:47.894260+0000","last_peered":"2024-03-08T06:01:47.894260+0000","last_clean":"2024-03-08T06:01:47.894260+0000","last_became_active":"2024-03-08T06:01:47.890103+0000","last_became_peered":"2024-03-08T06:01:47.890103+0000","last_unstale":"2024-03-08T06:01:47.894260+0000","last_undegraded":"2024-03-08T06:01:47.894260+0000","last_fullsized":"2024-03-08T06:01:47.894260+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:15.316795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.897566+0000","last_change":"2024-03-08T06:01:47.893733+0000","last_active":"2024-03-08T06:01:47.897566+0000","last_peered":"2024-03-08T06:01:47.897566+0000","last_clean":"2024-03-08T06:01:47.897566+0000","last_became_active":"2024-03-08T06:01:47.893022+0000","last_became_peered":"2024-03-08T06:01:47.893022+0000","last_unstale":"2024-03-08T06:01:47.897566+0000","last_undegraded":"2024-03-08T06:01:47.897566+0000","last_fullsized":"2024-03-08T06:01:47.897566+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:15.275207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.878688+0000","last_change":"2024-03-08T06:01:47.896697+0000","last_active":"2024-03-08T06:01:48.878688+0000","last_peered":"2024-03-08T06:01:48.878688+0000","last_clean":"2024-03-08T06:01:48.878688+0000","last_became_active":"2024-03-08T06:01:47.896436+0000","last_became_peered":"2024-03-08T06:01:47.896436+0000","last_unstale":"2024-03-08T06:01:48.878688+0000","last_undegraded":"2024-03-08T06:01:48.878688+0000","last_fullsized":"2024-03-08T06:01:48.878688+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:47:33.608607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.909075+0000","last_change":"2024-03-08T06:01:47.902949+0000","last_active":"2024-03-08T06:01:47.909075+0000","last_peered":"2024-03-08T06:01:47.909075+0000","last_clean":"2024-03-08T06:01:47.909075+0000","last_became_active":"2024-03-08T06:01:47.902690+0000","last_became_peered":"2024-03-08T06:01:47.902690+0000","last_unstale":"2024-03-08T06:01:47.909075+0000","last_undegraded":"2024-03-08T06:01:47.909075+0000","last_fullsized":"2024-03-08T06:01:47.909075+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:42:33.782761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.901875+0000","last_change":"2024-03-08T06:01:47.896269+0000","last_active":"2024-03-08T06:01:47.901875+0000","last_peered":"2024-03-08T06:01:47.901875+0000","last_clean":"2024-03-08T06:01:47.901875+0000","last_became_active":"2024-03-08T06:01:47.893623+0000","last_became_peered":"2024-03-08T06:01:47.893623+0000","last_unstale":"2024-03-08T06:01:47.901875+0000","last_undegraded":"2024-03-08T06:01:47.901875+0000","last_fullsized":"2024-03-08T06:01:47.901875+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:24:59.576847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.898254+0000","last_change":"2024-03-08T06:01:47.895173+0000","last_active":"2024-03-08T06:01:47.898254+0000","last_peered":"2024-03-08T06:01:47.898254+0000","last_clean":"2024-03-08T06:01:47.898254+0000","last_became_active":"2024-03-08T06:01:47.894860+0000","last_became_peered":"2024-03-08T06:01:47.894860+0000","last_unstale":"2024-03-08T06:01:47.898254+0000","last_undegraded":"2024-03-08T06:01:47.898254+0000","last_fullsized":"2024-03-08T06:01:47.898254+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:26:23.550550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.901071+0000","last_change":"2024-03-08T06:01:47.897828+0000","last_active":"2024-03-08T06:01:47.901071+0000","last_peered":"2024-03-08T06:01:47.901071+0000","last_clean":"2024-03-08T06:01:47.901071+0000","last_became_active":"2024-03-08T06:01:47.897575+0000","last_became_peered":"2024-03-08T06:01:47.897575+0000","last_unstale":"2024-03-08T06:01:47.901071+0000","last_undegraded":"2024-03-08T06:01:47.901071+0000","last_fullsized":"2024-03-08T06:01:47.901071+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:28.902386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.900199+0000","last_change":"2024-03-08T06:01:47.891993+0000","last_active":"2024-03-08T06:01:47.900199+0000","last_peered":"2024-03-08T06:01:47.900199+0000","last_clean":"2024-03-08T06:01:47.900199+0000","last_became_active":"2024-03-08T06:01:47.891669+0000","last_became_peered":"2024-03-08T06:01:47.891669+0000","last_unstale":"2024-03-08T06:01:47.900199+0000","last_undegraded":"2024-03-08T06:01:47.900199+0000","last_fullsized":"2024-03-08T06:01:47.900199+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:33:00.231357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879841+0000","last_change":"2024-03-08T06:01:47.893422+0000","last_active":"2024-03-08T06:01:48.879841+0000","last_peered":"2024-03-08T06:01:48.879841+0000","last_clean":"2024-03-08T06:01:48.879841+0000","last_became_active":"2024-03-08T06:01:47.893171+0000","last_became_peered":"2024-03-08T06:01:47.893171+0000","last_unstale":"2024-03-08T06:01:48.879841+0000","last_undegraded":"2024-03-08T06:01:48.879841+0000","last_fullsized":"2024-03-08T06:01:48.879841+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:53:30.002460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.899543+0000","last_change":"2024-03-08T06:01:47.893939+0000","last_active":"2024-03-08T06:01:47.899543+0000","last_peered":"2024-03-08T06:01:47.899543+0000","last_clean":"2024-03-08T06:01:47.899543+0000","last_became_active":"2024-03-08T06:01:47.893406+0000","last_became_peered":"2024-03-08T06:01:47.893406+0000","last_unstale":"2024-03-08T06:01:47.899543+0000","last_undegraded":"2024-03-08T06:01:47.899543+0000","last_fullsized":"2024-03-08T06:01:47.899543+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:53:30.002460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.909907+0000","last_change":"2024-03-08T06:01:47.899729+0000","last_active":"2024-03-08T06:01:47.909907+0000","last_peered":"2024-03-08T06:01:47.909907+0000","last_clean":"2024-03-08T06:01:47.909907+0000","last_became_active":"2024-03-08T06:01:47.898946+0000","last_became_peered":"2024-03-08T06:01:47.898946+0000","last_unstale":"2024-03-08T06:01:47.909907+0000","last_undegraded":"2024-03-08T06:01:47.909907+0000","last_fullsized":"2024-03-08T06:01:47.909907+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:02:08.833021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910249+0000","last_change":"2024-03-08T06:01:47.904656+0000","last_active":"2024-03-08T06:01:47.910249+0000","last_peered":"2024-03-08T06:01:47.910249+0000","last_clean":"2024-03-08T06:01:47.910249+0000","last_became_active":"2024-03-08T06:01:47.904385+0000","last_became_peered":"2024-03-08T06:01:47.904385+0000","last_unstale":"2024-03-08T06:01:47.910249+0000","last_undegraded":"2024-03-08T06:01:47.910249+0000","last_fullsized":"2024-03-08T06:01:47.910249+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:15.316795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.904787+0000","last_change":"2024-03-08T06:01:47.901117+0000","last_active":"2024-03-08T06:01:47.904787+0000","last_peered":"2024-03-08T06:01:47.904787+0000","last_clean":"2024-03-08T06:01:47.904787+0000","last_became_active":"2024-03-08T06:01:47.900784+0000","last_became_peered":"2024-03-08T06:01:47.900784+0000","last_unstale":"2024-03-08T06:01:47.904787+0000","last_undegraded":"2024-03-08T06:01:47.904787+0000","last_fullsized":"2024-03-08T06:01:47.904787+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:45:17.531673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.901114+0000","last_change":"2024-03-08T06:01:47.893890+0000","last_active":"2024-03-08T06:01:47.901114+0000","last_peered":"2024-03-08T06:01:47.901114+0000","last_clean":"2024-03-08T06:01:47.901114+0000","last_became_active":"2024-03-08T06:01:47.893642+0000","last_became_peered":"2024-03-08T06:01:47.893642+0000","last_unstale":"2024-03-08T06:01:47.901114+0000","last_undegraded":"2024-03-08T06:01:47.901114+0000","last_fullsized":"2024-03-08T06:01:47.901114+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:46:39.400065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.904062+0000","last_change":"2024-03-08T06:01:47.896780+0000","last_active":"2024-03-08T06:01:47.904062+0000","last_peered":"2024-03-08T06:01:47.904062+0000","last_clean":"2024-03-08T06:01:47.904062+0000","last_became_active":"2024-03-08T06:01:47.896517+0000","last_became_peered":"2024-03-08T06:01:47.896517+0000","last_unstale":"2024-03-08T06:01:47.904062+0000","last_undegraded":"2024-03-08T06:01:47.904062+0000","last_fullsized":"2024-03-08T06:01:47.904062+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:09:42.307553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.904674+0000","last_change":"2024-03-08T06:01:47.900637+0000","last_active":"2024-03-08T06:01:47.904674+0000","last_peered":"2024-03-08T06:01:47.904674+0000","last_clean":"2024-03-08T06:01:47.904674+0000","last_became_active":"2024-03-08T06:01:47.900340+0000","last_became_peered":"2024-03-08T06:01:47.900340+0000","last_unstale":"2024-03-08T06:01:47.904674+0000","last_undegraded":"2024-03-08T06:01:47.904674+0000","last_fullsized":"2024-03-08T06:01:47.904674+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:59:52.595386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.880118+0000","last_change":"2024-03-08T06:01:47.894955+0000","last_active":"2024-03-08T06:01:48.880118+0000","last_peered":"2024-03-08T06:01:48.880118+0000","last_clean":"2024-03-08T06:01:48.880118+0000","last_became_active":"2024-03-08T06:01:47.894679+0000","last_became_peered":"2024-03-08T06:01:47.894679+0000","last_unstale":"2024-03-08T06:01:48.880118+0000","last_undegraded":"2024-03-08T06:01:48.880118+0000","last_fullsized":"2024-03-08T06:01:48.880118+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:11:16.081832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910138+0000","last_change":"2024-03-08T06:01:47.904167+0000","last_active":"2024-03-08T06:01:47.910138+0000","last_peered":"2024-03-08T06:01:47.910138+0000","last_clean":"2024-03-08T06:01:47.910138+0000","last_became_active":"2024-03-08T06:01:47.903877+0000","last_became_peered":"2024-03-08T06:01:47.903877+0000","last_unstale":"2024-03-08T06:01:47.910138+0000","last_undegraded":"2024-03-08T06:01:47.910138+0000","last_fullsized":"2024-03-08T06:01:47.910138+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:19:00.645250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.901088+0000","last_change":"2024-03-08T06:01:47.893770+0000","last_active":"2024-03-08T06:01:47.901088+0000","last_peered":"2024-03-08T06:01:47.901088+0000","last_clean":"2024-03-08T06:01:47.901088+0000","last_became_active":"2024-03-08T06:01:47.893532+0000","last_became_peered":"2024-03-08T06:01:47.893532+0000","last_unstale":"2024-03-08T06:01:47.901088+0000","last_undegraded":"2024-03-08T06:01:47.901088+0000","last_fullsized":"2024-03-08T06:01:47.901088+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:33:16.443151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.904826+0000","last_change":"2024-03-08T06:01:47.900296+0000","last_active":"2024-03-08T06:01:47.904826+0000","last_peered":"2024-03-08T06:01:47.904826+0000","last_clean":"2024-03-08T06:01:47.904826+0000","last_became_active":"2024-03-08T06:01:47.897905+0000","last_became_peered":"2024-03-08T06:01:47.897905+0000","last_unstale":"2024-03-08T06:01:47.904826+0000","last_undegraded":"2024-03-08T06:01:47.904826+0000","last_fullsized":"2024-03-08T06:01:47.904826+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:53:52.111951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.907041+0000","last_change":"2024-03-08T06:01:47.901017+0000","last_active":"2024-03-08T06:01:47.907041+0000","last_peered":"2024-03-08T06:01:47.907041+0000","last_clean":"2024-03-08T06:01:47.907041+0000","last_became_active":"2024-03-08T06:01:47.900760+0000","last_became_peered":"2024-03-08T06:01:47.900760+0000","last_unstale":"2024-03-08T06:01:47.907041+0000","last_undegraded":"2024-03-08T06:01:47.907041+0000","last_fullsized":"2024-03-08T06:01:47.907041+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:33:00.231357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.904720+0000","last_change":"2024-03-08T06:01:47.899891+0000","last_active":"2024-03-08T06:01:47.904720+0000","last_peered":"2024-03-08T06:01:47.904720+0000","last_clean":"2024-03-08T06:01:47.904720+0000","last_became_active":"2024-03-08T06:01:47.899205+0000","last_became_peered":"2024-03-08T06:01:47.899205+0000","last_unstale":"2024-03-08T06:01:47.904720+0000","last_undegraded":"2024-03-08T06:01:47.904720+0000","last_fullsized":"2024-03-08T06:01:47.904720+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:18:21.554370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.897554+0000","last_change":"2024-03-08T06:01:47.893348+0000","last_active":"2024-03-08T06:01:47.897554+0000","last_peered":"2024-03-08T06:01:47.897554+0000","last_clean":"2024-03-08T06:01:47.897554+0000","last_became_active":"2024-03-08T06:01:47.891691+0000","last_became_peered":"2024-03-08T06:01:47.891691+0000","last_unstale":"2024-03-08T06:01:47.897554+0000","last_undegraded":"2024-03-08T06:01:47.897554+0000","last_fullsized":"2024-03-08T06:01:47.897554+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:42:33.782761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.891914+0000","last_change":"2024-03-08T06:01:47.889974+0000","last_active":"2024-03-08T06:01:47.891914+0000","last_peered":"2024-03-08T06:01:47.891914+0000","last_clean":"2024-03-08T06:01:47.891914+0000","last_became_active":"2024-03-08T06:01:47.889701+0000","last_became_peered":"2024-03-08T06:01:47.889701+0000","last_unstale":"2024-03-08T06:01:47.891914+0000","last_undegraded":"2024-03-08T06:01:47.891914+0000","last_fullsized":"2024-03-08T06:01:47.891914+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:59:30.643039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.904124+0000","last_change":"2024-03-08T06:01:47.897327+0000","last_active":"2024-03-08T06:01:47.904124+0000","last_peered":"2024-03-08T06:01:47.904124+0000","last_clean":"2024-03-08T06:01:47.904124+0000","last_became_active":"2024-03-08T06:01:47.897024+0000","last_became_peered":"2024-03-08T06:01:47.897024+0000","last_unstale":"2024-03-08T06:01:47.904124+0000","last_undegraded":"2024-03-08T06:01:47.904124+0000","last_fullsized":"2024-03-08T06:01:47.904124+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:25:21.032491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910171+0000","last_change":"2024-03-08T06:01:47.906859+0000","last_active":"2024-03-08T06:01:47.910171+0000","last_peered":"2024-03-08T06:01:47.910171+0000","last_clean":"2024-03-08T06:01:47.910171+0000","last_became_active":"2024-03-08T06:01:47.905488+0000","last_became_peered":"2024-03-08T06:01:47.905488+0000","last_unstale":"2024-03-08T06:01:47.910171+0000","last_undegraded":"2024-03-08T06:01:47.910171+0000","last_fullsized":"2024-03-08T06:01:47.910171+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:06:47.982479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.898388+0000","last_change":"2024-03-08T06:01:47.894973+0000","last_active":"2024-03-08T06:01:47.898388+0000","last_peered":"2024-03-08T06:01:47.898388+0000","last_clean":"2024-03-08T06:01:47.898388+0000","last_became_active":"2024-03-08T06:01:47.894725+0000","last_became_peered":"2024-03-08T06:01:47.894725+0000","last_unstale":"2024-03-08T06:01:47.898388+0000","last_undegraded":"2024-03-08T06:01:47.898388+0000","last_fullsized":"2024-03-08T06:01:47.898388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:06:47.982479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.909796+0000","last_change":"2024-03-08T06:01:47.899673+0000","last_active":"2024-03-08T06:01:47.909796+0000","last_peered":"2024-03-08T06:01:47.909796+0000","last_clean":"2024-03-08T06:01:47.909796+0000","last_became_active":"2024-03-08T06:01:47.899383+0000","last_became_peered":"2024-03-08T06:01:47.899383+0000","last_unstale":"2024-03-08T06:01:47.909796+0000","last_undegraded":"2024-03-08T06:01:47.909796+0000","last_fullsized":"2024-03-08T06:01:47.909796+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:53:52.111951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910041+0000","last_change":"2024-03-08T06:01:46.892159+0000","last_active":"2024-03-08T06:01:47.910041+0000","last_peered":"2024-03-08T06:01:47.910041+0000","last_clean":"2024-03-08T06:01:47.910041+0000","last_became_active":"2024-03-08T06:01:46.891861+0000","last_became_peered":"2024-03-08T06:01:46.891861+0000","last_unstale":"2024-03-08T06:01:47.910041+0000","last_undegraded":"2024-03-08T06:01:47.910041+0000","last_fullsized":"2024-03-08T06:01:47.910041+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:03:52.842458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.907093+0000","last_change":"2024-03-08T06:01:47.899191+0000","last_active":"2024-03-08T06:01:47.907093+0000","last_peered":"2024-03-08T06:01:47.907093+0000","last_clean":"2024-03-08T06:01:47.907093+0000","last_became_active":"2024-03-08T06:01:47.898945+0000","last_became_peered":"2024-03-08T06:01:47.898945+0000","last_unstale":"2024-03-08T06:01:47.907093+0000","last_undegraded":"2024-03-08T06:01:47.907093+0000","last_fullsized":"2024-03-08T06:01:47.907093+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:28.902386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910326+0000","last_change":"2024-03-08T06:01:46.894475+0000","last_active":"2024-03-08T06:01:47.910326+0000","last_peered":"2024-03-08T06:01:47.910326+0000","last_clean":"2024-03-08T06:01:47.910326+0000","last_became_active":"2024-03-08T06:01:46.894237+0000","last_became_peered":"2024-03-08T06:01:46.894237+0000","last_unstale":"2024-03-08T06:01:47.910326+0000","last_undegraded":"2024-03-08T06:01:47.910326+0000","last_fullsized":"2024-03-08T06:01:47.910326+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:34:32.641931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879915+0000","last_change":"2024-03-08T06:01:47.892760+0000","last_active":"2024-03-08T06:01:48.879915+0000","last_peered":"2024-03-08T06:01:48.879915+0000","last_clean":"2024-03-08T06:01:48.879915+0000","last_became_active":"2024-03-08T06:01:47.892491+0000","last_became_peered":"2024-03-08T06:01:47.892491+0000","last_unstale":"2024-03-08T06:01:48.879915+0000","last_undegraded":"2024-03-08T06:01:48.879915+0000","last_fullsized":"2024-03-08T06:01:48.879915+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:24:59.576847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910070+0000","last_change":"2024-03-08T06:01:46.893158+0000","last_active":"2024-03-08T06:01:47.910070+0000","last_peered":"2024-03-08T06:01:47.910070+0000","last_clean":"2024-03-08T06:01:47.910070+0000","last_became_active":"2024-03-08T06:01:46.892893+0000","last_became_peered":"2024-03-08T06:01:46.892893+0000","last_unstale":"2024-03-08T06:01:47.910070+0000","last_undegraded":"2024-03-08T06:01:47.910070+0000","last_fullsized":"2024-03-08T06:01:47.910070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:18:59.639546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879148+0000","last_change":"2024-03-08T06:01:47.890649+0000","last_active":"2024-03-08T06:01:48.879148+0000","last_peered":"2024-03-08T06:01:48.879148+0000","last_clean":"2024-03-08T06:01:48.879148+0000","last_became_active":"2024-03-08T06:01:47.890355+0000","last_became_peered":"2024-03-08T06:01:47.890355+0000","last_unstale":"2024-03-08T06:01:48.879148+0000","last_undegraded":"2024-03-08T06:01:48.879148+0000","last_fullsized":"2024-03-08T06:01:48.879148+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:53:38.958047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879917+0000","last_change":"2024-03-08T06:01:46.895277+0000","last_active":"2024-03-08T06:01:48.879917+0000","last_peered":"2024-03-08T06:01:48.879917+0000","last_clean":"2024-03-08T06:01:48.879917+0000","last_became_active":"2024-03-08T06:01:46.894895+0000","last_became_peered":"2024-03-08T06:01:46.894895+0000","last_unstale":"2024-03-08T06:01:48.879917+0000","last_undegraded":"2024-03-08T06:01:48.879917+0000","last_fullsized":"2024-03-08T06:01:48.879917+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:02:07.827318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.897205+0000","last_change":"2024-03-08T06:01:47.894588+0000","last_active":"2024-03-08T06:01:47.897205+0000","last_peered":"2024-03-08T06:01:47.897205+0000","last_clean":"2024-03-08T06:01:47.897205+0000","last_became_active":"2024-03-08T06:01:47.894307+0000","last_became_peered":"2024-03-08T06:01:47.894307+0000","last_unstale":"2024-03-08T06:01:47.897205+0000","last_undegraded":"2024-03-08T06:01:47.897205+0000","last_fullsized":"2024-03-08T06:01:47.897205+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:02.674261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.911715+0000","last_change":"2024-03-08T06:01:46.896374+0000","last_active":"2024-03-08T06:01:47.911715+0000","last_peered":"2024-03-08T06:01:47.911715+0000","last_clean":"2024-03-08T06:01:47.911715+0000","last_became_active":"2024-03-08T06:01:46.896060+0000","last_became_peered":"2024-03-08T06:01:46.896060+0000","last_unstale":"2024-03-08T06:01:47.911715+0000","last_undegraded":"2024-03-08T06:01:47.911715+0000","last_fullsized":"2024-03-08T06:01:47.911715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:27.896682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.902095+0000","last_change":"2024-03-08T06:01:47.897635+0000","last_active":"2024-03-08T06:01:47.902095+0000","last_peered":"2024-03-08T06:01:47.902095+0000","last_clean":"2024-03-08T06:01:47.902095+0000","last_became_active":"2024-03-08T06:01:47.897212+0000","last_became_peered":"2024-03-08T06:01:47.897212+0000","last_unstale":"2024-03-08T06:01:47.902095+0000","last_undegraded":"2024-03-08T06:01:47.902095+0000","last_fullsized":"2024-03-08T06:01:47.902095+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:11:16.081832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.903746+0000","last_change":"2024-03-08T06:01:46.893307+0000","last_active":"2024-03-08T06:01:47.903746+0000","last_peered":"2024-03-08T06:01:47.903746+0000","last_clean":"2024-03-08T06:01:47.903746+0000","last_became_active":"2024-03-08T06:01:46.893035+0000","last_became_peered":"2024-03-08T06:01:46.893035+0000","last_unstale":"2024-03-08T06:01:47.903746+0000","last_undegraded":"2024-03-08T06:01:47.903746+0000","last_fullsized":"2024-03-08T06:01:47.903746+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:03:52.842458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.880181+0000","last_change":"2024-03-08T06:01:47.898718+0000","last_active":"2024-03-08T06:01:48.880181+0000","last_peered":"2024-03-08T06:01:48.880181+0000","last_clean":"2024-03-08T06:01:48.880181+0000","last_became_active":"2024-03-08T06:01:47.898440+0000","last_became_peered":"2024-03-08T06:01:47.898440+0000","last_unstale":"2024-03-08T06:01:48.880181+0000","last_undegraded":"2024-03-08T06:01:48.880181+0000","last_fullsized":"2024-03-08T06:01:48.880181+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:59:52.595386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.887807+0000","last_change":"2024-03-08T06:01:46.897480+0000","last_active":"2024-03-08T06:01:47.887807+0000","last_peered":"2024-03-08T06:01:47.887807+0000","last_clean":"2024-03-08T06:01:47.887807+0000","last_became_active":"2024-03-08T06:01:46.896101+0000","last_became_peered":"2024-03-08T06:01:46.896101+0000","last_unstale":"2024-03-08T06:01:47.887807+0000","last_undegraded":"2024-03-08T06:01:47.887807+0000","last_fullsized":"2024-03-08T06:01:47.887807+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:45:16.525970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.878837+0000","last_change":"2024-03-08T06:01:47.901392+0000","last_active":"2024-03-08T06:01:48.878837+0000","last_peered":"2024-03-08T06:01:48.878837+0000","last_clean":"2024-03-08T06:01:48.878837+0000","last_became_active":"2024-03-08T06:01:47.901103+0000","last_became_peered":"2024-03-08T06:01:47.901103+0000","last_unstale":"2024-03-08T06:01:48.878837+0000","last_undegraded":"2024-03-08T06:01:48.878837+0000","last_fullsized":"2024-03-08T06:01:48.878837+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:59:06.692046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879423+0000","last_change":"2024-03-08T06:01:46.889608+0000","last_active":"2024-03-08T06:01:48.879423+0000","last_peered":"2024-03-08T06:01:48.879423+0000","last_clean":"2024-03-08T06:01:48.879423+0000","last_became_active":"2024-03-08T06:01:46.889326+0000","last_became_peered":"2024-03-08T06:01:46.889326+0000","last_unstale":"2024-03-08T06:01:48.879423+0000","last_undegraded":"2024-03-08T06:01:48.879423+0000","last_fullsized":"2024-03-08T06:01:48.879423+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:45:16.525970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.903765+0000","last_change":"2024-03-08T06:01:47.897009+0000","last_active":"2024-03-08T06:01:47.903765+0000","last_peered":"2024-03-08T06:01:47.903765+0000","last_clean":"2024-03-08T06:01:47.903765+0000","last_became_active":"2024-03-08T06:01:47.895961+0000","last_became_peered":"2024-03-08T06:01:47.895961+0000","last_unstale":"2024-03-08T06:01:47.903765+0000","last_undegraded":"2024-03-08T06:01:47.903765+0000","last_fullsized":"2024-03-08T06:01:47.903765+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:57.350326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879233+0000","last_change":"2024-03-08T06:01:46.891263+0000","last_active":"2024-03-08T06:01:48.879233+0000","last_peered":"2024-03-08T06:01:48.879233+0000","last_clean":"2024-03-08T06:01:48.879233+0000","last_became_active":"2024-03-08T06:01:46.891014+0000","last_became_peered":"2024-03-08T06:01:46.891014+0000","last_unstale":"2024-03-08T06:01:48.879233+0000","last_undegraded":"2024-03-08T06:01:48.879233+0000","last_fullsized":"2024-03-08T06:01:48.879233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:36:09.052707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.909852+0000","last_change":"2024-03-08T06:01:47.900108+0000","last_active":"2024-03-08T06:01:47.909852+0000","last_peered":"2024-03-08T06:01:47.909852+0000","last_clean":"2024-03-08T06:01:47.909852+0000","last_became_active":"2024-03-08T06:01:47.899563+0000","last_became_peered":"2024-03-08T06:01:47.899563+0000","last_unstale":"2024-03-08T06:01:47.909852+0000","last_undegraded":"2024-03-08T06:01:47.909852+0000","last_fullsized":"2024-03-08T06:01:47.909852+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:34:33.647634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.883615+0000","last_change":"2024-03-08T06:01:46.894871+0000","last_active":"2024-03-08T06:01:47.883615+0000","last_peered":"2024-03-08T06:01:47.883615+0000","last_clean":"2024-03-08T06:01:47.883615+0000","last_became_active":"2024-03-08T06:01:46.894585+0000","last_became_peered":"2024-03-08T06:01:46.894585+0000","last_unstale":"2024-03-08T06:01:47.883615+0000","last_undegraded":"2024-03-08T06:01:47.883615+0000","last_fullsized":"2024-03-08T06:01:47.883615+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:14.311091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.896311+0000","last_change":"2024-03-08T06:01:47.891350+0000","last_active":"2024-03-08T06:01:47.896311+0000","last_peered":"2024-03-08T06:01:47.896311+0000","last_clean":"2024-03-08T06:01:47.896311+0000","last_became_active":"2024-03-08T06:01:47.891097+0000","last_became_peered":"2024-03-08T06:01:47.891097+0000","last_unstale":"2024-03-08T06:01:47.896311+0000","last_undegraded":"2024-03-08T06:01:47.896311+0000","last_fullsized":"2024-03-08T06:01:47.896311+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:15.316795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.912537+0000","last_change":"2024-03-08T06:01:46.896161+0000","last_active":"2024-03-08T06:01:47.912537+0000","last_peered":"2024-03-08T06:01:47.912537+0000","last_clean":"2024-03-08T06:01:47.912537+0000","last_became_active":"2024-03-08T06:01:46.895815+0000","last_became_peered":"2024-03-08T06:01:46.895815+0000","last_unstale":"2024-03-08T06:01:47.912537+0000","last_undegraded":"2024-03-08T06:01:47.912537+0000","last_fullsized":"2024-03-08T06:01:47.912537+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:18:59.639546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879879+0000","last_change":"2024-03-08T06:01:47.895259+0000","last_active":"2024-03-08T06:01:48.879879+0000","last_peered":"2024-03-08T06:01:48.879879+0000","last_clean":"2024-03-08T06:01:48.879879+0000","last_became_active":"2024-03-08T06:01:47.894971+0000","last_became_peered":"2024-03-08T06:01:47.894971+0000","last_unstale":"2024-03-08T06:01:48.879879+0000","last_undegraded":"2024-03-08T06:01:48.879879+0000","last_fullsized":"2024-03-08T06:01:48.879879+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:44:12.918446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.878969+0000","last_change":"2024-03-08T06:01:46.890902+0000","last_active":"2024-03-08T06:01:48.878969+0000","last_peered":"2024-03-08T06:01:48.878969+0000","last_clean":"2024-03-08T06:01:48.878969+0000","last_became_active":"2024-03-08T06:01:46.890638+0000","last_became_peered":"2024-03-08T06:01:46.890638+0000","last_unstale":"2024-03-08T06:01:48.878969+0000","last_undegraded":"2024-03-08T06:01:48.878969+0000","last_fullsized":"2024-03-08T06:01:48.878969+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:11:15.076128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.906262+0000","last_change":"2024-03-08T06:01:47.900191+0000","last_active":"2024-03-08T06:01:47.906262+0000","last_peered":"2024-03-08T06:01:47.906262+0000","last_clean":"2024-03-08T06:01:47.906262+0000","last_became_active":"2024-03-08T06:01:47.899537+0000","last_became_peered":"2024-03-08T06:01:47.899537+0000","last_unstale":"2024-03-08T06:01:47.906262+0000","last_undegraded":"2024-03-08T06:01:47.906262+0000","last_fullsized":"2024-03-08T06:01:47.906262+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:39:55.488990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879099+0000","last_change":"2024-03-08T06:01:46.889586+0000","last_active":"2024-03-08T06:01:48.879099+0000","last_peered":"2024-03-08T06:01:48.879099+0000","last_clean":"2024-03-08T06:01:48.879099+0000","last_became_active":"2024-03-08T06:01:46.889263+0000","last_became_peered":"2024-03-08T06:01:46.889263+0000","last_unstale":"2024-03-08T06:01:48.879099+0000","last_undegraded":"2024-03-08T06:01:48.879099+0000","last_fullsized":"2024-03-08T06:01:48.879099+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:59:51.589682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.878775+0000","last_change":"2024-03-08T06:01:47.897815+0000","last_active":"2024-03-08T06:01:48.878775+0000","last_peered":"2024-03-08T06:01:48.878775+0000","last_clean":"2024-03-08T06:01:48.878775+0000","last_became_active":"2024-03-08T06:01:47.897563+0000","last_became_peered":"2024-03-08T06:01:47.897563+0000","last_unstale":"2024-03-08T06:01:48.878775+0000","last_undegraded":"2024-03-08T06:01:48.878775+0000","last_fullsized":"2024-03-08T06:01:48.878775+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:33:01.188292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.887861+0000","last_change":"2024-03-08T06:01:46.896455+0000","last_active":"2024-03-08T06:01:47.887861+0000","last_peered":"2024-03-08T06:01:47.887861+0000","last_clean":"2024-03-08T06:01:47.887861+0000","last_became_active":"2024-03-08T06:01:46.896196+0000","last_became_peered":"2024-03-08T06:01:46.896196+0000","last_unstale":"2024-03-08T06:01:47.887861+0000","last_undegraded":"2024-03-08T06:01:47.887861+0000","last_fullsized":"2024-03-08T06:01:47.887861+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:34:32.641931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.907616+0000","last_change":"2024-03-08T06:01:47.897001+0000","last_active":"2024-03-08T06:01:47.907616+0000","last_peered":"2024-03-08T06:01:47.907616+0000","last_clean":"2024-03-08T06:01:47.907616+0000","last_became_active":"2024-03-08T06:01:47.896717+0000","last_became_peered":"2024-03-08T06:01:47.896717+0000","last_unstale":"2024-03-08T06:01:47.907616+0000","last_undegraded":"2024-03-08T06:01:47.907616+0000","last_fullsized":"2024-03-08T06:01:47.907616+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:18:21.554370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.884190+0000","last_change":"2024-03-08T06:01:46.889002+0000","last_active":"2024-03-08T06:01:47.884190+0000","last_peered":"2024-03-08T06:01:47.884190+0000","last_clean":"2024-03-08T06:01:47.884190+0000","last_became_active":"2024-03-08T06:01:46.888723+0000","last_became_peered":"2024-03-08T06:01:46.888723+0000","last_unstale":"2024-03-08T06:01:47.884190+0000","last_undegraded":"2024-03-08T06:01:47.884190+0000","last_fullsized":"2024-03-08T06:01:47.884190+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:47:59.325645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.878943+0000","last_change":"2024-03-08T06:01:47.898597+0000","last_active":"2024-03-08T06:01:48.878943+0000","last_peered":"2024-03-08T06:01:48.878943+0000","last_clean":"2024-03-08T06:01:48.878943+0000","last_became_active":"2024-03-08T06:01:47.898192+0000","last_became_peered":"2024-03-08T06:01:47.898192+0000","last_unstale":"2024-03-08T06:01:48.878943+0000","last_undegraded":"2024-03-08T06:01:48.878943+0000","last_fullsized":"2024-03-08T06:01:48.878943+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:20:20.387478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910780+0000","last_change":"2024-03-08T06:01:46.892113+0000","last_active":"2024-03-08T06:01:47.910780+0000","last_peered":"2024-03-08T06:01:47.910780+0000","last_clean":"2024-03-08T06:01:47.910780+0000","last_became_active":"2024-03-08T06:01:46.891852+0000","last_became_peered":"2024-03-08T06:01:46.891852+0000","last_unstale":"2024-03-08T06:01:47.910780+0000","last_undegraded":"2024-03-08T06:01:47.910780+0000","last_fullsized":"2024-03-08T06:01:47.910780+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:59:29.637335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.878980+0000","last_change":"2024-03-08T06:01:47.889397+0000","last_active":"2024-03-08T06:01:48.878980+0000","last_peered":"2024-03-08T06:01:48.878980+0000","last_clean":"2024-03-08T06:01:48.878980+0000","last_became_active":"2024-03-08T06:01:47.889123+0000","last_became_peered":"2024-03-08T06:01:47.889123+0000","last_unstale":"2024-03-08T06:01:48.878980+0000","last_undegraded":"2024-03-08T06:01:48.878980+0000","last_fullsized":"2024-03-08T06:01:48.878980+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:33:23.152529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.912355+0000","last_change":"2024-03-08T06:01:46.892791+0000","last_active":"2024-03-08T06:01:47.912355+0000","last_peered":"2024-03-08T06:01:47.912355+0000","last_clean":"2024-03-08T06:01:47.912355+0000","last_became_active":"2024-03-08T06:01:46.892488+0000","last_became_peered":"2024-03-08T06:01:46.892488+0000","last_unstale":"2024-03-08T06:01:47.912355+0000","last_undegraded":"2024-03-08T06:01:47.912355+0000","last_fullsized":"2024-03-08T06:01:47.912355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:01.668558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.902017+0000","last_change":"2024-03-08T06:01:47.895748+0000","last_active":"2024-03-08T06:01:47.902017+0000","last_peered":"2024-03-08T06:01:47.902017+0000","last_clean":"2024-03-08T06:01:47.902017+0000","last_became_active":"2024-03-08T06:01:47.895475+0000","last_became_peered":"2024-03-08T06:01:47.895475+0000","last_unstale":"2024-03-08T06:01:47.902017+0000","last_undegraded":"2024-03-08T06:01:47.902017+0000","last_fullsized":"2024-03-08T06:01:47.902017+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:44:12.918446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.880004+0000","last_change":"2024-03-08T06:01:46.891434+0000","last_active":"2024-03-08T06:01:48.880004+0000","last_peered":"2024-03-08T06:01:48.880004+0000","last_clean":"2024-03-08T06:01:48.880004+0000","last_became_active":"2024-03-08T06:01:46.891170+0000","last_became_peered":"2024-03-08T06:01:46.891170+0000","last_unstale":"2024-03-08T06:01:48.880004+0000","last_undegraded":"2024-03-08T06:01:48.880004+0000","last_fullsized":"2024-03-08T06:01:48.880004+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:18:59.639546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879871+0000","last_change":"2024-03-08T06:01:47.890670+0000","last_active":"2024-03-08T06:01:48.879871+0000","last_peered":"2024-03-08T06:01:48.879871+0000","last_clean":"2024-03-08T06:01:48.879871+0000","last_became_active":"2024-03-08T06:01:47.890386+0000","last_became_peered":"2024-03-08T06:01:47.890386+0000","last_unstale":"2024-03-08T06:01:48.879871+0000","last_undegraded":"2024-03-08T06:01:48.879871+0000","last_fullsized":"2024-03-08T06:01:48.879871+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:26:23.550550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910486+0000","last_change":"2024-03-08T06:01:46.890485+0000","last_active":"2024-03-08T06:01:47.910486+0000","last_peered":"2024-03-08T06:01:47.910486+0000","last_clean":"2024-03-08T06:01:47.910486+0000","last_became_active":"2024-03-08T06:01:46.890245+0000","last_became_peered":"2024-03-08T06:01:46.890245+0000","last_unstale":"2024-03-08T06:01:47.910486+0000","last_undegraded":"2024-03-08T06:01:47.910486+0000","last_fullsized":"2024-03-08T06:01:47.910486+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:00:05.649277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.895778+0000","last_change":"2024-03-08T06:01:47.892473+0000","last_active":"2024-03-08T06:01:47.895778+0000","last_peered":"2024-03-08T06:01:47.895778+0000","last_clean":"2024-03-08T06:01:47.895778+0000","last_became_active":"2024-03-08T06:01:47.892209+0000","last_became_peered":"2024-03-08T06:01:47.892209+0000","last_unstale":"2024-03-08T06:01:47.895778+0000","last_undegraded":"2024-03-08T06:01:47.895778+0000","last_fullsized":"2024-03-08T06:01:47.895778+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:07:44.511982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.887954+0000","last_change":"2024-03-08T06:01:46.893168+0000","last_active":"2024-03-08T06:01:47.887954+0000","last_peered":"2024-03-08T06:01:47.887954+0000","last_clean":"2024-03-08T06:01:47.887954+0000","last_became_active":"2024-03-08T06:01:46.892901+0000","last_became_peered":"2024-03-08T06:01:46.892901+0000","last_unstale":"2024-03-08T06:01:47.887954+0000","last_undegraded":"2024-03-08T06:01:47.887954+0000","last_fullsized":"2024-03-08T06:01:47.887954+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:02:07.827318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910623+0000","last_change":"2024-03-08T06:01:47.905298+0000","last_active":"2024-03-08T06:01:47.910623+0000","last_peered":"2024-03-08T06:01:47.910623+0000","last_clean":"2024-03-08T06:01:47.910623+0000","last_became_active":"2024-03-08T06:01:47.904945+0000","last_became_peered":"2024-03-08T06:01:47.904945+0000","last_unstale":"2024-03-08T06:01:47.910623+0000","last_undegraded":"2024-03-08T06:01:47.910623+0000","last_fullsized":"2024-03-08T06:01:47.910623+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:02.674261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.903705+0000","last_change":"2024-03-08T06:01:46.891517+0000","last_active":"2024-03-08T06:01:47.903705+0000","last_peered":"2024-03-08T06:01:47.903705+0000","last_clean":"2024-03-08T06:01:47.903705+0000","last_became_active":"2024-03-08T06:01:46.891117+0000","last_became_peered":"2024-03-08T06:01:46.891117+0000","last_unstale":"2024-03-08T06:01:47.903705+0000","last_undegraded":"2024-03-08T06:01:47.903705+0000","last_fullsized":"2024-03-08T06:01:47.903705+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:27:58.111937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.909104+0000","last_change":"2024-03-08T06:01:47.905551+0000","last_active":"2024-03-08T06:01:47.909104+0000","last_peered":"2024-03-08T06:01:47.909104+0000","last_clean":"2024-03-08T06:01:47.909104+0000","last_became_active":"2024-03-08T06:01:47.905325+0000","last_became_peered":"2024-03-08T06:01:47.905325+0000","last_unstale":"2024-03-08T06:01:47.909104+0000","last_undegraded":"2024-03-08T06:01:47.909104+0000","last_fullsized":"2024-03-08T06:01:47.909104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:15.275207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.887288+0000","last_change":"2024-03-08T06:01:46.892096+0000","last_active":"2024-03-08T06:01:47.887288+0000","last_peered":"2024-03-08T06:01:47.887288+0000","last_clean":"2024-03-08T06:01:47.887288+0000","last_became_active":"2024-03-08T06:01:46.891853+0000","last_became_peered":"2024-03-08T06:01:46.891853+0000","last_unstale":"2024-03-08T06:01:47.887288+0000","last_undegraded":"2024-03-08T06:01:47.887288+0000","last_fullsized":"2024-03-08T06:01:47.887288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:15.634163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.894567+0000","last_change":"2024-03-08T06:01:47.886742+0000","last_active":"2024-03-08T06:01:47.894567+0000","last_peered":"2024-03-08T06:01:47.894567+0000","last_clean":"2024-03-08T06:01:47.894567+0000","last_became_active":"2024-03-08T06:01:47.886477+0000","last_became_peered":"2024-03-08T06:01:47.886477+0000","last_unstale":"2024-03-08T06:01:47.894567+0000","last_undegraded":"2024-03-08T06:01:47.894567+0000","last_fullsized":"2024-03-08T06:01:47.894567+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:39:55.488990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.889761+0000","last_change":"2024-03-08T06:01:46.893559+0000","last_active":"2024-03-08T06:01:47.889761+0000","last_peered":"2024-03-08T06:01:47.889761+0000","last_clean":"2024-03-08T06:01:47.889761+0000","last_became_active":"2024-03-08T06:01:46.893295+0000","last_became_peered":"2024-03-08T06:01:46.893295+0000","last_unstale":"2024-03-08T06:01:47.889761+0000","last_undegraded":"2024-03-08T06:01:47.889761+0000","last_fullsized":"2024-03-08T06:01:47.889761+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:25:20.026788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.878982+0000","last_change":"2024-03-08T06:01:47.891898+0000","last_active":"2024-03-08T06:01:48.878982+0000","last_peered":"2024-03-08T06:01:48.878982+0000","last_clean":"2024-03-08T06:01:48.878982+0000","last_became_active":"2024-03-08T06:01:47.891620+0000","last_became_peered":"2024-03-08T06:01:47.891620+0000","last_unstale":"2024-03-08T06:01:48.878982+0000","last_undegraded":"2024-03-08T06:01:48.878982+0000","last_fullsized":"2024-03-08T06:01:48.878982+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:16:20.468512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.885294+0000","last_change":"2024-03-08T06:01:46.887924+0000","last_active":"2024-03-08T06:01:47.885294+0000","last_peered":"2024-03-08T06:01:47.885294+0000","last_clean":"2024-03-08T06:01:47.885294+0000","last_became_active":"2024-03-08T06:01:46.887652+0000","last_became_peered":"2024-03-08T06:01:46.887652+0000","last_unstale":"2024-03-08T06:01:47.885294+0000","last_undegraded":"2024-03-08T06:01:47.885294+0000","last_fullsized":"2024-03-08T06:01:47.885294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:52:50.743455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.906959+0000","last_change":"2024-03-08T06:01:47.899190+0000","last_active":"2024-03-08T06:01:47.906959+0000","last_peered":"2024-03-08T06:01:47.906959+0000","last_clean":"2024-03-08T06:01:47.906959+0000","last_became_active":"2024-03-08T06:01:47.898926+0000","last_became_peered":"2024-03-08T06:01:47.898926+0000","last_unstale":"2024-03-08T06:01:47.906959+0000","last_undegraded":"2024-03-08T06:01:47.906959+0000","last_fullsized":"2024-03-08T06:01:47.906959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:39:55.488990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.912421+0000","last_change":"2024-03-08T06:01:46.896074+0000","last_active":"2024-03-08T06:01:47.912421+0000","last_peered":"2024-03-08T06:01:47.912421+0000","last_clean":"2024-03-08T06:01:47.912421+0000","last_became_active":"2024-03-08T06:01:46.895749+0000","last_became_peered":"2024-03-08T06:01:46.895749+0000","last_unstale":"2024-03-08T06:01:47.912421+0000","last_undegraded":"2024-03-08T06:01:47.912421+0000","last_fullsized":"2024-03-08T06:01:47.912421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:06:46.976776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.909831+0000","last_change":"2024-03-08T06:01:47.899886+0000","last_active":"2024-03-08T06:01:47.909831+0000","last_peered":"2024-03-08T06:01:47.909831+0000","last_clean":"2024-03-08T06:01:47.909831+0000","last_became_active":"2024-03-08T06:01:47.899574+0000","last_became_peered":"2024-03-08T06:01:47.899574+0000","last_unstale":"2024-03-08T06:01:47.909831+0000","last_undegraded":"2024-03-08T06:01:47.909831+0000","last_fullsized":"2024-03-08T06:01:47.909831+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:45:17.531673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.888454+0000","last_change":"2024-03-08T06:01:46.892634+0000","last_active":"2024-03-08T06:01:47.888454+0000","last_peered":"2024-03-08T06:01:47.888454+0000","last_clean":"2024-03-08T06:01:47.888454+0000","last_became_active":"2024-03-08T06:01:46.892390+0000","last_became_peered":"2024-03-08T06:01:46.892390+0000","last_unstale":"2024-03-08T06:01:47.888454+0000","last_undegraded":"2024-03-08T06:01:47.888454+0000","last_fullsized":"2024-03-08T06:01:47.888454+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:45:35.630850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879180+0000","last_change":"2024-03-08T06:01:47.886595+0000","last_active":"2024-03-08T06:01:48.879180+0000","last_peered":"2024-03-08T06:01:48.879180+0000","last_clean":"2024-03-08T06:01:48.879180+0000","last_became_active":"2024-03-08T06:01:47.886348+0000","last_became_peered":"2024-03-08T06:01:47.886348+0000","last_unstale":"2024-03-08T06:01:48.879180+0000","last_undegraded":"2024-03-08T06:01:48.879180+0000","last_fullsized":"2024-03-08T06:01:48.879180+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:21:21.582227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.887606+0000","last_change":"2024-03-08T06:01:46.887951+0000","last_active":"2024-03-08T06:01:47.887606+0000","last_peered":"2024-03-08T06:01:47.887606+0000","last_clean":"2024-03-08T06:01:47.887606+0000","last_became_active":"2024-03-08T06:01:46.887709+0000","last_became_peered":"2024-03-08T06:01:46.887709+0000","last_unstale":"2024-03-08T06:01:47.887606+0000","last_undegraded":"2024-03-08T06:01:47.887606+0000","last_fullsized":"2024-03-08T06:01:47.887606+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:18:59.639546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.897226+0000","last_change":"2024-03-08T06:01:47.894597+0000","last_active":"2024-03-08T06:01:47.897226+0000","last_peered":"2024-03-08T06:01:47.897226+0000","last_clean":"2024-03-08T06:01:47.897226+0000","last_became_active":"2024-03-08T06:01:47.894333+0000","last_became_peered":"2024-03-08T06:01:47.894333+0000","last_unstale":"2024-03-08T06:01:47.897226+0000","last_undegraded":"2024-03-08T06:01:47.897226+0000","last_fullsized":"2024-03-08T06:01:47.897226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:06:47.982479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.887037+0000","last_change":"2024-03-08T06:01:46.886717+0000","last_active":"2024-03-08T06:01:47.887037+0000","last_peered":"2024-03-08T06:01:47.887037+0000","last_clean":"2024-03-08T06:01:47.887037+0000","last_became_active":"2024-03-08T06:01:46.886354+0000","last_became_peered":"2024-03-08T06:01:46.886354+0000","last_unstale":"2024-03-08T06:01:47.887037+0000","last_undegraded":"2024-03-08T06:01:47.887037+0000","last_fullsized":"2024-03-08T06:01:47.887037+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:02:07.827318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.896050+0000","last_change":"2024-03-08T06:01:47.894097+0000","last_active":"2024-03-08T06:01:47.896050+0000","last_peered":"2024-03-08T06:01:47.896050+0000","last_clean":"2024-03-08T06:01:47.896050+0000","last_became_active":"2024-03-08T06:01:47.893838+0000","last_became_peered":"2024-03-08T06:01:47.893838+0000","last_unstale":"2024-03-08T06:01:47.896050+0000","last_undegraded":"2024-03-08T06:01:47.896050+0000","last_fullsized":"2024-03-08T06:01:47.896050+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:02.674261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.885054+0000","last_change":"2024-03-08T06:01:46.887914+0000","last_active":"2024-03-08T06:01:47.885054+0000","last_peered":"2024-03-08T06:01:47.885054+0000","last_clean":"2024-03-08T06:01:47.885054+0000","last_became_active":"2024-03-08T06:01:46.887654+0000","last_became_peered":"2024-03-08T06:01:46.887654+0000","last_unstale":"2024-03-08T06:01:47.885054+0000","last_undegraded":"2024-03-08T06:01:47.885054+0000","last_fullsized":"2024-03-08T06:01:47.885054+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:08:00.989881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879703+0000","last_change":"2024-03-08T06:01:47.887417+0000","last_active":"2024-03-08T06:01:48.879703+0000","last_peered":"2024-03-08T06:01:48.879703+0000","last_clean":"2024-03-08T06:01:48.879703+0000","last_became_active":"2024-03-08T06:01:47.887133+0000","last_became_peered":"2024-03-08T06:01:47.887133+0000","last_unstale":"2024-03-08T06:01:48.879703+0000","last_undegraded":"2024-03-08T06:01:48.879703+0000","last_fullsized":"2024-03-08T06:01:48.879703+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:15.275207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879477+0000","last_change":"2024-03-08T06:01:46.895158+0000","last_active":"2024-03-08T06:01:48.879477+0000","last_peered":"2024-03-08T06:01:48.879477+0000","last_clean":"2024-03-08T06:01:48.879477+0000","last_became_active":"2024-03-08T06:01:46.894873+0000","last_became_peered":"2024-03-08T06:01:46.894873+0000","last_unstale":"2024-03-08T06:01:48.879477+0000","last_undegraded":"2024-03-08T06:01:48.879477+0000","last_fullsized":"2024-03-08T06:01:48.879477+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:34:32.641931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910269+0000","last_change":"2024-03-08T06:01:47.906862+0000","last_active":"2024-03-08T06:01:47.910269+0000","last_peered":"2024-03-08T06:01:47.910269+0000","last_clean":"2024-03-08T06:01:47.910269+0000","last_became_active":"2024-03-08T06:01:47.905456+0000","last_became_peered":"2024-03-08T06:01:47.905456+0000","last_unstale":"2024-03-08T06:01:47.910269+0000","last_undegraded":"2024-03-08T06:01:47.910269+0000","last_fullsized":"2024-03-08T06:01:47.910269+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:28.902386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.878725+0000","last_change":"2024-03-08T06:01:46.892083+0000","last_active":"2024-03-08T06:01:48.878725+0000","last_peered":"2024-03-08T06:01:48.878725+0000","last_clean":"2024-03-08T06:01:48.878725+0000","last_became_active":"2024-03-08T06:01:46.891339+0000","last_became_peered":"2024-03-08T06:01:46.891339+0000","last_unstale":"2024-03-08T06:01:48.878725+0000","last_undegraded":"2024-03-08T06:01:48.878725+0000","last_fullsized":"2024-03-08T06:01:48.878725+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:24:58.571144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.904605+0000","last_change":"2024-03-08T06:01:47.901247+0000","last_active":"2024-03-08T06:01:47.904605+0000","last_peered":"2024-03-08T06:01:47.904605+0000","last_clean":"2024-03-08T06:01:47.904605+0000","last_became_active":"2024-03-08T06:01:47.900913+0000","last_became_peered":"2024-03-08T06:01:47.900913+0000","last_unstale":"2024-03-08T06:01:47.904605+0000","last_undegraded":"2024-03-08T06:01:47.904605+0000","last_fullsized":"2024-03-08T06:01:47.904605+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:36:10.058410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.888937+0000","last_change":"2024-03-08T06:01:46.897282+0000","last_active":"2024-03-08T06:01:47.888937+0000","last_peered":"2024-03-08T06:01:47.888937+0000","last_clean":"2024-03-08T06:01:47.888937+0000","last_became_active":"2024-03-08T06:01:46.897100+0000","last_became_peered":"2024-03-08T06:01:46.897100+0000","last_unstale":"2024-03-08T06:01:47.888937+0000","last_undegraded":"2024-03-08T06:01:47.888937+0000","last_fullsized":"2024-03-08T06:01:47.888937+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:09:41.301850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":203,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.888609+0000","last_change":"2024-03-08T06:00:26.178419+0000","last_active":"2024-03-08T06:01:47.888609+0000","last_peered":"2024-03-08T06:01:47.888609+0000","last_clean":"2024-03-08T06:01:47.888609+0000","last_became_active":"2024-03-08T06:00:26.175826+0000","last_became_peered":"2024-03-08T06:00:26.175826+0000","last_unstale":"2024-03-08T06:01:47.888609+0000","last_undegraded":"2024-03-08T06:01:47.888609+0000","last_fullsized":"2024-03-08T06:01:47.888609+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_clean_scrub_stamp":"2024-03-08T06:00:24.029492+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:36:00.682517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1060864,"data_stored":787572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":196104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":195634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":196126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":197168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":195592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":195612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":785924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:01:50.854 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:01:50.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.989+0000 7ff81946f640 1 Processor -- start 2024-03-08T06:01:50.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.989+0000 7ff81946f640 1 -- start start 2024-03-08T06:01:50.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.990+0000 7ff81946f640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff81415bcd0 0x7ff8140cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.990+0000 7ff81946f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff81409a250 0x7ff8140cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.990+0000 7ff81946f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8140ccbb0 con 0x7ff8140cf730 2024-03-08T06:01:50.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.990+0000 7ff81946f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8140cce90 con 0x7ff81409a250 2024-03-08T06:01:50.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.990+0000 7ff81946f640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8140cd170 con 0x7ff81415bcd0 2024-03-08T06:01:50.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.990+0000 7ff8137fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff81409a250 0x7ff8140cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.990+0000 7ff8137fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff81409a250 0x7ff8140cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51300/0 (socket says 172.21.15.76:51300) 2024-03-08T06:01:50.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.990+0000 7ff8137fe640 1 -- 172.21.15.76:0/3338547197 learned_addr learned my addr 172.21.15.76:0/3338547197 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:50.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.990+0000 7ff812ffd640 1 --2- 172.21.15.76:0/3338547197 >> v2:172.21.15.81:3300/0 conn(0x7ff81415bcd0 0x7ff8140cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.990+0000 7ff8137fe640 1 -- 172.21.15.76:0/3338547197 >> v2:172.21.15.81:3300/0 conn(0x7ff81415bcd0 msgr2=0x7ff8140cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:50.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.990+0000 7ff8137fe640 1 --2- 172.21.15.76:0/3338547197 >> v2:172.21.15.81:3300/0 conn(0x7ff81415bcd0 0x7ff8140cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.990+0000 7ff8137fe640 1 -- 172.21.15.76:0/3338547197 >> v1:172.21.15.81:6789/0 conn(0x7ff8140cf730 legacy=0x7ff814099b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.990+0000 7ff8137fe640 1 -- 172.21.15.76:0/3338547197 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8140cd450 con 0x7ff81409a250 2024-03-08T06:01:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.991+0000 7ff8137fe640 1 --2- 172.21.15.76:0/3338547197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff81409a250 0x7ff8140cdc50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff814072cb0 tx=0x7ff80c02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505dbf773414966e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.991+0000 7ff811ffb640 1 -- 172.21.15.76:0/3338547197 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c02cbb0 con 0x7ff81409a250 2024-03-08T06:01:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.991+0000 7ff811ffb640 1 -- 172.21.15.76:0/3338547197 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80c02cd50 con 0x7ff81409a250 2024-03-08T06:01:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.991+0000 7ff811ffb640 1 -- 172.21.15.76:0/3338547197 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c034610 con 0x7ff81409a250 2024-03-08T06:01:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.992+0000 7ff81946f640 1 -- 172.21.15.76:0/3338547197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff81409a250 msgr2=0x7ff8140cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.992+0000 7ff81946f640 1 --2- 172.21.15.76:0/3338547197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff81409a250 0x7ff8140cdc50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff814072cb0 tx=0x7ff80c02c010 comp rx=0 tx=0).stop 2024-03-08T06:01:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.992+0000 7ff81946f640 1 -- 172.21.15.76:0/3338547197 shutdown_connections 2024-03-08T06:01:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.992+0000 7ff81946f640 1 --2- 172.21.15.76:0/3338547197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff81409a250 0x7ff8140cdc50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.992+0000 7ff81946f640 1 --2- 172.21.15.76:0/3338547197 >> v2:172.21.15.81:3300/0 conn(0x7ff81415bcd0 0x7ff8140cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.992+0000 7ff81946f640 1 -- 172.21.15.76:0/3338547197 >> 172.21.15.76:0/3338547197 conn(0x7ff8140b3440 msgr2=0x7ff8140b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:50.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.992+0000 7ff81946f640 1 -- 172.21.15.76:0/3338547197 shutdown_connections 2024-03-08T06:01:50.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.992+0000 7ff81946f640 1 -- 172.21.15.76:0/3338547197 wait complete. 2024-03-08T06:01:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.993+0000 7ff81946f640 1 Processor -- start 2024-03-08T06:01:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.993+0000 7ff81946f640 1 -- start start 2024-03-08T06:01:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff81946f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff81409a250 0x7ff81415e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff81946f640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff8140cf730 0x7ff8141f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff81946f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff81415bcd0 0x7ff8141f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff81946f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81409b160 con 0x7ff81415bcd0 2024-03-08T06:01:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff81946f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81415c0b0 con 0x7ff81409a250 2024-03-08T06:01:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff81946f640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81409b2e0 con 0x7ff8140cf730 2024-03-08T06:01:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff8137fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff81415bcd0 0x7ff8141f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff8137fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff81415bcd0 0x7ff8141f3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51302/0 (socket says 172.21.15.76:51302) 2024-03-08T06:01:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff8137fe640 1 -- 172.21.15.76:0/2926644221 learned_addr learned my addr 172.21.15.76:0/2926644221 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff812ffd640 1 --2- 172.21.15.76:0/2926644221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff81409a250 0x7ff81415e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff8127fc640 1 --2- 172.21.15.76:0/2926644221 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff8140cf730 0x7ff8141f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff8137fe640 1 -- 172.21.15.76:0/2926644221 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff8140cf730 msgr2=0x7ff8141f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff8137fe640 1 --2- 172.21.15.76:0/2926644221 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff8140cf730 0x7ff8141f3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff8137fe640 1 -- 172.21.15.76:0/2926644221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff81409a250 msgr2=0x7ff81415e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff8137fe640 1 --2- 172.21.15.76:0/2926644221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff81409a250 0x7ff81415e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff8137fe640 1 -- 172.21.15.76:0/2926644221 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff814071450 con 0x7ff81415bcd0 2024-03-08T06:01:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.994+0000 7ff8127fc640 1 --2- 172.21.15.76:0/2926644221 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff8140cf730 0x7ff8141f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.995+0000 7ff8137fe640 1 --2- 172.21.15.76:0/2926644221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff81415bcd0 0x7ff8141f3d80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff80c036040 tx=0x7ff80c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.995+0000 7ff7fb7fe640 1 -- 172.21.15.76:0/2926644221 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c043020 con 0x7ff81415bcd0 2024-03-08T06:01:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.995+0000 7ff7fb7fe640 1 -- 172.21.15.76:0/2926644221 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80c03e450 con 0x7ff81415bcd0 2024-03-08T06:01:50.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.995+0000 7ff81946f640 1 -- 172.21.15.76:0/2926644221 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81415cf80 con 0x7ff81415bcd0 2024-03-08T06:01:50.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.995+0000 7ff7fb7fe640 1 -- 172.21.15.76:0/2926644221 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c009510 con 0x7ff81415bcd0 2024-03-08T06:01:50.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.995+0000 7ff81946f640 1 -- 172.21.15.76:0/2926644221 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81415d520 con 0x7ff81415bcd0 2024-03-08T06:01:50.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.996+0000 7ff81946f640 1 -- 172.21.15.76:0/2926644221 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81404f030 con 0x7ff81415bcd0 2024-03-08T06:01:50.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.996+0000 7ff7fb7fe640 1 -- 172.21.15.76:0/2926644221 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff80c046020 con 0x7ff81415bcd0 2024-03-08T06:01:50.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.996+0000 7ff7fb7fe640 1 --2- 172.21.15.76:0/2926644221 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff80006b4d0 0x7ff80006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:50.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.997+0000 7ff812ffd640 1 --2- 172.21.15.76:0/2926644221 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff80006b4d0 0x7ff80006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:50.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.997+0000 7ff7fb7fe640 1 -- 172.21.15.76:0/2926644221 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff80c0acab0 con 0x7ff81415bcd0 2024-03-08T06:01:50.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:50.997+0000 7ff812ffd640 1 --2- 172.21.15.76:0/2926644221 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff80006b4d0 0x7ff80006d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff804006fd0 tx=0x7ff804008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.001+0000 7ff7fb7fe640 1 -- 172.21.15.76:0/2926644221 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff80c04d030 con 0x7ff81415bcd0 2024-03-08T06:01:51.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.170+0000 7ff81946f640 1 -- 172.21.15.76:0/2926644221 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff81409a670 con 0x7ff80006b4d0 2024-03-08T06:01:51.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.175+0000 7ff7fb7fe640 1 -- 172.21.15.76:0/2926644221 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251903 (secure 0 0 0) 0x7ff81409a670 con 0x7ff80006b4d0 2024-03-08T06:01:51.178 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:51.179 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-08T06:01:51.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.178+0000 7ff81946f640 1 -- 172.21.15.76:0/2926644221 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff80006b4d0 msgr2=0x7ff80006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:51.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.178+0000 7ff81946f640 1 --2- 172.21.15.76:0/2926644221 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff80006b4d0 0x7ff80006d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff804006fd0 tx=0x7ff804008040 comp rx=0 tx=0).stop 2024-03-08T06:01:51.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.178+0000 7ff81946f640 1 -- 172.21.15.76:0/2926644221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff81415bcd0 msgr2=0x7ff8141f3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:51.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.178+0000 7ff81946f640 1 --2- 172.21.15.76:0/2926644221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff81415bcd0 0x7ff8141f3d80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff80c036040 tx=0x7ff80c004a40 comp rx=0 tx=0).stop 2024-03-08T06:01:51.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.179+0000 7ff81946f640 1 -- 172.21.15.76:0/2926644221 shutdown_connections 2024-03-08T06:01:51.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.179+0000 7ff81946f640 1 --2- 172.21.15.76:0/2926644221 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff80006b4d0 0x7ff80006d990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.179+0000 7ff81946f640 1 --2- 172.21.15.76:0/2926644221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff81415bcd0 0x7ff8141f3d80 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.179+0000 7ff81946f640 1 --2- 172.21.15.76:0/2926644221 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff8140cf730 0x7ff8141f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.179+0000 7ff81946f640 1 --2- 172.21.15.76:0/2926644221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff81409a250 0x7ff81415e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.179+0000 7ff81946f640 1 -- 172.21.15.76:0/2926644221 >> 172.21.15.76:0/2926644221 conn(0x7ff8140b3440 msgr2=0x7ff814099b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:51.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.179+0000 7ff81946f640 1 -- 172.21.15.76:0/2926644221 shutdown_connections 2024-03-08T06:01:51.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.179+0000 7ff81946f640 1 -- 172.21.15.76:0/2926644221 wait complete. 2024-03-08T06:01:51.190 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-03-08T06:01:49.935494+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220960,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753760,"statfs":{"total":773094113280,"available":772867850240,"internally_reserved":0,"allocated":4857856,"data_stored":2749732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.066288"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879211+0000","last_change":"2024-03-08T06:01:47.898935+0000","last_active":"2024-03-08T06:01:48.879211+0000","last_peered":"2024-03-08T06:01:48.879211+0000","last_clean":"2024-03-08T06:01:48.879211+0000","last_became_active":"2024-03-08T06:01:47.898668+0000","last_became_peered":"2024-03-08T06:01:47.898668+0000","last_unstale":"2024-03-08T06:01:48.879211+0000","last_undegraded":"2024-03-08T06:01:48.879211+0000","last_fullsized":"2024-03-08T06:01:48.879211+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:06:30.869091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879054+0000","last_change":"2024-03-08T06:01:47.896934+0000","last_active":"2024-03-08T06:01:48.879054+0000","last_peered":"2024-03-08T06:01:48.879054+0000","last_clean":"2024-03-08T06:01:48.879054+0000","last_became_active":"2024-03-08T06:01:47.896015+0000","last_became_peered":"2024-03-08T06:01:47.896015+0000","last_unstale":"2024-03-08T06:01:48.879054+0000","last_undegraded":"2024-03-08T06:01:48.879054+0000","last_fullsized":"2024-03-08T06:01:48.879054+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:16:48.835489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.906214+0000","last_change":"2024-03-08T06:01:47.900201+0000","last_active":"2024-03-08T06:01:47.906214+0000","last_peered":"2024-03-08T06:01:47.906214+0000","last_clean":"2024-03-08T06:01:47.906214+0000","last_became_active":"2024-03-08T06:01:47.899953+0000","last_became_peered":"2024-03-08T06:01:47.899953+0000","last_unstale":"2024-03-08T06:01:47.906214+0000","last_undegraded":"2024-03-08T06:01:47.906214+0000","last_fullsized":"2024-03-08T06:01:47.906214+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:33:00.231357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.903790+0000","last_change":"2024-03-08T06:01:47.897290+0000","last_active":"2024-03-08T06:01:47.903790+0000","last_peered":"2024-03-08T06:01:47.903790+0000","last_clean":"2024-03-08T06:01:47.903790+0000","last_became_active":"2024-03-08T06:01:47.897039+0000","last_became_peered":"2024-03-08T06:01:47.897039+0000","last_unstale":"2024-03-08T06:01:47.903790+0000","last_undegraded":"2024-03-08T06:01:47.903790+0000","last_fullsized":"2024-03-08T06:01:47.903790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:58:42.629432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.903794+0000","last_change":"2024-03-08T06:01:47.896786+0000","last_active":"2024-03-08T06:01:47.903794+0000","last_peered":"2024-03-08T06:01:47.903794+0000","last_clean":"2024-03-08T06:01:47.903794+0000","last_became_active":"2024-03-08T06:01:47.896623+0000","last_became_peered":"2024-03-08T06:01:47.896623+0000","last_unstale":"2024-03-08T06:01:47.903794+0000","last_undegraded":"2024-03-08T06:01:47.903794+0000","last_fullsized":"2024-03-08T06:01:47.903794+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:17:26.545703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.894260+0000","last_change":"2024-03-08T06:01:47.890376+0000","last_active":"2024-03-08T06:01:47.894260+0000","last_peered":"2024-03-08T06:01:47.894260+0000","last_clean":"2024-03-08T06:01:47.894260+0000","last_became_active":"2024-03-08T06:01:47.890103+0000","last_became_peered":"2024-03-08T06:01:47.890103+0000","last_unstale":"2024-03-08T06:01:47.894260+0000","last_undegraded":"2024-03-08T06:01:47.894260+0000","last_fullsized":"2024-03-08T06:01:47.894260+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:15.316795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.897566+0000","last_change":"2024-03-08T06:01:47.893733+0000","last_active":"2024-03-08T06:01:47.897566+0000","last_peered":"2024-03-08T06:01:47.897566+0000","last_clean":"2024-03-08T06:01:47.897566+0000","last_became_active":"2024-03-08T06:01:47.893022+0000","last_became_peered":"2024-03-08T06:01:47.893022+0000","last_unstale":"2024-03-08T06:01:47.897566+0000","last_undegraded":"2024-03-08T06:01:47.897566+0000","last_fullsized":"2024-03-08T06:01:47.897566+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:15.275207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.878688+0000","last_change":"2024-03-08T06:01:47.896697+0000","last_active":"2024-03-08T06:01:48.878688+0000","last_peered":"2024-03-08T06:01:48.878688+0000","last_clean":"2024-03-08T06:01:48.878688+0000","last_became_active":"2024-03-08T06:01:47.896436+0000","last_became_peered":"2024-03-08T06:01:47.896436+0000","last_unstale":"2024-03-08T06:01:48.878688+0000","last_undegraded":"2024-03-08T06:01:48.878688+0000","last_fullsized":"2024-03-08T06:01:48.878688+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:47:33.608607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.909075+0000","last_change":"2024-03-08T06:01:47.902949+0000","last_active":"2024-03-08T06:01:47.909075+0000","last_peered":"2024-03-08T06:01:47.909075+0000","last_clean":"2024-03-08T06:01:47.909075+0000","last_became_active":"2024-03-08T06:01:47.902690+0000","last_became_peered":"2024-03-08T06:01:47.902690+0000","last_unstale":"2024-03-08T06:01:47.909075+0000","last_undegraded":"2024-03-08T06:01:47.909075+0000","last_fullsized":"2024-03-08T06:01:47.909075+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:42:33.782761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.901875+0000","last_change":"2024-03-08T06:01:47.896269+0000","last_active":"2024-03-08T06:01:47.901875+0000","last_peered":"2024-03-08T06:01:47.901875+0000","last_clean":"2024-03-08T06:01:47.901875+0000","last_became_active":"2024-03-08T06:01:47.893623+0000","last_became_peered":"2024-03-08T06:01:47.893623+0000","last_unstale":"2024-03-08T06:01:47.901875+0000","last_undegraded":"2024-03-08T06:01:47.901875+0000","last_fullsized":"2024-03-08T06:01:47.901875+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:24:59.576847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.898254+0000","last_change":"2024-03-08T06:01:47.895173+0000","last_active":"2024-03-08T06:01:47.898254+0000","last_peered":"2024-03-08T06:01:47.898254+0000","last_clean":"2024-03-08T06:01:47.898254+0000","last_became_active":"2024-03-08T06:01:47.894860+0000","last_became_peered":"2024-03-08T06:01:47.894860+0000","last_unstale":"2024-03-08T06:01:47.898254+0000","last_undegraded":"2024-03-08T06:01:47.898254+0000","last_fullsized":"2024-03-08T06:01:47.898254+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:26:23.550550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.901071+0000","last_change":"2024-03-08T06:01:47.897828+0000","last_active":"2024-03-08T06:01:47.901071+0000","last_peered":"2024-03-08T06:01:47.901071+0000","last_clean":"2024-03-08T06:01:47.901071+0000","last_became_active":"2024-03-08T06:01:47.897575+0000","last_became_peered":"2024-03-08T06:01:47.897575+0000","last_unstale":"2024-03-08T06:01:47.901071+0000","last_undegraded":"2024-03-08T06:01:47.901071+0000","last_fullsized":"2024-03-08T06:01:47.901071+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:28.902386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.900199+0000","last_change":"2024-03-08T06:01:47.891993+0000","last_active":"2024-03-08T06:01:47.900199+0000","last_peered":"2024-03-08T06:01:47.900199+0000","last_clean":"2024-03-08T06:01:47.900199+0000","last_became_active":"2024-03-08T06:01:47.891669+0000","last_became_peered":"2024-03-08T06:01:47.891669+0000","last_unstale":"2024-03-08T06:01:47.900199+0000","last_undegraded":"2024-03-08T06:01:47.900199+0000","last_fullsized":"2024-03-08T06:01:47.900199+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:33:00.231357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879841+0000","last_change":"2024-03-08T06:01:47.893422+0000","last_active":"2024-03-08T06:01:48.879841+0000","last_peered":"2024-03-08T06:01:48.879841+0000","last_clean":"2024-03-08T06:01:48.879841+0000","last_became_active":"2024-03-08T06:01:47.893171+0000","last_became_peered":"2024-03-08T06:01:47.893171+0000","last_unstale":"2024-03-08T06:01:48.879841+0000","last_undegraded":"2024-03-08T06:01:48.879841+0000","last_fullsized":"2024-03-08T06:01:48.879841+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:53:30.002460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.899543+0000","last_change":"2024-03-08T06:01:47.893939+0000","last_active":"2024-03-08T06:01:47.899543+0000","last_peered":"2024-03-08T06:01:47.899543+0000","last_clean":"2024-03-08T06:01:47.899543+0000","last_became_active":"2024-03-08T06:01:47.893406+0000","last_became_peered":"2024-03-08T06:01:47.893406+0000","last_unstale":"2024-03-08T06:01:47.899543+0000","last_undegraded":"2024-03-08T06:01:47.899543+0000","last_fullsized":"2024-03-08T06:01:47.899543+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:53:30.002460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.909907+0000","last_change":"2024-03-08T06:01:47.899729+0000","last_active":"2024-03-08T06:01:47.909907+0000","last_peered":"2024-03-08T06:01:47.909907+0000","last_clean":"2024-03-08T06:01:47.909907+0000","last_became_active":"2024-03-08T06:01:47.898946+0000","last_became_peered":"2024-03-08T06:01:47.898946+0000","last_unstale":"2024-03-08T06:01:47.909907+0000","last_undegraded":"2024-03-08T06:01:47.909907+0000","last_fullsized":"2024-03-08T06:01:47.909907+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:02:08.833021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910249+0000","last_change":"2024-03-08T06:01:47.904656+0000","last_active":"2024-03-08T06:01:47.910249+0000","last_peered":"2024-03-08T06:01:47.910249+0000","last_clean":"2024-03-08T06:01:47.910249+0000","last_became_active":"2024-03-08T06:01:47.904385+0000","last_became_peered":"2024-03-08T06:01:47.904385+0000","last_unstale":"2024-03-08T06:01:47.910249+0000","last_undegraded":"2024-03-08T06:01:47.910249+0000","last_fullsized":"2024-03-08T06:01:47.910249+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:15.316795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.904787+0000","last_change":"2024-03-08T06:01:47.901117+0000","last_active":"2024-03-08T06:01:47.904787+0000","last_peered":"2024-03-08T06:01:47.904787+0000","last_clean":"2024-03-08T06:01:47.904787+0000","last_became_active":"2024-03-08T06:01:47.900784+0000","last_became_peered":"2024-03-08T06:01:47.900784+0000","last_unstale":"2024-03-08T06:01:47.904787+0000","last_undegraded":"2024-03-08T06:01:47.904787+0000","last_fullsized":"2024-03-08T06:01:47.904787+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:45:17.531673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.901114+0000","last_change":"2024-03-08T06:01:47.893890+0000","last_active":"2024-03-08T06:01:47.901114+0000","last_peered":"2024-03-08T06:01:47.901114+0000","last_clean":"2024-03-08T06:01:47.901114+0000","last_became_active":"2024-03-08T06:01:47.893642+0000","last_became_peered":"2024-03-08T06:01:47.893642+0000","last_unstale":"2024-03-08T06:01:47.901114+0000","last_undegraded":"2024-03-08T06:01:47.901114+0000","last_fullsized":"2024-03-08T06:01:47.901114+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:46:39.400065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.904062+0000","last_change":"2024-03-08T06:01:47.896780+0000","last_active":"2024-03-08T06:01:47.904062+0000","last_peered":"2024-03-08T06:01:47.904062+0000","last_clean":"2024-03-08T06:01:47.904062+0000","last_became_active":"2024-03-08T06:01:47.896517+0000","last_became_peered":"2024-03-08T06:01:47.896517+0000","last_unstale":"2024-03-08T06:01:47.904062+0000","last_undegraded":"2024-03-08T06:01:47.904062+0000","last_fullsized":"2024-03-08T06:01:47.904062+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:09:42.307553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.904674+0000","last_change":"2024-03-08T06:01:47.900637+0000","last_active":"2024-03-08T06:01:47.904674+0000","last_peered":"2024-03-08T06:01:47.904674+0000","last_clean":"2024-03-08T06:01:47.904674+0000","last_became_active":"2024-03-08T06:01:47.900340+0000","last_became_peered":"2024-03-08T06:01:47.900340+0000","last_unstale":"2024-03-08T06:01:47.904674+0000","last_undegraded":"2024-03-08T06:01:47.904674+0000","last_fullsized":"2024-03-08T06:01:47.904674+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:59:52.595386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.880118+0000","last_change":"2024-03-08T06:01:47.894955+0000","last_active":"2024-03-08T06:01:48.880118+0000","last_peered":"2024-03-08T06:01:48.880118+0000","last_clean":"2024-03-08T06:01:48.880118+0000","last_became_active":"2024-03-08T06:01:47.894679+0000","last_became_peered":"2024-03-08T06:01:47.894679+0000","last_unstale":"2024-03-08T06:01:48.880118+0000","last_undegraded":"2024-03-08T06:01:48.880118+0000","last_fullsized":"2024-03-08T06:01:48.880118+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:11:16.081832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910138+0000","last_change":"2024-03-08T06:01:47.904167+0000","last_active":"2024-03-08T06:01:47.910138+0000","last_peered":"2024-03-08T06:01:47.910138+0000","last_clean":"2024-03-08T06:01:47.910138+0000","last_became_active":"2024-03-08T06:01:47.903877+0000","last_became_peered":"2024-03-08T06:01:47.903877+0000","last_unstale":"2024-03-08T06:01:47.910138+0000","last_undegraded":"2024-03-08T06:01:47.910138+0000","last_fullsized":"2024-03-08T06:01:47.910138+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:19:00.645250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.901088+0000","last_change":"2024-03-08T06:01:47.893770+0000","last_active":"2024-03-08T06:01:47.901088+0000","last_peered":"2024-03-08T06:01:47.901088+0000","last_clean":"2024-03-08T06:01:47.901088+0000","last_became_active":"2024-03-08T06:01:47.893532+0000","last_became_peered":"2024-03-08T06:01:47.893532+0000","last_unstale":"2024-03-08T06:01:47.901088+0000","last_undegraded":"2024-03-08T06:01:47.901088+0000","last_fullsized":"2024-03-08T06:01:47.901088+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:33:16.443151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.904826+0000","last_change":"2024-03-08T06:01:47.900296+0000","last_active":"2024-03-08T06:01:47.904826+0000","last_peered":"2024-03-08T06:01:47.904826+0000","last_clean":"2024-03-08T06:01:47.904826+0000","last_became_active":"2024-03-08T06:01:47.897905+0000","last_became_peered":"2024-03-08T06:01:47.897905+0000","last_unstale":"2024-03-08T06:01:47.904826+0000","last_undegraded":"2024-03-08T06:01:47.904826+0000","last_fullsized":"2024-03-08T06:01:47.904826+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:53:52.111951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.907041+0000","last_change":"2024-03-08T06:01:47.901017+0000","last_active":"2024-03-08T06:01:47.907041+0000","last_peered":"2024-03-08T06:01:47.907041+0000","last_clean":"2024-03-08T06:01:47.907041+0000","last_became_active":"2024-03-08T06:01:47.900760+0000","last_became_peered":"2024-03-08T06:01:47.900760+0000","last_unstale":"2024-03-08T06:01:47.907041+0000","last_undegraded":"2024-03-08T06:01:47.907041+0000","last_fullsized":"2024-03-08T06:01:47.907041+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:33:00.231357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.904720+0000","last_change":"2024-03-08T06:01:47.899891+0000","last_active":"2024-03-08T06:01:47.904720+0000","last_peered":"2024-03-08T06:01:47.904720+0000","last_clean":"2024-03-08T06:01:47.904720+0000","last_became_active":"2024-03-08T06:01:47.899205+0000","last_became_peered":"2024-03-08T06:01:47.899205+0000","last_unstale":"2024-03-08T06:01:47.904720+0000","last_undegraded":"2024-03-08T06:01:47.904720+0000","last_fullsized":"2024-03-08T06:01:47.904720+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:18:21.554370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.897554+0000","last_change":"2024-03-08T06:01:47.893348+0000","last_active":"2024-03-08T06:01:47.897554+0000","last_peered":"2024-03-08T06:01:47.897554+0000","last_clean":"2024-03-08T06:01:47.897554+0000","last_became_active":"2024-03-08T06:01:47.891691+0000","last_became_peered":"2024-03-08T06:01:47.891691+0000","last_unstale":"2024-03-08T06:01:47.897554+0000","last_undegraded":"2024-03-08T06:01:47.897554+0000","last_fullsized":"2024-03-08T06:01:47.897554+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:42:33.782761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.891914+0000","last_change":"2024-03-08T06:01:47.889974+0000","last_active":"2024-03-08T06:01:47.891914+0000","last_peered":"2024-03-08T06:01:47.891914+0000","last_clean":"2024-03-08T06:01:47.891914+0000","last_became_active":"2024-03-08T06:01:47.889701+0000","last_became_peered":"2024-03-08T06:01:47.889701+0000","last_unstale":"2024-03-08T06:01:47.891914+0000","last_undegraded":"2024-03-08T06:01:47.891914+0000","last_fullsized":"2024-03-08T06:01:47.891914+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:59:30.643039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.904124+0000","last_change":"2024-03-08T06:01:47.897327+0000","last_active":"2024-03-08T06:01:47.904124+0000","last_peered":"2024-03-08T06:01:47.904124+0000","last_clean":"2024-03-08T06:01:47.904124+0000","last_became_active":"2024-03-08T06:01:47.897024+0000","last_became_peered":"2024-03-08T06:01:47.897024+0000","last_unstale":"2024-03-08T06:01:47.904124+0000","last_undegraded":"2024-03-08T06:01:47.904124+0000","last_fullsized":"2024-03-08T06:01:47.904124+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:25:21.032491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910171+0000","last_change":"2024-03-08T06:01:47.906859+0000","last_active":"2024-03-08T06:01:47.910171+0000","last_peered":"2024-03-08T06:01:47.910171+0000","last_clean":"2024-03-08T06:01:47.910171+0000","last_became_active":"2024-03-08T06:01:47.905488+0000","last_became_peered":"2024-03-08T06:01:47.905488+0000","last_unstale":"2024-03-08T06:01:47.910171+0000","last_undegraded":"2024-03-08T06:01:47.910171+0000","last_fullsized":"2024-03-08T06:01:47.910171+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:06:47.982479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.898388+0000","last_change":"2024-03-08T06:01:47.894973+0000","last_active":"2024-03-08T06:01:47.898388+0000","last_peered":"2024-03-08T06:01:47.898388+0000","last_clean":"2024-03-08T06:01:47.898388+0000","last_became_active":"2024-03-08T06:01:47.894725+0000","last_became_peered":"2024-03-08T06:01:47.894725+0000","last_unstale":"2024-03-08T06:01:47.898388+0000","last_undegraded":"2024-03-08T06:01:47.898388+0000","last_fullsized":"2024-03-08T06:01:47.898388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:06:47.982479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.909796+0000","last_change":"2024-03-08T06:01:47.899673+0000","last_active":"2024-03-08T06:01:47.909796+0000","last_peered":"2024-03-08T06:01:47.909796+0000","last_clean":"2024-03-08T06:01:47.909796+0000","last_became_active":"2024-03-08T06:01:47.899383+0000","last_became_peered":"2024-03-08T06:01:47.899383+0000","last_unstale":"2024-03-08T06:01:47.909796+0000","last_undegraded":"2024-03-08T06:01:47.909796+0000","last_fullsized":"2024-03-08T06:01:47.909796+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:53:52.111951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910041+0000","last_change":"2024-03-08T06:01:46.892159+0000","last_active":"2024-03-08T06:01:47.910041+0000","last_peered":"2024-03-08T06:01:47.910041+0000","last_clean":"2024-03-08T06:01:47.910041+0000","last_became_active":"2024-03-08T06:01:46.891861+0000","last_became_peered":"2024-03-08T06:01:46.891861+0000","last_unstale":"2024-03-08T06:01:47.910041+0000","last_undegraded":"2024-03-08T06:01:47.910041+0000","last_fullsized":"2024-03-08T06:01:47.910041+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:03:52.842458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.907093+0000","last_change":"2024-03-08T06:01:47.899191+0000","last_active":"2024-03-08T06:01:47.907093+0000","last_peered":"2024-03-08T06:01:47.907093+0000","last_clean":"2024-03-08T06:01:47.907093+0000","last_became_active":"2024-03-08T06:01:47.898945+0000","last_became_peered":"2024-03-08T06:01:47.898945+0000","last_unstale":"2024-03-08T06:01:47.907093+0000","last_undegraded":"2024-03-08T06:01:47.907093+0000","last_fullsized":"2024-03-08T06:01:47.907093+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:28.902386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910326+0000","last_change":"2024-03-08T06:01:46.894475+0000","last_active":"2024-03-08T06:01:47.910326+0000","last_peered":"2024-03-08T06:01:47.910326+0000","last_clean":"2024-03-08T06:01:47.910326+0000","last_became_active":"2024-03-08T06:01:46.894237+0000","last_became_peered":"2024-03-08T06:01:46.894237+0000","last_unstale":"2024-03-08T06:01:47.910326+0000","last_undegraded":"2024-03-08T06:01:47.910326+0000","last_fullsized":"2024-03-08T06:01:47.910326+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:34:32.641931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879915+0000","last_change":"2024-03-08T06:01:47.892760+0000","last_active":"2024-03-08T06:01:48.879915+0000","last_peered":"2024-03-08T06:01:48.879915+0000","last_clean":"2024-03-08T06:01:48.879915+0000","last_became_active":"2024-03-08T06:01:47.892491+0000","last_became_peered":"2024-03-08T06:01:47.892491+0000","last_unstale":"2024-03-08T06:01:48.879915+0000","last_undegraded":"2024-03-08T06:01:48.879915+0000","last_fullsized":"2024-03-08T06:01:48.879915+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:24:59.576847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910070+0000","last_change":"2024-03-08T06:01:46.893158+0000","last_active":"2024-03-08T06:01:47.910070+0000","last_peered":"2024-03-08T06:01:47.910070+0000","last_clean":"2024-03-08T06:01:47.910070+0000","last_became_active":"2024-03-08T06:01:46.892893+0000","last_became_peered":"2024-03-08T06:01:46.892893+0000","last_unstale":"2024-03-08T06:01:47.910070+0000","last_undegraded":"2024-03-08T06:01:47.910070+0000","last_fullsized":"2024-03-08T06:01:47.910070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:18:59.639546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879148+0000","last_change":"2024-03-08T06:01:47.890649+0000","last_active":"2024-03-08T06:01:48.879148+0000","last_peered":"2024-03-08T06:01:48.879148+0000","last_clean":"2024-03-08T06:01:48.879148+0000","last_became_active":"2024-03-08T06:01:47.890355+0000","last_became_peered":"2024-03-08T06:01:47.890355+0000","last_unstale":"2024-03-08T06:01:48.879148+0000","last_undegraded":"2024-03-08T06:01:48.879148+0000","last_fullsized":"2024-03-08T06:01:48.879148+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:53:38.958047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879917+0000","last_change":"2024-03-08T06:01:46.895277+0000","last_active":"2024-03-08T06:01:48.879917+0000","last_peered":"2024-03-08T06:01:48.879917+0000","last_clean":"2024-03-08T06:01:48.879917+0000","last_became_active":"2024-03-08T06:01:46.894895+0000","last_became_peered":"2024-03-08T06:01:46.894895+0000","last_unstale":"2024-03-08T06:01:48.879917+0000","last_undegraded":"2024-03-08T06:01:48.879917+0000","last_fullsized":"2024-03-08T06:01:48.879917+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:02:07.827318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.897205+0000","last_change":"2024-03-08T06:01:47.894588+0000","last_active":"2024-03-08T06:01:47.897205+0000","last_peered":"2024-03-08T06:01:47.897205+0000","last_clean":"2024-03-08T06:01:47.897205+0000","last_became_active":"2024-03-08T06:01:47.894307+0000","last_became_peered":"2024-03-08T06:01:47.894307+0000","last_unstale":"2024-03-08T06:01:47.897205+0000","last_undegraded":"2024-03-08T06:01:47.897205+0000","last_fullsized":"2024-03-08T06:01:47.897205+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:02.674261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.911715+0000","last_change":"2024-03-08T06:01:46.896374+0000","last_active":"2024-03-08T06:01:47.911715+0000","last_peered":"2024-03-08T06:01:47.911715+0000","last_clean":"2024-03-08T06:01:47.911715+0000","last_became_active":"2024-03-08T06:01:46.896060+0000","last_became_peered":"2024-03-08T06:01:46.896060+0000","last_unstale":"2024-03-08T06:01:47.911715+0000","last_undegraded":"2024-03-08T06:01:47.911715+0000","last_fullsized":"2024-03-08T06:01:47.911715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:27.896682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.902095+0000","last_change":"2024-03-08T06:01:47.897635+0000","last_active":"2024-03-08T06:01:47.902095+0000","last_peered":"2024-03-08T06:01:47.902095+0000","last_clean":"2024-03-08T06:01:47.902095+0000","last_became_active":"2024-03-08T06:01:47.897212+0000","last_became_peered":"2024-03-08T06:01:47.897212+0000","last_unstale":"2024-03-08T06:01:47.902095+0000","last_undegraded":"2024-03-08T06:01:47.902095+0000","last_fullsized":"2024-03-08T06:01:47.902095+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:11:16.081832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.903746+0000","last_change":"2024-03-08T06:01:46.893307+0000","last_active":"2024-03-08T06:01:47.903746+0000","last_peered":"2024-03-08T06:01:47.903746+0000","last_clean":"2024-03-08T06:01:47.903746+0000","last_became_active":"2024-03-08T06:01:46.893035+0000","last_became_peered":"2024-03-08T06:01:46.893035+0000","last_unstale":"2024-03-08T06:01:47.903746+0000","last_undegraded":"2024-03-08T06:01:47.903746+0000","last_fullsized":"2024-03-08T06:01:47.903746+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:03:52.842458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.880181+0000","last_change":"2024-03-08T06:01:47.898718+0000","last_active":"2024-03-08T06:01:48.880181+0000","last_peered":"2024-03-08T06:01:48.880181+0000","last_clean":"2024-03-08T06:01:48.880181+0000","last_became_active":"2024-03-08T06:01:47.898440+0000","last_became_peered":"2024-03-08T06:01:47.898440+0000","last_unstale":"2024-03-08T06:01:48.880181+0000","last_undegraded":"2024-03-08T06:01:48.880181+0000","last_fullsized":"2024-03-08T06:01:48.880181+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:59:52.595386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.887807+0000","last_change":"2024-03-08T06:01:46.897480+0000","last_active":"2024-03-08T06:01:47.887807+0000","last_peered":"2024-03-08T06:01:47.887807+0000","last_clean":"2024-03-08T06:01:47.887807+0000","last_became_active":"2024-03-08T06:01:46.896101+0000","last_became_peered":"2024-03-08T06:01:46.896101+0000","last_unstale":"2024-03-08T06:01:47.887807+0000","last_undegraded":"2024-03-08T06:01:47.887807+0000","last_fullsized":"2024-03-08T06:01:47.887807+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:45:16.525970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.878837+0000","last_change":"2024-03-08T06:01:47.901392+0000","last_active":"2024-03-08T06:01:48.878837+0000","last_peered":"2024-03-08T06:01:48.878837+0000","last_clean":"2024-03-08T06:01:48.878837+0000","last_became_active":"2024-03-08T06:01:47.901103+0000","last_became_peered":"2024-03-08T06:01:47.901103+0000","last_unstale":"2024-03-08T06:01:48.878837+0000","last_undegraded":"2024-03-08T06:01:48.878837+0000","last_fullsized":"2024-03-08T06:01:48.878837+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:59:06.692046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879423+0000","last_change":"2024-03-08T06:01:46.889608+0000","last_active":"2024-03-08T06:01:48.879423+0000","last_peered":"2024-03-08T06:01:48.879423+0000","last_clean":"2024-03-08T06:01:48.879423+0000","last_became_active":"2024-03-08T06:01:46.889326+0000","last_became_peered":"2024-03-08T06:01:46.889326+0000","last_unstale":"2024-03-08T06:01:48.879423+0000","last_undegraded":"2024-03-08T06:01:48.879423+0000","last_fullsized":"2024-03-08T06:01:48.879423+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:45:16.525970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.903765+0000","last_change":"2024-03-08T06:01:47.897009+0000","last_active":"2024-03-08T06:01:47.903765+0000","last_peered":"2024-03-08T06:01:47.903765+0000","last_clean":"2024-03-08T06:01:47.903765+0000","last_became_active":"2024-03-08T06:01:47.895961+0000","last_became_peered":"2024-03-08T06:01:47.895961+0000","last_unstale":"2024-03-08T06:01:47.903765+0000","last_undegraded":"2024-03-08T06:01:47.903765+0000","last_fullsized":"2024-03-08T06:01:47.903765+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:57.350326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879233+0000","last_change":"2024-03-08T06:01:46.891263+0000","last_active":"2024-03-08T06:01:48.879233+0000","last_peered":"2024-03-08T06:01:48.879233+0000","last_clean":"2024-03-08T06:01:48.879233+0000","last_became_active":"2024-03-08T06:01:46.891014+0000","last_became_peered":"2024-03-08T06:01:46.891014+0000","last_unstale":"2024-03-08T06:01:48.879233+0000","last_undegraded":"2024-03-08T06:01:48.879233+0000","last_fullsized":"2024-03-08T06:01:48.879233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:36:09.052707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.909852+0000","last_change":"2024-03-08T06:01:47.900108+0000","last_active":"2024-03-08T06:01:47.909852+0000","last_peered":"2024-03-08T06:01:47.909852+0000","last_clean":"2024-03-08T06:01:47.909852+0000","last_became_active":"2024-03-08T06:01:47.899563+0000","last_became_peered":"2024-03-08T06:01:47.899563+0000","last_unstale":"2024-03-08T06:01:47.909852+0000","last_undegraded":"2024-03-08T06:01:47.909852+0000","last_fullsized":"2024-03-08T06:01:47.909852+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:34:33.647634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.883615+0000","last_change":"2024-03-08T06:01:46.894871+0000","last_active":"2024-03-08T06:01:47.883615+0000","last_peered":"2024-03-08T06:01:47.883615+0000","last_clean":"2024-03-08T06:01:47.883615+0000","last_became_active":"2024-03-08T06:01:46.894585+0000","last_became_peered":"2024-03-08T06:01:46.894585+0000","last_unstale":"2024-03-08T06:01:47.883615+0000","last_undegraded":"2024-03-08T06:01:47.883615+0000","last_fullsized":"2024-03-08T06:01:47.883615+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:14.311091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.896311+0000","last_change":"2024-03-08T06:01:47.891350+0000","last_active":"2024-03-08T06:01:47.896311+0000","last_peered":"2024-03-08T06:01:47.896311+0000","last_clean":"2024-03-08T06:01:47.896311+0000","last_became_active":"2024-03-08T06:01:47.891097+0000","last_became_peered":"2024-03-08T06:01:47.891097+0000","last_unstale":"2024-03-08T06:01:47.896311+0000","last_undegraded":"2024-03-08T06:01:47.896311+0000","last_fullsized":"2024-03-08T06:01:47.896311+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:15.316795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.912537+0000","last_change":"2024-03-08T06:01:46.896161+0000","last_active":"2024-03-08T06:01:47.912537+0000","last_peered":"2024-03-08T06:01:47.912537+0000","last_clean":"2024-03-08T06:01:47.912537+0000","last_became_active":"2024-03-08T06:01:46.895815+0000","last_became_peered":"2024-03-08T06:01:46.895815+0000","last_unstale":"2024-03-08T06:01:47.912537+0000","last_undegraded":"2024-03-08T06:01:47.912537+0000","last_fullsized":"2024-03-08T06:01:47.912537+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:18:59.639546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879879+0000","last_change":"2024-03-08T06:01:47.895259+0000","last_active":"2024-03-08T06:01:48.879879+0000","last_peered":"2024-03-08T06:01:48.879879+0000","last_clean":"2024-03-08T06:01:48.879879+0000","last_became_active":"2024-03-08T06:01:47.894971+0000","last_became_peered":"2024-03-08T06:01:47.894971+0000","last_unstale":"2024-03-08T06:01:48.879879+0000","last_undegraded":"2024-03-08T06:01:48.879879+0000","last_fullsized":"2024-03-08T06:01:48.879879+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:44:12.918446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.878969+0000","last_change":"2024-03-08T06:01:46.890902+0000","last_active":"2024-03-08T06:01:48.878969+0000","last_peered":"2024-03-08T06:01:48.878969+0000","last_clean":"2024-03-08T06:01:48.878969+0000","last_became_active":"2024-03-08T06:01:46.890638+0000","last_became_peered":"2024-03-08T06:01:46.890638+0000","last_unstale":"2024-03-08T06:01:48.878969+0000","last_undegraded":"2024-03-08T06:01:48.878969+0000","last_fullsized":"2024-03-08T06:01:48.878969+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:11:15.076128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.906262+0000","last_change":"2024-03-08T06:01:47.900191+0000","last_active":"2024-03-08T06:01:47.906262+0000","last_peered":"2024-03-08T06:01:47.906262+0000","last_clean":"2024-03-08T06:01:47.906262+0000","last_became_active":"2024-03-08T06:01:47.899537+0000","last_became_peered":"2024-03-08T06:01:47.899537+0000","last_unstale":"2024-03-08T06:01:47.906262+0000","last_undegraded":"2024-03-08T06:01:47.906262+0000","last_fullsized":"2024-03-08T06:01:47.906262+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:39:55.488990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879099+0000","last_change":"2024-03-08T06:01:46.889586+0000","last_active":"2024-03-08T06:01:48.879099+0000","last_peered":"2024-03-08T06:01:48.879099+0000","last_clean":"2024-03-08T06:01:48.879099+0000","last_became_active":"2024-03-08T06:01:46.889263+0000","last_became_peered":"2024-03-08T06:01:46.889263+0000","last_unstale":"2024-03-08T06:01:48.879099+0000","last_undegraded":"2024-03-08T06:01:48.879099+0000","last_fullsized":"2024-03-08T06:01:48.879099+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:59:51.589682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.878775+0000","last_change":"2024-03-08T06:01:47.897815+0000","last_active":"2024-03-08T06:01:48.878775+0000","last_peered":"2024-03-08T06:01:48.878775+0000","last_clean":"2024-03-08T06:01:48.878775+0000","last_became_active":"2024-03-08T06:01:47.897563+0000","last_became_peered":"2024-03-08T06:01:47.897563+0000","last_unstale":"2024-03-08T06:01:48.878775+0000","last_undegraded":"2024-03-08T06:01:48.878775+0000","last_fullsized":"2024-03-08T06:01:48.878775+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:33:01.188292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.887861+0000","last_change":"2024-03-08T06:01:46.896455+0000","last_active":"2024-03-08T06:01:47.887861+0000","last_peered":"2024-03-08T06:01:47.887861+0000","last_clean":"2024-03-08T06:01:47.887861+0000","last_became_active":"2024-03-08T06:01:46.896196+0000","last_became_peered":"2024-03-08T06:01:46.896196+0000","last_unstale":"2024-03-08T06:01:47.887861+0000","last_undegraded":"2024-03-08T06:01:47.887861+0000","last_fullsized":"2024-03-08T06:01:47.887861+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:34:32.641931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.907616+0000","last_change":"2024-03-08T06:01:47.897001+0000","last_active":"2024-03-08T06:01:47.907616+0000","last_peered":"2024-03-08T06:01:47.907616+0000","last_clean":"2024-03-08T06:01:47.907616+0000","last_became_active":"2024-03-08T06:01:47.896717+0000","last_became_peered":"2024-03-08T06:01:47.896717+0000","last_unstale":"2024-03-08T06:01:47.907616+0000","last_undegraded":"2024-03-08T06:01:47.907616+0000","last_fullsized":"2024-03-08T06:01:47.907616+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:18:21.554370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.884190+0000","last_change":"2024-03-08T06:01:46.889002+0000","last_active":"2024-03-08T06:01:47.884190+0000","last_peered":"2024-03-08T06:01:47.884190+0000","last_clean":"2024-03-08T06:01:47.884190+0000","last_became_active":"2024-03-08T06:01:46.888723+0000","last_became_peered":"2024-03-08T06:01:46.888723+0000","last_unstale":"2024-03-08T06:01:47.884190+0000","last_undegraded":"2024-03-08T06:01:47.884190+0000","last_fullsized":"2024-03-08T06:01:47.884190+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:47:59.325645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.878943+0000","last_change":"2024-03-08T06:01:47.898597+0000","last_active":"2024-03-08T06:01:48.878943+0000","last_peered":"2024-03-08T06:01:48.878943+0000","last_clean":"2024-03-08T06:01:48.878943+0000","last_became_active":"2024-03-08T06:01:47.898192+0000","last_became_peered":"2024-03-08T06:01:47.898192+0000","last_unstale":"2024-03-08T06:01:48.878943+0000","last_undegraded":"2024-03-08T06:01:48.878943+0000","last_fullsized":"2024-03-08T06:01:48.878943+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:20:20.387478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910780+0000","last_change":"2024-03-08T06:01:46.892113+0000","last_active":"2024-03-08T06:01:47.910780+0000","last_peered":"2024-03-08T06:01:47.910780+0000","last_clean":"2024-03-08T06:01:47.910780+0000","last_became_active":"2024-03-08T06:01:46.891852+0000","last_became_peered":"2024-03-08T06:01:46.891852+0000","last_unstale":"2024-03-08T06:01:47.910780+0000","last_undegraded":"2024-03-08T06:01:47.910780+0000","last_fullsized":"2024-03-08T06:01:47.910780+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:59:29.637335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.878980+0000","last_change":"2024-03-08T06:01:47.889397+0000","last_active":"2024-03-08T06:01:48.878980+0000","last_peered":"2024-03-08T06:01:48.878980+0000","last_clean":"2024-03-08T06:01:48.878980+0000","last_became_active":"2024-03-08T06:01:47.889123+0000","last_became_peered":"2024-03-08T06:01:47.889123+0000","last_unstale":"2024-03-08T06:01:48.878980+0000","last_undegraded":"2024-03-08T06:01:48.878980+0000","last_fullsized":"2024-03-08T06:01:48.878980+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:33:23.152529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.912355+0000","last_change":"2024-03-08T06:01:46.892791+0000","last_active":"2024-03-08T06:01:47.912355+0000","last_peered":"2024-03-08T06:01:47.912355+0000","last_clean":"2024-03-08T06:01:47.912355+0000","last_became_active":"2024-03-08T06:01:46.892488+0000","last_became_peered":"2024-03-08T06:01:46.892488+0000","last_unstale":"2024-03-08T06:01:47.912355+0000","last_undegraded":"2024-03-08T06:01:47.912355+0000","last_fullsized":"2024-03-08T06:01:47.912355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:01.668558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.902017+0000","last_change":"2024-03-08T06:01:47.895748+0000","last_active":"2024-03-08T06:01:47.902017+0000","last_peered":"2024-03-08T06:01:47.902017+0000","last_clean":"2024-03-08T06:01:47.902017+0000","last_became_active":"2024-03-08T06:01:47.895475+0000","last_became_peered":"2024-03-08T06:01:47.895475+0000","last_unstale":"2024-03-08T06:01:47.902017+0000","last_undegraded":"2024-03-08T06:01:47.902017+0000","last_fullsized":"2024-03-08T06:01:47.902017+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:44:12.918446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.880004+0000","last_change":"2024-03-08T06:01:46.891434+0000","last_active":"2024-03-08T06:01:48.880004+0000","last_peered":"2024-03-08T06:01:48.880004+0000","last_clean":"2024-03-08T06:01:48.880004+0000","last_became_active":"2024-03-08T06:01:46.891170+0000","last_became_peered":"2024-03-08T06:01:46.891170+0000","last_unstale":"2024-03-08T06:01:48.880004+0000","last_undegraded":"2024-03-08T06:01:48.880004+0000","last_fullsized":"2024-03-08T06:01:48.880004+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:18:59.639546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879871+0000","last_change":"2024-03-08T06:01:47.890670+0000","last_active":"2024-03-08T06:01:48.879871+0000","last_peered":"2024-03-08T06:01:48.879871+0000","last_clean":"2024-03-08T06:01:48.879871+0000","last_became_active":"2024-03-08T06:01:47.890386+0000","last_became_peered":"2024-03-08T06:01:47.890386+0000","last_unstale":"2024-03-08T06:01:48.879871+0000","last_undegraded":"2024-03-08T06:01:48.879871+0000","last_fullsized":"2024-03-08T06:01:48.879871+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:26:23.550550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910486+0000","last_change":"2024-03-08T06:01:46.890485+0000","last_active":"2024-03-08T06:01:47.910486+0000","last_peered":"2024-03-08T06:01:47.910486+0000","last_clean":"2024-03-08T06:01:47.910486+0000","last_became_active":"2024-03-08T06:01:46.890245+0000","last_became_peered":"2024-03-08T06:01:46.890245+0000","last_unstale":"2024-03-08T06:01:47.910486+0000","last_undegraded":"2024-03-08T06:01:47.910486+0000","last_fullsized":"2024-03-08T06:01:47.910486+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:00:05.649277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.895778+0000","last_change":"2024-03-08T06:01:47.892473+0000","last_active":"2024-03-08T06:01:47.895778+0000","last_peered":"2024-03-08T06:01:47.895778+0000","last_clean":"2024-03-08T06:01:47.895778+0000","last_became_active":"2024-03-08T06:01:47.892209+0000","last_became_peered":"2024-03-08T06:01:47.892209+0000","last_unstale":"2024-03-08T06:01:47.895778+0000","last_undegraded":"2024-03-08T06:01:47.895778+0000","last_fullsized":"2024-03-08T06:01:47.895778+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:07:44.511982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.887954+0000","last_change":"2024-03-08T06:01:46.893168+0000","last_active":"2024-03-08T06:01:47.887954+0000","last_peered":"2024-03-08T06:01:47.887954+0000","last_clean":"2024-03-08T06:01:47.887954+0000","last_became_active":"2024-03-08T06:01:46.892901+0000","last_became_peered":"2024-03-08T06:01:46.892901+0000","last_unstale":"2024-03-08T06:01:47.887954+0000","last_undegraded":"2024-03-08T06:01:47.887954+0000","last_fullsized":"2024-03-08T06:01:47.887954+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:02:07.827318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910623+0000","last_change":"2024-03-08T06:01:47.905298+0000","last_active":"2024-03-08T06:01:47.910623+0000","last_peered":"2024-03-08T06:01:47.910623+0000","last_clean":"2024-03-08T06:01:47.910623+0000","last_became_active":"2024-03-08T06:01:47.904945+0000","last_became_peered":"2024-03-08T06:01:47.904945+0000","last_unstale":"2024-03-08T06:01:47.910623+0000","last_undegraded":"2024-03-08T06:01:47.910623+0000","last_fullsized":"2024-03-08T06:01:47.910623+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:02.674261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.903705+0000","last_change":"2024-03-08T06:01:46.891517+0000","last_active":"2024-03-08T06:01:47.903705+0000","last_peered":"2024-03-08T06:01:47.903705+0000","last_clean":"2024-03-08T06:01:47.903705+0000","last_became_active":"2024-03-08T06:01:46.891117+0000","last_became_peered":"2024-03-08T06:01:46.891117+0000","last_unstale":"2024-03-08T06:01:47.903705+0000","last_undegraded":"2024-03-08T06:01:47.903705+0000","last_fullsized":"2024-03-08T06:01:47.903705+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:27:58.111937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.909104+0000","last_change":"2024-03-08T06:01:47.905551+0000","last_active":"2024-03-08T06:01:47.909104+0000","last_peered":"2024-03-08T06:01:47.909104+0000","last_clean":"2024-03-08T06:01:47.909104+0000","last_became_active":"2024-03-08T06:01:47.905325+0000","last_became_peered":"2024-03-08T06:01:47.905325+0000","last_unstale":"2024-03-08T06:01:47.909104+0000","last_undegraded":"2024-03-08T06:01:47.909104+0000","last_fullsized":"2024-03-08T06:01:47.909104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:15.275207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.887288+0000","last_change":"2024-03-08T06:01:46.892096+0000","last_active":"2024-03-08T06:01:47.887288+0000","last_peered":"2024-03-08T06:01:47.887288+0000","last_clean":"2024-03-08T06:01:47.887288+0000","last_became_active":"2024-03-08T06:01:46.891853+0000","last_became_peered":"2024-03-08T06:01:46.891853+0000","last_unstale":"2024-03-08T06:01:47.887288+0000","last_undegraded":"2024-03-08T06:01:47.887288+0000","last_fullsized":"2024-03-08T06:01:47.887288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:15.634163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.894567+0000","last_change":"2024-03-08T06:01:47.886742+0000","last_active":"2024-03-08T06:01:47.894567+0000","last_peered":"2024-03-08T06:01:47.894567+0000","last_clean":"2024-03-08T06:01:47.894567+0000","last_became_active":"2024-03-08T06:01:47.886477+0000","last_became_peered":"2024-03-08T06:01:47.886477+0000","last_unstale":"2024-03-08T06:01:47.894567+0000","last_undegraded":"2024-03-08T06:01:47.894567+0000","last_fullsized":"2024-03-08T06:01:47.894567+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:39:55.488990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.889761+0000","last_change":"2024-03-08T06:01:46.893559+0000","last_active":"2024-03-08T06:01:47.889761+0000","last_peered":"2024-03-08T06:01:47.889761+0000","last_clean":"2024-03-08T06:01:47.889761+0000","last_became_active":"2024-03-08T06:01:46.893295+0000","last_became_peered":"2024-03-08T06:01:46.893295+0000","last_unstale":"2024-03-08T06:01:47.889761+0000","last_undegraded":"2024-03-08T06:01:47.889761+0000","last_fullsized":"2024-03-08T06:01:47.889761+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:25:20.026788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.878982+0000","last_change":"2024-03-08T06:01:47.891898+0000","last_active":"2024-03-08T06:01:48.878982+0000","last_peered":"2024-03-08T06:01:48.878982+0000","last_clean":"2024-03-08T06:01:48.878982+0000","last_became_active":"2024-03-08T06:01:47.891620+0000","last_became_peered":"2024-03-08T06:01:47.891620+0000","last_unstale":"2024-03-08T06:01:48.878982+0000","last_undegraded":"2024-03-08T06:01:48.878982+0000","last_fullsized":"2024-03-08T06:01:48.878982+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:16:20.468512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.885294+0000","last_change":"2024-03-08T06:01:46.887924+0000","last_active":"2024-03-08T06:01:47.885294+0000","last_peered":"2024-03-08T06:01:47.885294+0000","last_clean":"2024-03-08T06:01:47.885294+0000","last_became_active":"2024-03-08T06:01:46.887652+0000","last_became_peered":"2024-03-08T06:01:46.887652+0000","last_unstale":"2024-03-08T06:01:47.885294+0000","last_undegraded":"2024-03-08T06:01:47.885294+0000","last_fullsized":"2024-03-08T06:01:47.885294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:52:50.743455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.906959+0000","last_change":"2024-03-08T06:01:47.899190+0000","last_active":"2024-03-08T06:01:47.906959+0000","last_peered":"2024-03-08T06:01:47.906959+0000","last_clean":"2024-03-08T06:01:47.906959+0000","last_became_active":"2024-03-08T06:01:47.898926+0000","last_became_peered":"2024-03-08T06:01:47.898926+0000","last_unstale":"2024-03-08T06:01:47.906959+0000","last_undegraded":"2024-03-08T06:01:47.906959+0000","last_fullsized":"2024-03-08T06:01:47.906959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:39:55.488990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.912421+0000","last_change":"2024-03-08T06:01:46.896074+0000","last_active":"2024-03-08T06:01:47.912421+0000","last_peered":"2024-03-08T06:01:47.912421+0000","last_clean":"2024-03-08T06:01:47.912421+0000","last_became_active":"2024-03-08T06:01:46.895749+0000","last_became_peered":"2024-03-08T06:01:46.895749+0000","last_unstale":"2024-03-08T06:01:47.912421+0000","last_undegraded":"2024-03-08T06:01:47.912421+0000","last_fullsized":"2024-03-08T06:01:47.912421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:06:46.976776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.909831+0000","last_change":"2024-03-08T06:01:47.899886+0000","last_active":"2024-03-08T06:01:47.909831+0000","last_peered":"2024-03-08T06:01:47.909831+0000","last_clean":"2024-03-08T06:01:47.909831+0000","last_became_active":"2024-03-08T06:01:47.899574+0000","last_became_peered":"2024-03-08T06:01:47.899574+0000","last_unstale":"2024-03-08T06:01:47.909831+0000","last_undegraded":"2024-03-08T06:01:47.909831+0000","last_fullsized":"2024-03-08T06:01:47.909831+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:45:17.531673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.888454+0000","last_change":"2024-03-08T06:01:46.892634+0000","last_active":"2024-03-08T06:01:47.888454+0000","last_peered":"2024-03-08T06:01:47.888454+0000","last_clean":"2024-03-08T06:01:47.888454+0000","last_became_active":"2024-03-08T06:01:46.892390+0000","last_became_peered":"2024-03-08T06:01:46.892390+0000","last_unstale":"2024-03-08T06:01:47.888454+0000","last_undegraded":"2024-03-08T06:01:47.888454+0000","last_fullsized":"2024-03-08T06:01:47.888454+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:45:35.630850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879180+0000","last_change":"2024-03-08T06:01:47.886595+0000","last_active":"2024-03-08T06:01:48.879180+0000","last_peered":"2024-03-08T06:01:48.879180+0000","last_clean":"2024-03-08T06:01:48.879180+0000","last_became_active":"2024-03-08T06:01:47.886348+0000","last_became_peered":"2024-03-08T06:01:47.886348+0000","last_unstale":"2024-03-08T06:01:48.879180+0000","last_undegraded":"2024-03-08T06:01:48.879180+0000","last_fullsized":"2024-03-08T06:01:48.879180+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:21:21.582227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.887606+0000","last_change":"2024-03-08T06:01:46.887951+0000","last_active":"2024-03-08T06:01:47.887606+0000","last_peered":"2024-03-08T06:01:47.887606+0000","last_clean":"2024-03-08T06:01:47.887606+0000","last_became_active":"2024-03-08T06:01:46.887709+0000","last_became_peered":"2024-03-08T06:01:46.887709+0000","last_unstale":"2024-03-08T06:01:47.887606+0000","last_undegraded":"2024-03-08T06:01:47.887606+0000","last_fullsized":"2024-03-08T06:01:47.887606+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:18:59.639546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.897226+0000","last_change":"2024-03-08T06:01:47.894597+0000","last_active":"2024-03-08T06:01:47.897226+0000","last_peered":"2024-03-08T06:01:47.897226+0000","last_clean":"2024-03-08T06:01:47.897226+0000","last_became_active":"2024-03-08T06:01:47.894333+0000","last_became_peered":"2024-03-08T06:01:47.894333+0000","last_unstale":"2024-03-08T06:01:47.897226+0000","last_undegraded":"2024-03-08T06:01:47.897226+0000","last_fullsized":"2024-03-08T06:01:47.897226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:06:47.982479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.887037+0000","last_change":"2024-03-08T06:01:46.886717+0000","last_active":"2024-03-08T06:01:47.887037+0000","last_peered":"2024-03-08T06:01:47.887037+0000","last_clean":"2024-03-08T06:01:47.887037+0000","last_became_active":"2024-03-08T06:01:46.886354+0000","last_became_peered":"2024-03-08T06:01:46.886354+0000","last_unstale":"2024-03-08T06:01:47.887037+0000","last_undegraded":"2024-03-08T06:01:47.887037+0000","last_fullsized":"2024-03-08T06:01:47.887037+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:02:07.827318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.896050+0000","last_change":"2024-03-08T06:01:47.894097+0000","last_active":"2024-03-08T06:01:47.896050+0000","last_peered":"2024-03-08T06:01:47.896050+0000","last_clean":"2024-03-08T06:01:47.896050+0000","last_became_active":"2024-03-08T06:01:47.893838+0000","last_became_peered":"2024-03-08T06:01:47.893838+0000","last_unstale":"2024-03-08T06:01:47.896050+0000","last_undegraded":"2024-03-08T06:01:47.896050+0000","last_fullsized":"2024-03-08T06:01:47.896050+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:22:02.674261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.885054+0000","last_change":"2024-03-08T06:01:46.887914+0000","last_active":"2024-03-08T06:01:47.885054+0000","last_peered":"2024-03-08T06:01:47.885054+0000","last_clean":"2024-03-08T06:01:47.885054+0000","last_became_active":"2024-03-08T06:01:46.887654+0000","last_became_peered":"2024-03-08T06:01:46.887654+0000","last_unstale":"2024-03-08T06:01:47.885054+0000","last_undegraded":"2024-03-08T06:01:47.885054+0000","last_fullsized":"2024-03-08T06:01:47.885054+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:08:00.989881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879703+0000","last_change":"2024-03-08T06:01:47.887417+0000","last_active":"2024-03-08T06:01:48.879703+0000","last_peered":"2024-03-08T06:01:48.879703+0000","last_clean":"2024-03-08T06:01:48.879703+0000","last_became_active":"2024-03-08T06:01:47.887133+0000","last_became_peered":"2024-03-08T06:01:47.887133+0000","last_unstale":"2024-03-08T06:01:48.879703+0000","last_undegraded":"2024-03-08T06:01:48.879703+0000","last_fullsized":"2024-03-08T06:01:48.879703+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:15.275207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.879477+0000","last_change":"2024-03-08T06:01:46.895158+0000","last_active":"2024-03-08T06:01:48.879477+0000","last_peered":"2024-03-08T06:01:48.879477+0000","last_clean":"2024-03-08T06:01:48.879477+0000","last_became_active":"2024-03-08T06:01:46.894873+0000","last_became_peered":"2024-03-08T06:01:46.894873+0000","last_unstale":"2024-03-08T06:01:48.879477+0000","last_undegraded":"2024-03-08T06:01:48.879477+0000","last_fullsized":"2024-03-08T06:01:48.879477+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:34:32.641931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.910269+0000","last_change":"2024-03-08T06:01:47.906862+0000","last_active":"2024-03-08T06:01:47.910269+0000","last_peered":"2024-03-08T06:01:47.910269+0000","last_clean":"2024-03-08T06:01:47.910269+0000","last_became_active":"2024-03-08T06:01:47.905456+0000","last_became_peered":"2024-03-08T06:01:47.905456+0000","last_unstale":"2024-03-08T06:01:47.910269+0000","last_undegraded":"2024-03-08T06:01:47.910269+0000","last_fullsized":"2024-03-08T06:01:47.910269+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:28.902386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-08T06:01:48.878725+0000","last_change":"2024-03-08T06:01:46.892083+0000","last_active":"2024-03-08T06:01:48.878725+0000","last_peered":"2024-03-08T06:01:48.878725+0000","last_clean":"2024-03-08T06:01:48.878725+0000","last_became_active":"2024-03-08T06:01:46.891339+0000","last_became_peered":"2024-03-08T06:01:46.891339+0000","last_unstale":"2024-03-08T06:01:48.878725+0000","last_undegraded":"2024-03-08T06:01:48.878725+0000","last_fullsized":"2024-03-08T06:01:48.878725+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:24:58.571144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.904605+0000","last_change":"2024-03-08T06:01:47.901247+0000","last_active":"2024-03-08T06:01:47.904605+0000","last_peered":"2024-03-08T06:01:47.904605+0000","last_clean":"2024-03-08T06:01:47.904605+0000","last_became_active":"2024-03-08T06:01:47.900913+0000","last_became_peered":"2024-03-08T06:01:47.900913+0000","last_unstale":"2024-03-08T06:01:47.904605+0000","last_undegraded":"2024-03-08T06:01:47.904605+0000","last_fullsized":"2024-03-08T06:01:47.904605+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:46.868764+0000","last_clean_scrub_stamp":"2024-03-08T06:01:46.868764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:36:10.058410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.888937+0000","last_change":"2024-03-08T06:01:46.897282+0000","last_active":"2024-03-08T06:01:47.888937+0000","last_peered":"2024-03-08T06:01:47.888937+0000","last_clean":"2024-03-08T06:01:47.888937+0000","last_became_active":"2024-03-08T06:01:46.897100+0000","last_became_peered":"2024-03-08T06:01:46.897100+0000","last_unstale":"2024-03-08T06:01:47.888937+0000","last_undegraded":"2024-03-08T06:01:47.888937+0000","last_fullsized":"2024-03-08T06:01:47.888937+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:01:45.863060+0000","last_clean_scrub_stamp":"2024-03-08T06:01:45.863060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:09:41.301850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":203,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-08T06:01:47.888609+0000","last_change":"2024-03-08T06:00:26.178419+0000","last_active":"2024-03-08T06:01:47.888609+0000","last_peered":"2024-03-08T06:01:47.888609+0000","last_clean":"2024-03-08T06:01:47.888609+0000","last_became_active":"2024-03-08T06:00:26.175826+0000","last_became_peered":"2024-03-08T06:00:26.175826+0000","last_unstale":"2024-03-08T06:01:47.888609+0000","last_undegraded":"2024-03-08T06:01:47.888609+0000","last_fullsized":"2024-03-08T06:01:47.888609+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_clean_scrub_stamp":"2024-03-08T06:00:24.029492+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:36:00.682517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1060864,"data_stored":787572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":196104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":195634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":196126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":197168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":195592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":195612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":785924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:01:51.193 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-08T06:01:51.193 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-08T06:01:51.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.336+0000 7fb54cac3640 1 Processor -- start 2024-03-08T06:01:51.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.336+0000 7fb54cac3640 1 -- start start 2024-03-08T06:01:51.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb54cac3640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb5480cee10 0x7fb5480714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb54cac3640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb548071a00 0x7fb54815e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb54cac3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5480af050 con 0x7fb54815e9d0 2024-03-08T06:01:51.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb54cac3640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb548071e60 con 0x7fb548071a00 2024-03-08T06:01:51.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb54cac3640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54815f210 con 0x7fb5480cee10 2024-03-08T06:01:51.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb546d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb54815e9d0 0x7fb5481618a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39264/0 (socket says 172.21.15.76:39264) 2024-03-08T06:01:51.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb546d9d640 1 -- 172.21.15.76:0/1096334690 learned_addr learned my addr 172.21.15.76:0/1096334690 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:51.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb545d9b640 1 -- 172.21.15.76:0/1096334690 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3046635625 0 0) 0x7fb5480af050 con 0x7fb54815e9d0 2024-03-08T06:01:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb545d9b640 1 -- 172.21.15.76:0/1096334690 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb534003520 con 0x7fb54815e9d0 2024-03-08T06:01:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb54659c640 1 --2- 172.21.15.76:0/1096334690 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb5480cee10 0x7fb5480714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb53ffff640 1 --2- 172.21.15.76:0/1096334690 >> v2:172.21.15.81:3300/0 conn(0x7fb548071a00 0x7fb54815e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb545d9b640 1 -- 172.21.15.76:0/1096334690 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 811217380 0 0) 0x7fb534003520 con 0x7fb54815e9d0 2024-03-08T06:01:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb545d9b640 1 -- 172.21.15.76:0/1096334690 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb5480cee10 msgr2=0x7fb5480714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb545d9b640 1 --2- 172.21.15.76:0/1096334690 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb5480cee10 0x7fb5480714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb545d9b640 1 -- 172.21.15.76:0/1096334690 >> v2:172.21.15.81:3300/0 conn(0x7fb548071a00 msgr2=0x7fb54815e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb545d9b640 1 --2- 172.21.15.76:0/1096334690 >> v2:172.21.15.81:3300/0 conn(0x7fb548071a00 0x7fb54815e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb545d9b640 1 -- 172.21.15.76:0/1096334690 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54806d5c0 con 0x7fb54815e9d0 2024-03-08T06:01:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb545d9b640 1 -- 172.21.15.76:0/1096334690 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fb5400019f0 con 0x7fb54815e9d0 2024-03-08T06:01:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb545d9b640 1 -- 172.21.15.76:0/1096334690 >> v1:172.21.15.76:6789/0 conn(0x7fb54815e9d0 legacy=0x7fb5481618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb545d9b640 1 --2- 172.21.15.76:0/1096334690 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb5480cee10 0x7fb5340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb545d9b640 1 --2- 172.21.15.76:0/1096334690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb534003f30 0x7fb534004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb545d9b640 1 --2- 172.21.15.76:0/1096334690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb534004d40 0x7fb5340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb545d9b640 1 -- 172.21.15.76:0/1096334690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54815f210 con 0x7fb534003f30 2024-03-08T06:01:51.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb545d9b640 1 -- 172.21.15.76:0/1096334690 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53400b9c0 con 0x7fb534004d40 2024-03-08T06:01:51.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.337+0000 7fb545d9b640 1 -- 172.21.15.76:0/1096334690 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53400bb90 con 0x7fb5480cee10 2024-03-08T06:01:51.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.338+0000 7fb53ffff640 1 --2- 172.21.15.76:0/1096334690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb534003f30 0x7fb534004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.338+0000 7fb54659c640 1 --2- 172.21.15.76:0/1096334690 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb5480cee10 0x7fb5340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.338+0000 7fb546d9d640 1 --2- 172.21.15.76:0/1096334690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb534004d40 0x7fb5340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.338+0000 7fb53ffff640 1 -- 172.21.15.76:0/1096334690 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb5480cee10 msgr2=0x7fb5340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:51.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.338+0000 7fb53ffff640 1 --2- 172.21.15.76:0/1096334690 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb5480cee10 0x7fb5340039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.338+0000 7fb53ffff640 1 -- 172.21.15.76:0/1096334690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb534004d40 msgr2=0x7fb5340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:51.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.338+0000 7fb53ffff640 1 --2- 172.21.15.76:0/1096334690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb534004d40 0x7fb5340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.338+0000 7fb53ffff640 1 -- 172.21.15.76:0/1096334690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb534003520 con 0x7fb534003f30 2024-03-08T06:01:51.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.339+0000 7fb53ffff640 1 --2- 172.21.15.76:0/1096334690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb534003f30 0x7fb534004800 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb53000ba30 tx=0x7fb534005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbea5fc640c6176b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:51.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.339+0000 7fb545d9b640 1 -- 172.21.15.76:0/1096334690 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530036020 con 0x7fb534003f30 2024-03-08T06:01:51.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.339+0000 7fb545d9b640 1 -- 172.21.15.76:0/1096334690 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb530032680 con 0x7fb534003f30 2024-03-08T06:01:51.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.339+0000 7fb545d9b640 1 -- 172.21.15.76:0/1096334690 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53000b600 con 0x7fb534003f30 2024-03-08T06:01:51.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.339+0000 7fb54cac3640 1 -- 172.21.15.76:0/1096334690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb534003f30 msgr2=0x7fb534004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:51.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.339+0000 7fb54cac3640 1 --2- 172.21.15.76:0/1096334690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb534003f30 0x7fb534004800 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb53000ba30 tx=0x7fb534005830 comp rx=0 tx=0).stop 2024-03-08T06:01:51.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.339+0000 7fb546d9d640 1 -- 172.21.15.76:0/1096334690 reap_dead start 2024-03-08T06:01:51.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.339+0000 7fb54cac3640 1 -- 172.21.15.76:0/1096334690 shutdown_connections 2024-03-08T06:01:51.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.339+0000 7fb54cac3640 1 -- 172.21.15.76:0/1096334690 >> 172.21.15.76:0/1096334690 conn(0x7fb5480b1840 msgr2=0x7fb5480b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:51.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.339+0000 7fb54cac3640 1 -- 172.21.15.76:0/1096334690 shutdown_connections 2024-03-08T06:01:51.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.339+0000 7fb54cac3640 1 -- 172.21.15.76:0/1096334690 wait complete. 2024-03-08T06:01:51.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.341+0000 7fb54cac3640 1 Processor -- start 2024-03-08T06:01:51.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.341+0000 7fb54cac3640 1 -- start start 2024-03-08T06:01:51.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.341+0000 7fb54cac3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb534003f30 0x7fb54813de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.341+0000 7fb54cac3640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb54813f860 0x7fb54815b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.341+0000 7fb54cac3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb54813ffb0 0x7fb5480b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.341+0000 7fb54cac3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54806de60 con 0x7fb54813ffb0 2024-03-08T06:01:51.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.341+0000 7fb54cac3640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54806db80 con 0x7fb534003f30 2024-03-08T06:01:51.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.341+0000 7fb54cac3640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480cf1f0 con 0x7fb54813f860 2024-03-08T06:01:51.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.341+0000 7fb546d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb54813ffb0 0x7fb5480b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.341+0000 7fb546d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb54813ffb0 0x7fb5480b38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51308/0 (socket says 172.21.15.76:51308) 2024-03-08T06:01:51.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.341+0000 7fb546d9d640 1 -- 172.21.15.76:0/4113368672 learned_addr learned my addr 172.21.15.76:0/4113368672 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:51.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.342+0000 7fb54659c640 1 --2- 172.21.15.76:0/4113368672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb534003f30 0x7fb54813de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.342+0000 7fb53ffff640 1 --2- 172.21.15.76:0/4113368672 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb54813f860 0x7fb54815b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.342+0000 7fb546d9d640 1 -- 172.21.15.76:0/4113368672 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb54813f860 msgr2=0x7fb54815b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.342+0000 7fb546d9d640 1 --2- 172.21.15.76:0/4113368672 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb54813f860 0x7fb54815b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.342+0000 7fb546d9d640 1 -- 172.21.15.76:0/4113368672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb534003f30 msgr2=0x7fb54813de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.342+0000 7fb546d9d640 1 --2- 172.21.15.76:0/4113368672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb534003f30 0x7fb54813de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.342+0000 7fb546d9d640 1 -- 172.21.15.76:0/4113368672 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54813fd10 con 0x7fb54813ffb0 2024-03-08T06:01:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.342+0000 7fb546d9d640 1 --2- 172.21.15.76:0/4113368672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb54813ffb0 0x7fb5480b38d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb540000f80 tx=0x7fb540007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.342+0000 7fb53effd640 1 -- 172.21.15.76:0/4113368672 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540019070 con 0x7fb54813ffb0 2024-03-08T06:01:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.342+0000 7fb53effd640 1 -- 172.21.15.76:0/4113368672 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54000ab20 con 0x7fb54813ffb0 2024-03-08T06:01:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.342+0000 7fb54cac3640 1 -- 172.21.15.76:0/4113368672 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb548140460 con 0x7fb54813ffb0 2024-03-08T06:01:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.342+0000 7fb53effd640 1 -- 172.21.15.76:0/4113368672 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5400133f0 con 0x7fb54813ffb0 2024-03-08T06:01:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.342+0000 7fb54cac3640 1 -- 172.21.15.76:0/4113368672 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb548144030 con 0x7fb54813ffb0 2024-03-08T06:01:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.343+0000 7fb54cac3640 1 -- 172.21.15.76:0/4113368672 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb54806a1d0 con 0x7fb54813ffb0 2024-03-08T06:01:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.343+0000 7fb53effd640 1 -- 172.21.15.76:0/4113368672 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb54001c070 con 0x7fb54813ffb0 2024-03-08T06:01:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.344+0000 7fb53effd640 1 --2- 172.21.15.76:0/4113368672 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb52806b4d0 0x7fb52806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.344+0000 7fb53effd640 1 -- 172.21.15.76:0/4113368672 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb540089ec0 con 0x7fb54813ffb0 2024-03-08T06:01:51.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.344+0000 7fb54659c640 1 --2- 172.21.15.76:0/4113368672 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb52806b4d0 0x7fb52806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.344+0000 7fb54659c640 1 --2- 172.21.15.76:0/4113368672 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb52806b4d0 0x7fb52806d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb54813b0a0 tx=0x7fb538008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:51.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.348+0000 7fb53effd640 1 -- 172.21.15.76:0/4113368672 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb540058bb0 con 0x7fb54813ffb0 2024-03-08T06:01:51.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.581+0000 7fb54cac3640 1 -- 172.21.15.76:0/4113368672 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb54806ed30 con 0x7fb54813ffb0 2024-03-08T06:01:51.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.584+0000 7fb53effd640 1 -- 172.21.15.76:0/4113368672 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fb5400584b0 con 0x7fb54813ffb0 2024-03-08T06:01:51.584 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2024-03-08T06:01:51.584 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQBKqepl3elrGRAAXLq7DBlUer23iSXOoBZatA== 2024-03-08T06:01:51.584 INFO:teuthology.orchestra.run.smithi076.stdout: caps mds = "allow" 2024-03-08T06:01:51.584 INFO:teuthology.orchestra.run.smithi076.stdout: caps mon = "allow r" 2024-03-08T06:01:51.585 INFO:teuthology.orchestra.run.smithi076.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-08T06:01:51.585 INFO:teuthology.orchestra.run.smithi076.stderr:updated caps for client.0 2024-03-08T06:01:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.586+0000 7fb54cac3640 1 -- 172.21.15.76:0/4113368672 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb52806b4d0 msgr2=0x7fb52806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.586+0000 7fb54cac3640 1 --2- 172.21.15.76:0/4113368672 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb52806b4d0 0x7fb52806d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb54813b0a0 tx=0x7fb538008040 comp rx=0 tx=0).stop 2024-03-08T06:01:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.586+0000 7fb54cac3640 1 -- 172.21.15.76:0/4113368672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb54813ffb0 msgr2=0x7fb5480b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.586+0000 7fb54cac3640 1 --2- 172.21.15.76:0/4113368672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb54813ffb0 0x7fb5480b38d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb540000f80 tx=0x7fb540007b00 comp rx=0 tx=0).stop 2024-03-08T06:01:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.586+0000 7fb54cac3640 1 -- 172.21.15.76:0/4113368672 shutdown_connections 2024-03-08T06:01:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.586+0000 7fb54cac3640 1 --2- 172.21.15.76:0/4113368672 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb52806b4d0 0x7fb52806d990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.586+0000 7fb54cac3640 1 --2- 172.21.15.76:0/4113368672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb54813ffb0 0x7fb5480b38d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.586+0000 7fb54cac3640 1 --2- 172.21.15.76:0/4113368672 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb54813f860 0x7fb54815b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.586+0000 7fb54cac3640 1 --2- 172.21.15.76:0/4113368672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb534003f30 0x7fb54813de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.586+0000 7fb54cac3640 1 -- 172.21.15.76:0/4113368672 >> 172.21.15.76:0/4113368672 conn(0x7fb5480b1840 msgr2=0x7fb54806b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.586+0000 7fb54cac3640 1 -- 172.21.15.76:0/4113368672 shutdown_connections 2024-03-08T06:01:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.586+0000 7fb54cac3640 1 -- 172.21.15.76:0/4113368672 wait complete. 2024-03-08T06:01:51.597 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:01:51.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.696+0000 7f9ab41c7640 1 Processor -- start 2024-03-08T06:01:51.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.697+0000 7f9ab41c7640 1 -- start start 2024-03-08T06:01:51.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.697+0000 7f9ab41c7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9aac0cd310 0x7f9aac0cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.697+0000 7f9ab41c7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aac158050 con 0x7f9aac157c70 2024-03-08T06:01:51.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.697+0000 7f9ab41c7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aac15b160 con 0x7f9aac06eb70 2024-03-08T06:01:51.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.697+0000 7f9ab41c7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac15b440 con 0x7f9aac0cd310 2024-03-08T06:01:51.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.697+0000 7f9ab1f63640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9aac157c70 0x7f9aac15a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39276/0 (socket says 172.21.15.76:39276) 2024-03-08T06:01:51.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab1f63640 1 -- 172.21.15.76:0/1116186289 learned_addr learned my addr 172.21.15.76:0/1116186289 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:01:51.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab1762640 1 --2- 172.21.15.76:0/1116186289 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9aac0cd310 0x7f9aac0cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab0f61640 1 -- 172.21.15.76:0/1116186289 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251740009 0 0) 0x7f9aac158050 con 0x7f9aac157c70 2024-03-08T06:01:51.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab0f61640 1 -- 172.21.15.76:0/1116186289 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a98003520 con 0x7f9aac157c70 2024-03-08T06:01:51.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab0f61640 1 -- 172.21.15.76:0/1116186289 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1589804002 0 0) 0x7f9a98003520 con 0x7f9aac157c70 2024-03-08T06:01:51.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab0f61640 1 -- 172.21.15.76:0/1116186289 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9aac0cd310 msgr2=0x7f9aac0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:51.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab0f61640 1 --2- 172.21.15.76:0/1116186289 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9aac0cd310 0x7f9aac0cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab0f61640 1 -- 172.21.15.76:0/1116186289 >> v1:172.21.15.81:6789/0 conn(0x7f9aac06eb70 legacy=0x7f9aac071c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:51.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab0f61640 1 -- 172.21.15.76:0/1116186289 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac06e370 con 0x7f9aac157c70 2024-03-08T06:01:51.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab0f61640 1 -- 172.21.15.76:0/1116186289 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f9aa8001670 con 0x7f9aac157c70 2024-03-08T06:01:51.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab0f61640 1 -- 172.21.15.76:0/1116186289 >> v1:172.21.15.76:6789/0 conn(0x7f9aac157c70 legacy=0x7f9aac15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:51.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab0f61640 1 --2- 172.21.15.76:0/1116186289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a98003a50 0x7f9a98003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab0f61640 1 --2- 172.21.15.76:0/1116186289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a98004800 0x7f9a98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab0f61640 1 --2- 172.21.15.76:0/1116186289 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9aac0cd310 0x7f9a980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab0f61640 1 -- 172.21.15.76:0/1116186289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac15b440 con 0x7f9a98003a50 2024-03-08T06:01:51.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab0f61640 1 -- 172.21.15.76:0/1116186289 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9800bac0 con 0x7f9a98004800 2024-03-08T06:01:51.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab0f61640 1 -- 172.21.15.76:0/1116186289 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9800bcc0 con 0x7f9aac0cd310 2024-03-08T06:01:51.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab1f63640 1 --2- 172.21.15.76:0/1116186289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a98003a50 0x7f9a98003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab2764640 1 --2- 172.21.15.76:0/1116186289 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9aac0cd310 0x7f9a980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab1762640 1 --2- 172.21.15.76:0/1116186289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a98004800 0x7f9a98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab1f63640 1 -- 172.21.15.76:0/1116186289 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9aac0cd310 msgr2=0x7f9a980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:51.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab1f63640 1 --2- 172.21.15.76:0/1116186289 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9aac0cd310 0x7f9a980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab1f63640 1 -- 172.21.15.76:0/1116186289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a98004800 msgr2=0x7f9a98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:51.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.698+0000 7f9ab1f63640 1 --2- 172.21.15.76:0/1116186289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a98004800 0x7f9a98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.699+0000 7f9ab1f63640 1 -- 172.21.15.76:0/1116186289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a98003520 con 0x7f9a98003a50 2024-03-08T06:01:51.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.699+0000 7f9ab1f63640 1 --2- 172.21.15.76:0/1116186289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a98003a50 0x7f9a98003e70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800c240 tx=0x7f9a98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34792501a8f5300e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:51.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.699+0000 7f9ab0f61640 1 -- 172.21.15.76:0/1116186289 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8007d10 con 0x7f9a98003a50 2024-03-08T06:01:51.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.699+0000 7f9ab0f61640 1 -- 172.21.15.76:0/1116186289 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa8030070 con 0x7f9a98003a50 2024-03-08T06:01:51.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.699+0000 7f9ab0f61640 1 -- 172.21.15.76:0/1116186289 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa802cd80 con 0x7f9a98003a50 2024-03-08T06:01:51.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.700+0000 7f9ab41c7640 1 -- 172.21.15.76:0/1116186289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a98003a50 msgr2=0x7f9a98003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:51.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.700+0000 7f9ab41c7640 1 --2- 172.21.15.76:0/1116186289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a98003a50 0x7f9a98003e70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800c240 tx=0x7f9a98005830 comp rx=0 tx=0).stop 2024-03-08T06:01:51.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.700+0000 7f9ab2764640 1 -- 172.21.15.76:0/1116186289 reap_dead start 2024-03-08T06:01:51.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.700+0000 7f9ab41c7640 1 -- 172.21.15.76:0/1116186289 shutdown_connections 2024-03-08T06:01:51.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.700+0000 7f9ab41c7640 1 -- 172.21.15.76:0/1116186289 >> 172.21.15.76:0/1116186289 conn(0x7f9aac0cf740 msgr2=0x7f9aac0cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:51.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.700+0000 7f9ab41c7640 1 -- 172.21.15.76:0/1116186289 shutdown_connections 2024-03-08T06:01:51.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.700+0000 7f9ab41c7640 1 -- 172.21.15.76:0/1116186289 wait complete. 2024-03-08T06:01:51.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.701+0000 7f9ab41c7640 1 Processor -- start 2024-03-08T06:01:51.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.701+0000 7f9ab41c7640 1 -- start start 2024-03-08T06:01:51.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab41c7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a98003a50 0x7f9aac149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab41c7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9aac14b910 0x7f9aac158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab41c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9aac1587b0 0x7f9aac15c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab41c7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac0cacf0 con 0x7f9aac1587b0 2024-03-08T06:01:51.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab41c7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac06e020 con 0x7f9aac14b910 2024-03-08T06:01:51.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab41c7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac15ad80 con 0x7f9a98003a50 2024-03-08T06:01:51.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab2764640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9aac1587b0 0x7f9aac15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab2764640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9aac1587b0 0x7f9aac15c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51314/0 (socket says 172.21.15.76:51314) 2024-03-08T06:01:51.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab2764640 1 -- 172.21.15.76:0/2042362106 learned_addr learned my addr 172.21.15.76:0/2042362106 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:51.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab1f63640 1 --2- 172.21.15.76:0/2042362106 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a98003a50 0x7f9aac149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab1762640 1 --2- 172.21.15.76:0/2042362106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9aac14b910 0x7f9aac158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab2764640 1 -- 172.21.15.76:0/2042362106 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a98003a50 msgr2=0x7f9aac149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:51.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab2764640 1 --2- 172.21.15.76:0/2042362106 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a98003a50 0x7f9aac149bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab2764640 1 -- 172.21.15.76:0/2042362106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9aac14b910 msgr2=0x7f9aac158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:51.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab2764640 1 --2- 172.21.15.76:0/2042362106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9aac14b910 0x7f9aac158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab2764640 1 -- 172.21.15.76:0/2042362106 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac0af260 con 0x7f9aac1587b0 2024-03-08T06:01:51.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab1f63640 1 --2- 172.21.15.76:0/2042362106 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a98003a50 0x7f9aac149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:51.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.702+0000 7f9ab2764640 1 --2- 172.21.15.76:0/2042362106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9aac1587b0 0x7f9aac15c330 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0cd220 tx=0x7f9aa400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:51.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.703+0000 7f9a9e7fc640 1 -- 172.21.15.76:0/2042362106 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4003970 con 0x7f9aac1587b0 2024-03-08T06:01:51.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.703+0000 7f9a9e7fc640 1 -- 172.21.15.76:0/2042362106 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa4018070 con 0x7f9aac1587b0 2024-03-08T06:01:51.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.703+0000 7f9ab41c7640 1 -- 172.21.15.76:0/2042362106 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aac0af540 con 0x7f9aac1587b0 2024-03-08T06:01:51.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.703+0000 7f9a9e7fc640 1 -- 172.21.15.76:0/2042362106 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4003d50 con 0x7f9aac1587b0 2024-03-08T06:01:51.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.703+0000 7f9ab41c7640 1 -- 172.21.15.76:0/2042362106 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aac0afb00 con 0x7f9aac1587b0 2024-03-08T06:01:51.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.703+0000 7f9ab41c7640 1 -- 172.21.15.76:0/2042362106 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aac04f030 con 0x7f9aac1587b0 2024-03-08T06:01:51.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.704+0000 7f9a9e7fc640 1 -- 172.21.15.76:0/2042362106 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9aa40050d0 con 0x7f9aac1587b0 2024-03-08T06:01:51.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.704+0000 7f9a9e7fc640 1 --2- 172.21.15.76:0/2042362106 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9a9006b4d0 0x7f9a9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:51.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.704+0000 7f9a9e7fc640 1 -- 172.21.15.76:0/2042362106 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9aa408a020 con 0x7f9aac1587b0 2024-03-08T06:01:51.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.704+0000 7f9ab1f63640 1 --2- 172.21.15.76:0/2042362106 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9a9006b4d0 0x7f9a9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:51.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.705+0000 7f9ab1f63640 1 --2- 172.21.15.76:0/2042362106 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9a9006b4d0 0x7f9a9006d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8034000 tx=0x7f9aa8009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:51.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.709+0000 7f9a9e7fc640 1 -- 172.21.15.76:0/2042362106 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9aa4058d10 con 0x7f9aac1587b0 2024-03-08T06:01:51.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.916+0000 7f9ab41c7640 1 -- 172.21.15.76:0/2042362106 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9aac0ca1e0 con 0x7f9aac1587b0 2024-03-08T06:01:51.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.917+0000 7f9a9e7fc640 1 -- 172.21.15.76:0/2042362106 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f9aa4058610 con 0x7f9aac1587b0 2024-03-08T06:01:51.918 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:01:51.918 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/606148582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":606148582},{"type":"v1","addr":"172.21.15.81:6835","nonce":606148582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/278988477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":278988477},{"type":"v1","addr":"172.21.15.76:6836","nonce":278988477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2526423312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":2526423312},{"type":"v1","addr":"172.21.15.76:6837","nonce":2526423312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:01:47.232184+0000","modified":"2024-03-08T06:01:49.227002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5358":{"gid":5358,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6834/1503895376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1503895376},{"type":"v1","addr":"172.21.15.81:6834","nonce":1503895376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T06:01:51.918 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 15 2024-03-08T06:01:51.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.919+0000 7f9ab41c7640 1 -- 172.21.15.76:0/2042362106 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9a9006b4d0 msgr2=0x7f9a9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.919+0000 7f9ab41c7640 1 --2- 172.21.15.76:0/2042362106 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9a9006b4d0 0x7f9a9006d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8034000 tx=0x7f9aa8009030 comp rx=0 tx=0).stop 2024-03-08T06:01:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.919+0000 7f9ab41c7640 1 -- 172.21.15.76:0/2042362106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9aac1587b0 msgr2=0x7f9aac15c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.919+0000 7f9ab41c7640 1 --2- 172.21.15.76:0/2042362106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9aac1587b0 0x7f9aac15c330 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0cd220 tx=0x7f9aa400ec10 comp rx=0 tx=0).stop 2024-03-08T06:01:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.919+0000 7f9ab41c7640 1 -- 172.21.15.76:0/2042362106 shutdown_connections 2024-03-08T06:01:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.919+0000 7f9ab41c7640 1 --2- 172.21.15.76:0/2042362106 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f9a9006b4d0 0x7f9a9006d990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.919+0000 7f9ab41c7640 1 --2- 172.21.15.76:0/2042362106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9aac1587b0 0x7f9aac15c330 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.919+0000 7f9ab41c7640 1 --2- 172.21.15.76:0/2042362106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9aac14b910 0x7f9aac158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.919+0000 7f9ab41c7640 1 --2- 172.21.15.76:0/2042362106 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a98003a50 0x7f9aac149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.919+0000 7f9ab41c7640 1 -- 172.21.15.76:0/2042362106 >> 172.21.15.76:0/2042362106 conn(0x7f9aac0cf740 msgr2=0x7f9aac0c8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.919+0000 7f9ab41c7640 1 -- 172.21.15.76:0/2042362106 shutdown_connections 2024-03-08T06:01:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:51.919+0000 7f9ab41c7640 1 -- 172.21.15.76:0/2042362106 wait complete. 2024-03-08T06:01:52.333 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-08T06:01:47.232184+0000', 'modified': '2024-03-08T06:01:49.227002+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5358}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5358': {'gid': 5358, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.81:6834/1503895376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6832', 'nonce': 1503895376}, {'type': 'v1', 'addr': '172.21.15.81:6834', 'nonce': 1503895376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-08T06:01:52.334 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-08T06:01:52.334 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-08T06:01:52.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.436+0000 7f2378c8f640 1 Processor -- start 2024-03-08T06:01:52.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.436+0000 7f2378c8f640 1 -- start start 2024-03-08T06:01:52.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.437+0000 7f2378c8f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2374161210 0x7f23740cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.437+0000 7f2378c8f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2374070470 con 0x7f237406d040 2024-03-08T06:01:52.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.437+0000 7f2378c8f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237406bfa0 con 0x7f2374071670 2024-03-08T06:01:52.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.437+0000 7f2378c8f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237406c280 con 0x7f2374161210 2024-03-08T06:01:52.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.437+0000 7f237259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2374161210 0x7f23740cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.438+0000 7f237259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2374161210 0x7f23740cfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51328/0 (socket says 172.21.15.76:51328) 2024-03-08T06:01:52.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.438+0000 7f2371d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f237406d040 0x7f2374070f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39278/0 (socket says 172.21.15.76:39278) 2024-03-08T06:01:52.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.438+0000 7f237259c640 1 -- 172.21.15.76:0/2729671139 learned_addr learned my addr 172.21.15.76:0/2729671139 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:52.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.438+0000 7f237259c640 1 -- 172.21.15.76:0/2729671139 >> v1:172.21.15.81:6789/0 conn(0x7f2374071670 legacy=0x7f23740cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:52.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.438+0000 7f237259c640 1 -- 172.21.15.76:0/2729671139 >> v1:172.21.15.76:6789/0 conn(0x7f237406d040 legacy=0x7f2374070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:52.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.438+0000 7f237159a640 1 -- 172.21.15.76:0/2729671139 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747934574 0 0) 0x7f2364001110 con 0x7f237406d040 2024-03-08T06:01:52.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.438+0000 7f237259c640 1 -- 172.21.15.76:0/2729671139 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237406c560 con 0x7f2374161210 2024-03-08T06:01:52.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.439+0000 7f237259c640 1 --2- 172.21.15.76:0/2729671139 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2374161210 0x7f23740cfeb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2368009870 tx=0x7f236802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2db92e68fa72ec8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:52.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.439+0000 7f237159a640 1 -- 172.21.15.76:0/2729671139 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368034070 con 0x7f2374161210 2024-03-08T06:01:52.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.439+0000 7f237159a640 1 -- 172.21.15.76:0/2729671139 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2368030860 con 0x7f2374161210 2024-03-08T06:01:52.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.439+0000 7f237159a640 1 -- 172.21.15.76:0/2729671139 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368030c30 con 0x7f2374161210 2024-03-08T06:01:52.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.439+0000 7f2378c8f640 1 -- 172.21.15.76:0/2729671139 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2374161210 msgr2=0x7f23740cfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:52.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.439+0000 7f2378c8f640 1 --2- 172.21.15.76:0/2729671139 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2374161210 0x7f23740cfeb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2368009870 tx=0x7f236802b9a0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.439+0000 7f2378c8f640 1 -- 172.21.15.76:0/2729671139 shutdown_connections 2024-03-08T06:01:52.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.439+0000 7f2378c8f640 1 --2- 172.21.15.76:0/2729671139 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2374161210 0x7f23740cfeb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.439+0000 7f2378c8f640 1 -- 172.21.15.76:0/2729671139 >> 172.21.15.76:0/2729671139 conn(0x7f237406f530 msgr2=0x7f237406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:52.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.439+0000 7f2378c8f640 1 -- 172.21.15.76:0/2729671139 shutdown_connections 2024-03-08T06:01:52.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.439+0000 7f2378c8f640 1 -- 172.21.15.76:0/2729671139 wait complete. 2024-03-08T06:01:52.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.441+0000 7f2378c8f640 1 Processor -- start 2024-03-08T06:01:52.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.441+0000 7f2378c8f640 1 -- start start 2024-03-08T06:01:52.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.441+0000 7f2378c8f640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f237406d040 0x7f23740c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.441+0000 7f2378c8f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2374071670 0x7f237415ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.441+0000 7f2378c8f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2374161210 0x7f23741f3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.441+0000 7f2378c8f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237406d4d0 con 0x7f2374071670 2024-03-08T06:01:52.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.441+0000 7f2378c8f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237406cb20 con 0x7f2374161210 2024-03-08T06:01:52.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.441+0000 7f2378c8f640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237406d650 con 0x7f237406d040 2024-03-08T06:01:52.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.441+0000 7f2371d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2374071670 0x7f237415ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.442+0000 7f2371d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2374071670 0x7f237415ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51330/0 (socket says 172.21.15.76:51330) 2024-03-08T06:01:52.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.442+0000 7f2371d9b640 1 -- 172.21.15.76:0/2012077079 learned_addr learned my addr 172.21.15.76:0/2012077079 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:52.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.442+0000 7f2372d9d640 1 --2- 172.21.15.76:0/2012077079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2374161210 0x7f23741f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.442+0000 7f237259c640 1 --2- 172.21.15.76:0/2012077079 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f237406d040 0x7f23740c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.442+0000 7f2371d9b640 1 -- 172.21.15.76:0/2012077079 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f237406d040 msgr2=0x7f23740c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:52.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.442+0000 7f2371d9b640 1 --2- 172.21.15.76:0/2012077079 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f237406d040 0x7f23740c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.442+0000 7f2371d9b640 1 -- 172.21.15.76:0/2012077079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2374161210 msgr2=0x7f23741f3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:52.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.442+0000 7f2371d9b640 1 --2- 172.21.15.76:0/2012077079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2374161210 0x7f23741f3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.442+0000 7f2371d9b640 1 -- 172.21.15.76:0/2012077079 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237415b380 con 0x7f2374071670 2024-03-08T06:01:52.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.442+0000 7f2371d9b640 1 --2- 172.21.15.76:0/2012077079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2374071670 0x7f237415ae40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f236400c7b0 tx=0x7f236400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:52.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.442+0000 7f2362ffd640 1 -- 172.21.15.76:0/2012077079 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364018070 con 0x7f2374071670 2024-03-08T06:01:52.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.442+0000 7f2362ffd640 1 -- 172.21.15.76:0/2012077079 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23640072d0 con 0x7f2374071670 2024-03-08T06:01:52.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.442+0000 7f2378c8f640 1 -- 172.21.15.76:0/2012077079 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2374159da0 con 0x7f2374071670 2024-03-08T06:01:52.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.442+0000 7f2362ffd640 1 -- 172.21.15.76:0/2012077079 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23640047e0 con 0x7f2374071670 2024-03-08T06:01:52.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.442+0000 7f2378c8f640 1 -- 172.21.15.76:0/2012077079 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237415a360 con 0x7f2374071670 2024-03-08T06:01:52.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.443+0000 7f2362ffd640 1 -- 172.21.15.76:0/2012077079 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f236401b070 con 0x7f2374071670 2024-03-08T06:01:52.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.444+0000 7f2362ffd640 1 --2- 172.21.15.76:0/2012077079 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f235406b500 0x7f235406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.444+0000 7f2362ffd640 1 -- 172.21.15.76:0/2012077079 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f23640895d0 con 0x7f2374071670 2024-03-08T06:01:52.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.444+0000 7f237259c640 1 --2- 172.21.15.76:0/2012077079 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f235406b500 0x7f235406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.444+0000 7f237259c640 1 --2- 172.21.15.76:0/2012077079 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f235406b500 0x7f235406d9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2368036000 tx=0x7f236803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:52.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.445+0000 7f237b861740 1 Processor -- start 2024-03-08T06:01:52.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.445+0000 7f237b861740 1 -- start start 2024-03-08T06:01:52.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f237b861740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0f97870 0x558dc0f97c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f237b861740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0f9b8c0 0x558dc0f9d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f237b861740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fc7600 0x558dc0fc7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f237b861740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc0f016b0 con 0x558dc0f97870 2024-03-08T06:01:52.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f237b861740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc0eaaa60 con 0x558dc0fc7600 2024-03-08T06:01:52.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f237b861740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x558dc0efd750 con 0x558dc0f9b8c0 2024-03-08T06:01:52.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f237259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0f97870 0x558dc0f97c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f237259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0f97870 0x558dc0f97c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51340/0 (socket says 172.21.15.76:51340) 2024-03-08T06:01:52.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f237259c640 1 -- 172.21.15.76:0/780158091 learned_addr learned my addr 172.21.15.76:0/780158091 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:52.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f2371d9b640 1 --2- 172.21.15.76:0/780158091 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0f9b8c0 0x558dc0f9d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f2372d9d640 1 --2- 172.21.15.76:0/780158091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fc7600 0x558dc0fc7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f237259c640 1 -- 172.21.15.76:0/780158091 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0f9b8c0 msgr2=0x558dc0f9d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:52.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f237259c640 1 --2- 172.21.15.76:0/780158091 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0f9b8c0 0x558dc0f9d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f237259c640 1 -- 172.21.15.76:0/780158091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fc7600 msgr2=0x558dc0fc7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:52.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f237259c640 1 --2- 172.21.15.76:0/780158091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fc7600 0x558dc0fc7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f237259c640 1 -- 172.21.15.76:0/780158091 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dc0ec8f30 con 0x558dc0f97870 2024-03-08T06:01:52.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f237259c640 1 --2- 172.21.15.76:0/780158091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0f97870 0x558dc0f97c50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f23680090b0 tx=0x7f2368030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754f14546044de80 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:52.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f2353fff640 1 -- 172.21.15.76:0/780158091 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368053070 con 0x558dc0f97870 2024-03-08T06:01:52.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.446+0000 7f2353fff640 1 -- 172.21.15.76:0/780158091 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2368039aa0 con 0x558dc0f97870 2024-03-08T06:01:52.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.447+0000 7f2353fff640 1 -- 172.21.15.76:0/780158091 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368034040 con 0x558dc0f97870 2024-03-08T06:01:52.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.447+0000 7f237b861740 1 -- 172.21.15.76:0/780158091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0f97870 msgr2=0x558dc0f97c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:52.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.447+0000 7f237b861740 1 --2- 172.21.15.76:0/780158091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0f97870 0x558dc0f97c50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f23680090b0 tx=0x7f2368030000 comp rx=0 tx=0).stop 2024-03-08T06:01:52.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.447+0000 7f237b861740 1 -- 172.21.15.76:0/780158091 shutdown_connections 2024-03-08T06:01:52.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.447+0000 7f237b861740 1 --2- 172.21.15.76:0/780158091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fc7600 0x558dc0fc7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.447+0000 7f237b861740 1 --2- 172.21.15.76:0/780158091 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0f9b8c0 0x558dc0f9d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.447+0000 7f237b861740 1 --2- 172.21.15.76:0/780158091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0f97870 0x558dc0f97c50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.447+0000 7f237b861740 1 -- 172.21.15.76:0/780158091 >> 172.21.15.76:0/780158091 conn(0x558dc0f998f0 msgr2=0x558dc0f9b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:52.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.447+0000 7f237b861740 1 -- 172.21.15.76:0/780158091 shutdown_connections 2024-03-08T06:01:52.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.447+0000 7f237b861740 1 -- 172.21.15.76:0/780158091 wait complete. 2024-03-08T06:01:52.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.448+0000 7f237b861740 1 -- start start 2024-03-08T06:01:52.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.448+0000 7f237b861740 1 Processor -- start 2024-03-08T06:01:52.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f237b861740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0fc7600 0x558dc0fbc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f237b861740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fba4f0 0x558dc0fba930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f237b861740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fc2340 0x558dc0fc27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f237b861740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc0f9bd50 con 0x558dc0fba4f0 2024-03-08T06:01:52.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f237b861740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc0ebdea0 con 0x558dc0fc2340 2024-03-08T06:01:52.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f237b861740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x558dc0f9c030 con 0x558dc0fc7600 2024-03-08T06:01:52.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f237b861740 10 client.0 _ll_register_callbacks cb 0x558dc0ebd870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:01:52.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f2371d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fba4f0 0x558dc0fba930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f2371d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fba4f0 0x558dc0fba930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51348/0 (socket says 172.21.15.76:51348) 2024-03-08T06:01:52.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f2371d9b640 1 -- 172.21.15.76:0/3066779303 learned_addr learned my addr 172.21.15.76:0/3066779303 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:52.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f237259c640 1 --2- 172.21.15.76:0/3066779303 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0fc7600 0x558dc0fbc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f2372d9d640 1 --2- 172.21.15.76:0/3066779303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fc2340 0x558dc0fc27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f2371d9b640 1 -- 172.21.15.76:0/3066779303 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0fc7600 msgr2=0x558dc0fbc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:52.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f2371d9b640 1 --2- 172.21.15.76:0/3066779303 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0fc7600 0x558dc0fbc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f2371d9b640 1 -- 172.21.15.76:0/3066779303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fc2340 msgr2=0x558dc0fc27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:52.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f2371d9b640 1 --2- 172.21.15.76:0/3066779303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fc2340 0x558dc0fc27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f2371d9b640 1 -- 172.21.15.76:0/3066779303 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558dc0ebe270 con 0x558dc0fba4f0 2024-03-08T06:01:52.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f2371d9b640 1 --2- 172.21.15.76:0/3066779303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fba4f0 0x558dc0fba930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2364087a00 tx=0x7f2364054020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fd0f3ff9f6dff4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:52.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f23517fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-08T06:01:52.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f23517fa640 1 -- 172.21.15.76:0/3066779303 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23640572a0 con 0x558dc0fba4f0 2024-03-08T06:01:52.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f23517fa640 1 -- 172.21.15.76:0/3066779303 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f23640890b0 con 0x558dc0fba4f0 2024-03-08T06:01:52.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.449+0000 7f237b861740 1 -- 172.21.15.76:0/3066779303 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2368039aa0 con 0x558dc0fba4f0 2024-03-08T06:01:52.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.450+0000 7f23517fa640 1 -- 172.21.15.76:0/3066779303 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f236405e070 con 0x558dc0fba4f0 2024-03-08T06:01:52.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.450+0000 7f237b861740 10 client.5526 fetch_fsmap learned FSMap version 15 2024-03-08T06:01:52.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.450+0000 7f237b861740 1 -- 172.21.15.76:0/3066779303 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x558dc0c162d0 con 0x558dc0fba4f0 2024-03-08T06:01:52.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.450+0000 7f23517fa640 1 -- 172.21.15.76:0/3066779303 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2368039aa0 con 0x558dc0fba4f0 2024-03-08T06:01:52.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.450+0000 7f23517fa640 1 -- 172.21.15.76:0/3066779303 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f23640892b0 con 0x558dc0fba4f0 2024-03-08T06:01:52.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.450+0000 7f237b861740 10 client.5526 fetch_fsmap finished waiting for FSMap version 15 2024-03-08T06:01:52.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.450+0000 7f237b861740 10 client.5526 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-08T06:01:52.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.450+0000 7f237b861740 20 client.5526 populate_metadata read hostname 'smithi076' 2024-03-08T06:01:52.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.450+0000 7f237b861740 1 --2- 172.21.15.76:0/3066779303 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x558dc0fc3d00 0x558dc1001880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.450+0000 7f237b861740 4 client.5526 mds_command: new command op to 5358 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T06:01:52.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.450+0000 7f237b861740 1 -- 172.21.15.76:0/3066779303 --> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558dc0f0ebf0 con 0x558dc0fc3d00 2024-03-08T06:01:52.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.450+0000 7f237259c640 1 --2- 172.21.15.76:0/3066779303 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x558dc0fc3d00 0x558dc1001880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.451+0000 7f237259c640 1 --2- 172.21.15.76:0/3066779303 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x558dc0fc3d00 0x558dc1001880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94875674305eff13 server_cookie=e35592000075ef53 in_seq=0 out_seq=0 2024-03-08T06:01:52.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.451+0000 7f23517fa640 10 client.5526 ms_handle_connect on v2:172.21.15.81:6832/1503895376 2024-03-08T06:01:52.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.451+0000 7f23517fa640 1 -- 172.21.15.76:0/3066779303 <== mds.0 v2:172.21.15.81:6832/1503895376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558dc0f0ebf0 con 0x558dc0fc3d00 2024-03-08T06:01:52.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.451+0000 7f23517fa640 10 client.5526 handle_command_reply: tid=0 2024-03-08T06:01:52.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.451+0000 7f237b861740 1 client.5526 shutdown 2024-03-08T06:01:52.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.452+0000 7f237b861740 1 -- 172.21.15.76:0/3066779303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fba4f0 msgr2=0x558dc0fba930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:52.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.452+0000 7f237b861740 1 --2- 172.21.15.76:0/3066779303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fba4f0 0x558dc0fba930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2364087a00 tx=0x7f2364054020 comp rx=0 tx=0).stop 2024-03-08T06:01:52.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.452+0000 7f237b861740 1 -- 172.21.15.76:0/3066779303 shutdown_connections 2024-03-08T06:01:52.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.452+0000 7f237b861740 1 --2- 172.21.15.76:0/3066779303 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x558dc0fc3d00 0x558dc1001880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.452+0000 7f237b861740 1 --2- 172.21.15.76:0/3066779303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fc2340 0x558dc0fc27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.452+0000 7f237b861740 1 --2- 172.21.15.76:0/3066779303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fba4f0 0x558dc0fba930 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.452+0000 7f237b861740 1 --2- 172.21.15.76:0/3066779303 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0fc7600 0x558dc0fbc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.452+0000 7f23517fa640 0 client.5526 ms_handle_reset on v2:172.21.15.81:6832/1503895376 2024-03-08T06:01:52.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.452+0000 7f237b861740 1 -- 172.21.15.76:0/3066779303 >> 172.21.15.76:0/3066779303 conn(0x558dc0f97870 msgr2=0x558dc0fc55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:52.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.452+0000 7f237b861740 1 -- 172.21.15.76:0/3066779303 shutdown_connections 2024-03-08T06:01:52.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.452+0000 7f237b861740 1 -- 172.21.15.76:0/3066779303 wait complete. 2024-03-08T06:01:52.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.452+0000 7f237b861740 20 client.5526 trim_cache size 0 max 16384 2024-03-08T06:01:52.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.467+0000 7f237b861740 1 Processor -- start 2024-03-08T06:01:52.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.467+0000 7f237b861740 1 -- start start 2024-03-08T06:01:52.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.468+0000 7f237b861740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fba4f0 0x558dc0f9df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.468+0000 7f237b861740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0fc2340 0x558dc0f96510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.468+0000 7f237b861740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fc7600 0x558dc0f96a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.468+0000 7f237b861740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23742007c0 con 0x558dc0fc7600 2024-03-08T06:01:52.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.468+0000 7f237b861740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23741f7530 con 0x558dc0fba4f0 2024-03-08T06:01:52.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.468+0000 7f237b861740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237414d820 con 0x558dc0fc2340 2024-03-08T06:01:52.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.468+0000 7f2372d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fc7600 0x558dc0f96a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.468+0000 7f2372d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fc7600 0x558dc0f96a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51354/0 (socket says 172.21.15.76:51354) 2024-03-08T06:01:52.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.468+0000 7f2372d9d640 1 -- 172.21.15.76:0/118406183 learned_addr learned my addr 172.21.15.76:0/118406183 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:52.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f2371d9b640 1 --2- 172.21.15.76:0/118406183 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0fc2340 0x558dc0f96510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f237259c640 1 --2- 172.21.15.76:0/118406183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fba4f0 0x558dc0f9df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f2372d9d640 1 -- 172.21.15.76:0/118406183 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0fc2340 msgr2=0x558dc0f96510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:52.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f2372d9d640 1 --2- 172.21.15.76:0/118406183 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0fc2340 0x558dc0f96510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f2372d9d640 1 -- 172.21.15.76:0/118406183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fba4f0 msgr2=0x558dc0f9df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:52.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f2372d9d640 1 --2- 172.21.15.76:0/118406183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fba4f0 0x558dc0f9df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f2372d9d640 1 -- 172.21.15.76:0/118406183 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dc0ce06b0 con 0x558dc0fc7600 2024-03-08T06:01:52.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f2371d9b640 1 --2- 172.21.15.76:0/118406183 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0fc2340 0x558dc0f96510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:52.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f2372d9d640 1 --2- 172.21.15.76:0/118406183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fc7600 0x558dc0f96a50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f236c005560 tx=0x7f236c0136a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c88b906b64a7dc server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f23537fe640 1 -- 172.21.15.76:0/118406183 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c01a030 con 0x558dc0fc7600 2024-03-08T06:01:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f23537fe640 1 -- 172.21.15.76:0/118406183 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c013e50 con 0x558dc0fc7600 2024-03-08T06:01:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f23537fe640 1 -- 172.21.15.76:0/118406183 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c006b50 con 0x558dc0fc7600 2024-03-08T06:01:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f237b861740 1 -- 172.21.15.76:0/118406183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fc7600 msgr2=0x558dc0f96a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f237b861740 1 --2- 172.21.15.76:0/118406183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fc7600 0x558dc0f96a50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f236c005560 tx=0x7f236c0136a0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f237b861740 1 -- 172.21.15.76:0/118406183 shutdown_connections 2024-03-08T06:01:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f237b861740 1 --2- 172.21.15.76:0/118406183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fc7600 0x558dc0f96a50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f237b861740 1 --2- 172.21.15.76:0/118406183 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0fc2340 0x558dc0f96510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f237b861740 1 --2- 172.21.15.76:0/118406183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fba4f0 0x558dc0f9df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f237b861740 1 -- 172.21.15.76:0/118406183 >> 172.21.15.76:0/118406183 conn(0x558dc0f97870 msgr2=0x558dc0f92e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f237b861740 1 -- 172.21.15.76:0/118406183 shutdown_connections 2024-03-08T06:01:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.469+0000 7f237b861740 1 -- 172.21.15.76:0/118406183 wait complete. 2024-03-08T06:01:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.470+0000 7f237b861740 1 -- start start 2024-03-08T06:01:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.470+0000 7f237b861740 1 Processor -- start 2024-03-08T06:01:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f237b861740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fba4f0 0x558dc0f96510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f237b861740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0fc2340 0x558dc0f96a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f237b861740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fc7600 0x558dc0fc7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f237b861740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc0fbfed0 con 0x558dc0fba4f0 2024-03-08T06:01:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f237b861740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237414de20 con 0x558dc0fc7600 2024-03-08T06:01:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f237b861740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x558dc0fc9f60 con 0x558dc0fc2340 2024-03-08T06:01:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f237b861740 10 client.0 _ll_register_callbacks cb 0x558dc0fc0cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:01:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f237259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fba4f0 0x558dc0f96510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f237259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fba4f0 0x558dc0f96510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51370/0 (socket says 172.21.15.76:51370) 2024-03-08T06:01:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f237259c640 1 -- 172.21.15.76:0/2791715309 learned_addr learned my addr 172.21.15.76:0/2791715309 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f2371d9b640 1 --2- 172.21.15.76:0/2791715309 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0fc2340 0x558dc0f96a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f2372d9d640 1 --2- 172.21.15.76:0/2791715309 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fc7600 0x558dc0fc7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f237259c640 1 -- 172.21.15.76:0/2791715309 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0fc2340 msgr2=0x558dc0f96a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f237259c640 1 --2- 172.21.15.76:0/2791715309 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0fc2340 0x558dc0f96a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f237259c640 1 -- 172.21.15.76:0/2791715309 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fc7600 msgr2=0x558dc0fc7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f237259c640 1 --2- 172.21.15.76:0/2791715309 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fc7600 0x558dc0fc7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f237259c640 1 -- 172.21.15.76:0/2791715309 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558dc10565a0 con 0x558dc0fba4f0 2024-03-08T06:01:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f2372d9d640 1 --2- 172.21.15.76:0/2791715309 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fc7600 0x558dc0fc7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f237259c640 1 --2- 172.21.15.76:0/2791715309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fba4f0 0x558dc0f96510 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f23680090b0 tx=0x7f236802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b34cda942b6d39 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.471+0000 7f23517fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-08T06:01:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.472+0000 7f23517fa640 1 -- 172.21.15.76:0/2791715309 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368053040 con 0x558dc0fba4f0 2024-03-08T06:01:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.472+0000 7f23517fa640 1 -- 172.21.15.76:0/2791715309 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f23680308e0 con 0x558dc0fba4f0 2024-03-08T06:01:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.472+0000 7f237b861740 1 -- 172.21.15.76:0/2791715309 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f236c013e50 con 0x558dc0fba4f0 2024-03-08T06:01:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.472+0000 7f23517fa640 1 -- 172.21.15.76:0/2791715309 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2368066050 con 0x558dc0fba4f0 2024-03-08T06:01:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.472+0000 7f237b861740 10 client.5532 fetch_fsmap learned FSMap version 15 2024-03-08T06:01:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.472+0000 7f237b861740 1 -- 172.21.15.76:0/2791715309 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x558dc0fbe2f0 con 0x558dc0fba4f0 2024-03-08T06:01:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.472+0000 7f23517fa640 1 -- 172.21.15.76:0/2791715309 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c013e50 con 0x558dc0fba4f0 2024-03-08T06:01:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.472+0000 7f23517fa640 1 -- 172.21.15.76:0/2791715309 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2368069020 con 0x558dc0fba4f0 2024-03-08T06:01:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.472+0000 7f237b861740 10 client.5532 fetch_fsmap finished waiting for FSMap version 15 2024-03-08T06:01:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.472+0000 7f237b861740 10 client.5532 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-08T06:01:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.472+0000 7f237b861740 20 client.5532 populate_metadata read hostname 'smithi076' 2024-03-08T06:01:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.472+0000 7f237b861740 1 --2- 172.21.15.76:0/2791715309 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x558dc0fc8680 0x558dc0f9bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.472+0000 7f237b861740 4 client.5532 mds_command: new command op to 5358 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T06:01:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.472+0000 7f237b861740 1 -- 172.21.15.76:0/2791715309 --> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558dc0fc0520 con 0x558dc0fc8680 2024-03-08T06:01:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.472+0000 7f2371d9b640 1 --2- 172.21.15.76:0/2791715309 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x558dc0fc8680 0x558dc0f9bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f2371d9b640 1 --2- 172.21.15.76:0/2791715309 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x558dc0fc8680 0x558dc0f9bf60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ca414093daf0647 server_cookie=cf595d7baaae9d60 in_seq=0 out_seq=0 2024-03-08T06:01:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f23517fa640 10 client.5532 ms_handle_connect on v2:172.21.15.81:6832/1503895376 2024-03-08T06:01:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f23517fa640 1 -- 172.21.15.76:0/2791715309 <== mds.0 v2:172.21.15.81:6832/1503895376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x558dc0fc0520 con 0x558dc0fc8680 2024-03-08T06:01:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f23517fa640 10 client.5532 handle_command_reply: tid=0 2024-03-08T06:01:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f237b861740 1 client.5532 shutdown 2024-03-08T06:01:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f237b861740 1 -- 172.21.15.76:0/2791715309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fba4f0 msgr2=0x558dc0f96510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f237b861740 1 --2- 172.21.15.76:0/2791715309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fba4f0 0x558dc0f96510 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f23680090b0 tx=0x7f236802e000 comp rx=0 tx=0).stop 2024-03-08T06:01:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f237b861740 1 -- 172.21.15.76:0/2791715309 shutdown_connections 2024-03-08T06:01:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f237b861740 1 --2- 172.21.15.76:0/2791715309 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x558dc0fc8680 0x558dc0f9bf60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f237b861740 1 --2- 172.21.15.76:0/2791715309 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558dc0fc7600 0x558dc0fc7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f237b861740 1 --2- 172.21.15.76:0/2791715309 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x558dc0fc2340 0x558dc0f96a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f237b861740 1 --2- 172.21.15.76:0/2791715309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dc0fba4f0 0x558dc0f96510 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f23517fa640 0 client.5532 ms_handle_reset on v2:172.21.15.81:6832/1503895376 2024-03-08T06:01:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f237b861740 1 -- 172.21.15.76:0/2791715309 >> 172.21.15.76:0/2791715309 conn(0x558dc0f97870 msgr2=0x558dc0fc2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:52.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f237b861740 1 -- 172.21.15.76:0/2791715309 shutdown_connections 2024-03-08T06:01:52.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f237b861740 1 -- 172.21.15.76:0/2791715309 wait complete. 2024-03-08T06:01:52.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.473+0000 7f237b861740 20 client.5532 trim_cache size 0 max 16384 2024-03-08T06:01:52.482 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-08T06:01:52.482 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "8a42235b-b7c3-45b2-aa98-336e6b1846d1", 2024-03-08T06:01:52.482 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-03-08T06:01:52.482 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 5358, 2024-03-08T06:01:52.482 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-03-08T06:01:52.483 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-03-08T06:01:52.483 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-03-08T06:01:52.483 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 4.5981633119999996, 2024-03-08T06:01:52.483 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 15, 2024-03-08T06:01:52.483 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 40, 2024-03-08T06:01:52.483 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 40, 2024-03-08T06:01:52.483 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 11.472718259000001 2024-03-08T06:01:52.483 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-08T06:01:52.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.474+0000 7f2378c8f640 1 -- 172.21.15.76:0/2012077079 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f235406b500 msgr2=0x7f235406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:52.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.474+0000 7f2378c8f640 1 --2- 172.21.15.76:0/2012077079 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f235406b500 0x7f235406d9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2368036000 tx=0x7f236803b040 comp rx=0 tx=0).stop 2024-03-08T06:01:52.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.474+0000 7f2378c8f640 1 -- 172.21.15.76:0/2012077079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2374071670 msgr2=0x7f237415ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:52.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.474+0000 7f2378c8f640 1 --2- 172.21.15.76:0/2012077079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2374071670 0x7f237415ae40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f236400c7b0 tx=0x7f236400cc70 comp rx=0 tx=0).stop 2024-03-08T06:01:52.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.474+0000 7f2378c8f640 1 -- 172.21.15.76:0/2012077079 shutdown_connections 2024-03-08T06:01:52.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.474+0000 7f2378c8f640 1 --2- 172.21.15.76:0/2012077079 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f235406b500 0x7f235406d9c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.474+0000 7f2378c8f640 1 --2- 172.21.15.76:0/2012077079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2374161210 0x7f23741f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.474+0000 7f2378c8f640 1 --2- 172.21.15.76:0/2012077079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2374071670 0x7f237415ae40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.474+0000 7f2378c8f640 1 --2- 172.21.15.76:0/2012077079 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f237406d040 0x7f23740c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.474+0000 7f2378c8f640 1 -- 172.21.15.76:0/2012077079 >> 172.21.15.76:0/2012077079 conn(0x7f237406f530 msgr2=0x7f23741615f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:52.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.474+0000 7f2378c8f640 1 -- 172.21.15.76:0/2012077079 shutdown_connections 2024-03-08T06:01:52.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.474+0000 7f2378c8f640 1 -- 172.21.15.76:0/2012077079 wait complete. 2024-03-08T06:01:52.486 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T06:01:52.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.586+0000 7fe0fb284640 1 Processor -- start 2024-03-08T06:01:52.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.586+0000 7fe0fb284640 1 -- start start 2024-03-08T06:01:52.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.587+0000 7fe0fb284640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0f40cfa90 0x7fe0f415ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.587+0000 7fe0fb284640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe0f406c2e0 0x7fe0f406ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.587+0000 7fe0fb284640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe0f4071c70 0x7fe0f4072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.587+0000 7fe0fb284640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f40cfe70 con 0x7fe0f4071c70 2024-03-08T06:01:52.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.587+0000 7fe0fb284640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f4158670 con 0x7fe0f406c2e0 2024-03-08T06:01:52.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.587+0000 7fe0fb284640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f40725b0 con 0x7fe0f40cfa90 2024-03-08T06:01:52.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.587+0000 7fe0f9821640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe0f4071c70 0x7fe0f4072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.587+0000 7fe0f9821640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe0f4071c70 0x7fe0f4072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51384/0 (socket says 172.21.15.76:51384) 2024-03-08T06:01:52.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.587+0000 7fe0f9821640 1 -- 172.21.15.76:0/2288391613 learned_addr learned my addr 172.21.15.76:0/2288391613 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:52.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.588+0000 7fe0f9020640 1 --2- 172.21.15.76:0/2288391613 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0f40cfa90 0x7fe0f415ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.588+0000 7fe0f881f640 1 --2- 172.21.15.76:0/2288391613 >> v2:172.21.15.81:3300/0 conn(0x7fe0f406c2e0 0x7fe0f406ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.588+0000 7fe0f9821640 1 -- 172.21.15.76:0/2288391613 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0f40cfa90 msgr2=0x7fe0f415ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:52.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.588+0000 7fe0f9821640 1 --2- 172.21.15.76:0/2288391613 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0f40cfa90 0x7fe0f415ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.588+0000 7fe0f9821640 1 -- 172.21.15.76:0/2288391613 >> v2:172.21.15.81:3300/0 conn(0x7fe0f406c2e0 msgr2=0x7fe0f406ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:52.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.588+0000 7fe0f9821640 1 --2- 172.21.15.76:0/2288391613 >> v2:172.21.15.81:3300/0 conn(0x7fe0f406c2e0 0x7fe0f406ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.588+0000 7fe0f9821640 1 -- 172.21.15.76:0/2288391613 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f40ad090 con 0x7fe0f4071c70 2024-03-08T06:01:52.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.588+0000 7fe0f9020640 1 --2- 172.21.15.76:0/2288391613 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0f40cfa90 0x7fe0f415ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:01:52.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.588+0000 7fe0f9821640 1 --2- 172.21.15.76:0/2288391613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0f4071c70 0x7fe0f4072070 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe0f40ad260 tx=0x7fe0ec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8bcaae3f21a8ea server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:52.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.589+0000 7fe0ebfff640 1 -- 172.21.15.76:0/2288391613 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec035020 con 0x7fe0f4071c70 2024-03-08T06:01:52.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.589+0000 7fe0ebfff640 1 -- 172.21.15.76:0/2288391613 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ec031640 con 0x7fe0f4071c70 2024-03-08T06:01:52.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.589+0000 7fe0ebfff640 1 -- 172.21.15.76:0/2288391613 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec02d040 con 0x7fe0f4071c70 2024-03-08T06:01:52.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.589+0000 7fe0fb284640 1 -- 172.21.15.76:0/2288391613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0f4071c70 msgr2=0x7fe0f4072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:52.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.589+0000 7fe0fb284640 1 --2- 172.21.15.76:0/2288391613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0f4071c70 0x7fe0f4072070 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe0f40ad260 tx=0x7fe0ec02b8f0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.589+0000 7fe0fb284640 1 -- 172.21.15.76:0/2288391613 shutdown_connections 2024-03-08T06:01:52.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.589+0000 7fe0fb284640 1 --2- 172.21.15.76:0/2288391613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0f4071c70 0x7fe0f4072070 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.589+0000 7fe0fb284640 1 --2- 172.21.15.76:0/2288391613 >> v2:172.21.15.81:3300/0 conn(0x7fe0f406c2e0 0x7fe0f406ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.589+0000 7fe0fb284640 1 --2- 172.21.15.76:0/2288391613 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0f40cfa90 0x7fe0f415ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.589+0000 7fe0fb284640 1 -- 172.21.15.76:0/2288391613 >> 172.21.15.76:0/2288391613 conn(0x7fe0f40af650 msgr2=0x7fe0f40b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:52.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.589+0000 7fe0fb284640 1 -- 172.21.15.76:0/2288391613 shutdown_connections 2024-03-08T06:01:52.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.589+0000 7fe0fb284640 1 -- 172.21.15.76:0/2288391613 wait complete. 2024-03-08T06:01:52.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.590+0000 7fe0fb284640 1 Processor -- start 2024-03-08T06:01:52.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.590+0000 7fe0fb284640 1 -- start start 2024-03-08T06:01:52.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0fb284640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0f406c2e0 0x7fe0f41f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0fb284640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0f4071c70 0x7fe0f41f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0fb284640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0f40cfa90 0x7fe0f414cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0fb284640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f406e020 con 0x7fe0f406c2e0 2024-03-08T06:01:52.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0fb284640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f406dd40 con 0x7fe0f4071c70 2024-03-08T06:01:52.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0fb284640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f406fa70 con 0x7fe0f40cfa90 2024-03-08T06:01:52.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0f9020640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0f406c2e0 0x7fe0f41f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0f9020640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0f406c2e0 0x7fe0f41f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51392/0 (socket says 172.21.15.76:51392) 2024-03-08T06:01:52.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0f9020640 1 -- 172.21.15.76:0/1535873774 learned_addr learned my addr 172.21.15.76:0/1535873774 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:52.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0f9821640 1 --2- 172.21.15.76:0/1535873774 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0f40cfa90 0x7fe0f414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0f881f640 1 --2- 172.21.15.76:0/1535873774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0f4071c70 0x7fe0f41f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0f9020640 1 -- 172.21.15.76:0/1535873774 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0f40cfa90 msgr2=0x7fe0f414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:52.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0f9020640 1 --2- 172.21.15.76:0/1535873774 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0f40cfa90 0x7fe0f414cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0f9020640 1 -- 172.21.15.76:0/1535873774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0f4071c70 msgr2=0x7fe0f41f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:52.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0f9020640 1 --2- 172.21.15.76:0/1535873774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0f4071c70 0x7fe0f41f42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0f9020640 1 -- 172.21.15.76:0/1535873774 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f414ebf0 con 0x7fe0f406c2e0 2024-03-08T06:01:52.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0f881f640 1 --2- 172.21.15.76:0/1535873774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0f4071c70 0x7fe0f41f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:52.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.591+0000 7fe0f9821640 1 --2- 172.21.15.76:0/1535873774 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0f40cfa90 0x7fe0f414cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:52.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.592+0000 7fe0f9020640 1 --2- 172.21.15.76:0/1535873774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0f406c2e0 0x7fe0f41f4a00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009fd0 tx=0x7fe0f000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:52.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.592+0000 7fe0e9ffb640 1 -- 172.21.15.76:0/1535873774 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f001a070 con 0x7fe0f406c2e0 2024-03-08T06:01:52.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.592+0000 7fe0e9ffb640 1 -- 172.21.15.76:0/1535873774 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f0003bb0 con 0x7fe0f406c2e0 2024-03-08T06:01:52.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.592+0000 7fe0fb284640 1 -- 172.21.15.76:0/1535873774 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f414eed0 con 0x7fe0f406c2e0 2024-03-08T06:01:52.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.592+0000 7fe0e9ffb640 1 -- 172.21.15.76:0/1535873774 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0004030 con 0x7fe0f406c2e0 2024-03-08T06:01:52.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.592+0000 7fe0fb284640 1 -- 172.21.15.76:0/1535873774 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f414f490 con 0x7fe0f406c2e0 2024-03-08T06:01:52.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.593+0000 7fe0fb284640 1 -- 172.21.15.76:0/1535873774 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f404f030 con 0x7fe0f406c2e0 2024-03-08T06:01:52.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.593+0000 7fe0e9ffb640 1 -- 172.21.15.76:0/1535873774 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0f001d070 con 0x7fe0f406c2e0 2024-03-08T06:01:52.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.593+0000 7fe0e9ffb640 1 --2- 172.21.15.76:0/1535873774 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe0d806b4d0 0x7fe0d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:52.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.594+0000 7fe0e9ffb640 1 -- 172.21.15.76:0/1535873774 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe0f008a4d0 con 0x7fe0f406c2e0 2024-03-08T06:01:52.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.594+0000 7fe0f881f640 1 --2- 172.21.15.76:0/1535873774 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe0d806b4d0 0x7fe0d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:52.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.594+0000 7fe0f881f640 1 --2- 172.21.15.76:0/1535873774 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe0d806b4d0 0x7fe0d806d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4150cf0 tx=0x7fe0e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:52.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.598+0000 7fe0e9ffb640 1 -- 172.21.15.76:0/1535873774 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0f0010070 con 0x7fe0f406c2e0 2024-03-08T06:01:52.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.805+0000 7fe0fb284640 1 -- 172.21.15.76:0/1535873774 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe0f40ae140 con 0x7fe0f406c2e0 2024-03-08T06:01:52.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.805+0000 7fe0e9ffb640 1 -- 172.21.15.76:0/1535873774 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe0f0058c00 con 0x7fe0f406c2e0 2024-03-08T06:01:52.806 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-08T06:01:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.807+0000 7fe0fb284640 1 -- 172.21.15.76:0/1535873774 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe0d806b4d0 msgr2=0x7fe0d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.807+0000 7fe0fb284640 1 --2- 172.21.15.76:0/1535873774 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe0d806b4d0 0x7fe0d806d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4150cf0 tx=0x7fe0e4008040 comp rx=0 tx=0).stop 2024-03-08T06:01:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.807+0000 7fe0fb284640 1 -- 172.21.15.76:0/1535873774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0f406c2e0 msgr2=0x7fe0f41f4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.807+0000 7fe0fb284640 1 --2- 172.21.15.76:0/1535873774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0f406c2e0 0x7fe0f41f4a00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009fd0 tx=0x7fe0f000d560 comp rx=0 tx=0).stop 2024-03-08T06:01:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.807+0000 7fe0fb284640 1 -- 172.21.15.76:0/1535873774 shutdown_connections 2024-03-08T06:01:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.807+0000 7fe0fb284640 1 --2- 172.21.15.76:0/1535873774 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe0d806b4d0 0x7fe0d806d990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.807+0000 7fe0fb284640 1 --2- 172.21.15.76:0/1535873774 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0f40cfa90 0x7fe0f414cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.807+0000 7fe0fb284640 1 --2- 172.21.15.76:0/1535873774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0f4071c70 0x7fe0f41f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.807+0000 7fe0fb284640 1 --2- 172.21.15.76:0/1535873774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0f406c2e0 0x7fe0f41f4a00 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:52.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.807+0000 7fe0fb284640 1 -- 172.21.15.76:0/1535873774 >> 172.21.15.76:0/1535873774 conn(0x7fe0f40af650 msgr2=0x7fe0f4160970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:52.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.807+0000 7fe0fb284640 1 -- 172.21.15.76:0/1535873774 shutdown_connections 2024-03-08T06:01:52.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:52.807+0000 7fe0fb284640 1 -- 172.21.15.76:0/1535873774 wait complete. 2024-03-08T06:01:52.818 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-08T06:01:52.818 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T06:01:52.818 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-03-08T06:01:52.818 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-08T06:01:52.818 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-08T06:01:52.819 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-08T06:01:52.819 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-08T06:01:52.819 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T06:01:52.819 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-08T06:01:52.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:52.819 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-03-08T06:01:52.834 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:01:52.834 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:01:52.834 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:01:52.834 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:52.834 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-03-08T06:01:52.834 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:52.834 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:52.834 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:52.834 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-03-08T06:01:52.834 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:52.835 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:52.835 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-03-08T06:01:52.835 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:52.835 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:52.835 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T06:01:52.835 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:52.835 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-08T06:01:52.835 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42694sec preferred_lft 42694sec 2024-03-08T06:01:52.835 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-03-08T06:01:52.835 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:52.835 INFO:teuthology.orchestra.run.smithi076.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T06:01:52.835 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 12:50:99:e1:10:c2 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:52.835 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-08T06:01:52.835 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:52.835 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::b48c:6dff:fea8:e46d/64 scope link 2024-03-08T06:01:52.835 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:52.836 INFO:teuthology.orchestra.run.smithi076.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T06:01:52.836 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 16:af:2b:28:6e:f4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T06:01:52.836 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::14af:2bff:fe28:6ef4/64 scope link 2024-03-08T06:01:52.836 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:52.836 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T06:01:52.836 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 12:50:99:e1:10:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-08T06:01:52.836 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::1050:99ff:fee1:10c2/64 scope link 2024-03-08T06:01:52.836 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:52.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:52.836 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T06:01:52.859 INFO:tasks.ceph.mds.b.smithi081.stderr:2024-03-08T06:01:52.858+0000 7f5129836640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-08T06:01:52.900 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-03-08T06:01:52.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:52.902 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-03-08T06:01:52.958 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-03-08T06:01:52.958 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T06:01:52.958 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-08T06:01:52.958 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-08T06:01:52.958 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T06:01:52.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:52.959 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2024-03-08T06:01:52.959 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-08T06:01:52.959 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-08T06:01:52.959 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-08T06:01:52.959 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-08T06:01:52.959 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-08T06:01:53.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:53.054 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-03-08T06:01:53.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-08T06:01:53.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-08T06:01:53.069 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-08T06:01:53.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:53.069 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2024-03-08T06:01:53.136 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:01:53.136 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-08T06:01:53.190 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-08T06:01:53.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:01:53.191 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-08T06:01:53.245 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-08T06:01:53.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:53.245 DEBUG:teuthology.orchestra.run.smithi076:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:53.330 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:53.330 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T06:01:53.343 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:53.343 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T06:01:53.343 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.76:3300,172.21.15.81:3300,172.21.15.81:3301" 2024-03-08T06:01:53.344 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 0@8a42235b-b7c3-45b2-aa98-336e6b1846d1.cephfs=/ 2024-03-08T06:01:53.344 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.76:3300/172.21.15.81:3300/172.21.15.81:3301" will pass to kernel 2024-03-08T06:01:53.344 INFO:teuthology.orchestra.run.smithi076.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-08T06:01:53.344 INFO:teuthology.orchestra.run.smithi076.stdout: You just mounted a file system that supports labels which does not 2024-03-08T06:01:53.344 INFO:teuthology.orchestra.run.smithi076.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T06:01:53.345 INFO:teuthology.orchestra.run.smithi076.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T06:01:53.345 INFO:teuthology.orchestra.run.smithi076.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T06:01:53.348 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T06:01:53.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:53.348 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-08T06:01:53.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:53.377 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:01:53.441 INFO:teuthology.orchestra.run.smithi076.stdout:client5541 2024-03-08T06:01:53.441 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:01:53.441 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:01:53.497 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T06:01:53.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.636+0000 7f81e7835640 1 Processor -- start 2024-03-08T06:01:53.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.636+0000 7f81e7835640 1 -- start start 2024-03-08T06:01:53.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.637+0000 7f81e7835640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f81e006ac80 0x7f81e006b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:53.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.637+0000 7f81e7835640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f81e00cc4c0 0x7f81e00cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:53.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.637+0000 7f81e7835640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e015c2b0 con 0x7f81e015bb70 2024-03-08T06:01:53.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.637+0000 7f81e7835640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e015cec0 con 0x7f81e006ac80 2024-03-08T06:01:53.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.637+0000 7f81e7835640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e015c590 con 0x7f81e00cc4c0 2024-03-08T06:01:53.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.637+0000 7f81e4dd0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f81e006ac80 0x7f81e006b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:53.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.637+0000 7f81e4dd0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f81e006ac80 0x7f81e006b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51400/0 (socket says 172.21.15.76:51400) 2024-03-08T06:01:53.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.637+0000 7f81e4dd0640 1 -- 172.21.15.76:0/207294127 learned_addr learned my addr 172.21.15.76:0/207294127 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:53.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.638+0000 7f81e4dd0640 1 -- 172.21.15.76:0/207294127 >> v2:172.21.15.81:3300/0 conn(0x7f81e00cc4c0 msgr2=0x7f81e00cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:53.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.638+0000 7f81e5dd2640 1 --2- 172.21.15.76:0/207294127 >> v2:172.21.15.81:3300/0 conn(0x7f81e00cc4c0 0x7f81e00cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:53.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.638+0000 7f81e4dd0640 1 --2- 172.21.15.76:0/207294127 >> v2:172.21.15.81:3300/0 conn(0x7f81e00cc4c0 0x7f81e00cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:53.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.638+0000 7f81e4dd0640 1 -- 172.21.15.76:0/207294127 >> v1:172.21.15.81:6789/0 conn(0x7f81e015bb70 legacy=0x7f81e00d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:01:53.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.638+0000 7f81e4dd0640 1 -- 172.21.15.76:0/207294127 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e015c870 con 0x7f81e006ac80 2024-03-08T06:01:53.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.638+0000 7f81e5dd2640 1 --2- 172.21.15.76:0/207294127 >> v2:172.21.15.81:3300/0 conn(0x7f81e00cc4c0 0x7f81e00cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:53.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.638+0000 7f81e4dd0640 1 --2- 172.21.15.76:0/207294127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81e006ac80 0x7f81e006b130 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f81e00c4330 tx=0x7f81d002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d05392befc666ea server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:53.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.638+0000 7f81d7fff640 1 -- 172.21.15.76:0/207294127 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0035020 con 0x7f81e006ac80 2024-03-08T06:01:53.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.639+0000 7f81d7fff640 1 -- 172.21.15.76:0/207294127 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d0031870 con 0x7f81e006ac80 2024-03-08T06:01:53.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.639+0000 7f81d7fff640 1 -- 172.21.15.76:0/207294127 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0031c40 con 0x7f81e006ac80 2024-03-08T06:01:53.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.639+0000 7f81e7835640 1 -- 172.21.15.76:0/207294127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81e006ac80 msgr2=0x7f81e006b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:53.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.639+0000 7f81e7835640 1 --2- 172.21.15.76:0/207294127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81e006ac80 0x7f81e006b130 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f81e00c4330 tx=0x7f81d002b910 comp rx=0 tx=0).stop 2024-03-08T06:01:53.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.639+0000 7f81e7835640 1 -- 172.21.15.76:0/207294127 shutdown_connections 2024-03-08T06:01:53.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.639+0000 7f81e7835640 1 --2- 172.21.15.76:0/207294127 >> v2:172.21.15.81:3300/0 conn(0x7f81e00cc4c0 0x7f81e00cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:53.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.639+0000 7f81e7835640 1 --2- 172.21.15.76:0/207294127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81e006ac80 0x7f81e006b130 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:53.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.639+0000 7f81e7835640 1 -- 172.21.15.76:0/207294127 >> 172.21.15.76:0/207294127 conn(0x7f81e0099a60 msgr2=0x7f81e00d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:53.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.639+0000 7f81e7835640 1 -- 172.21.15.76:0/207294127 shutdown_connections 2024-03-08T06:01:53.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.639+0000 7f81e7835640 1 -- 172.21.15.76:0/207294127 wait complete. 2024-03-08T06:01:53.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.640+0000 7f81e7835640 1 Processor -- start 2024-03-08T06:01:53.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.640+0000 7f81e7835640 1 -- start start 2024-03-08T06:01:53.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.641+0000 7f81e7835640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f81e006ac80 0x7f81e00b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:53.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.641+0000 7f81e7835640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81e00cc4c0 0x7f81e00b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:53.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.641+0000 7f81e7835640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e015bb70 0x7f81e00c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:53.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.641+0000 7f81e7835640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e015bf50 con 0x7f81e00cc4c0 2024-03-08T06:01:53.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.641+0000 7f81e7835640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e00d0510 con 0x7f81e015bb70 2024-03-08T06:01:53.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.641+0000 7f81e7835640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e00d1670 con 0x7f81e006ac80 2024-03-08T06:01:53.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.641+0000 7f81e4dd0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81e00cc4c0 0x7f81e00b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:53.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.641+0000 7f81e4dd0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81e00cc4c0 0x7f81e00b23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51416/0 (socket says 172.21.15.76:51416) 2024-03-08T06:01:53.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.641+0000 7f81e4dd0640 1 -- 172.21.15.76:0/3861398827 learned_addr learned my addr 172.21.15.76:0/3861398827 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:01:53.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.641+0000 7f81e4dd0640 1 -- 172.21.15.76:0/3861398827 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f81e006ac80 msgr2=0x7f81e00b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:53.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.641+0000 7f81e5dd2640 1 --2- 172.21.15.76:0/3861398827 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e015bb70 0x7f81e00c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.641+0000 7f81e55d1640 1 --2- 172.21.15.76:0/3861398827 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f81e006ac80 0x7f81e00b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.642+0000 7f81e4dd0640 1 --2- 172.21.15.76:0/3861398827 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f81e006ac80 0x7f81e00b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.642+0000 7f81e4dd0640 1 -- 172.21.15.76:0/3861398827 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e015bb70 msgr2=0x7f81e00c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.642+0000 7f81e4dd0640 1 --2- 172.21.15.76:0/3861398827 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e015bb70 0x7f81e00c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.642+0000 7f81e4dd0640 1 -- 172.21.15.76:0/3861398827 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e00c37e0 con 0x7f81e00cc4c0 2024-03-08T06:01:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.642+0000 7f81e55d1640 1 --2- 172.21.15.76:0/3861398827 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f81e006ac80 0x7f81e00b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.642+0000 7f81e5dd2640 1 --2- 172.21.15.76:0/3861398827 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e015bb70 0x7f81e00c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:01:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.642+0000 7f81e4dd0640 1 --2- 172.21.15.76:0/3861398827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81e00cc4c0 0x7f81e00b23c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f81d0000c00 tx=0x7f81d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.642+0000 7f81d5ffb640 1 -- 172.21.15.76:0/3861398827 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0040070 con 0x7f81e00cc4c0 2024-03-08T06:01:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.642+0000 7f81d5ffb640 1 -- 172.21.15.76:0/3861398827 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d003a420 con 0x7f81e00cc4c0 2024-03-08T06:01:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.642+0000 7f81e7835640 1 -- 172.21.15.76:0/3861398827 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e01f7560 con 0x7f81e00cc4c0 2024-03-08T06:01:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.642+0000 7f81d5ffb640 1 -- 172.21.15.76:0/3861398827 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0035040 con 0x7f81e00cc4c0 2024-03-08T06:01:53.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.642+0000 7f81e7835640 1 -- 172.21.15.76:0/3861398827 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e01f7b20 con 0x7f81e00cc4c0 2024-03-08T06:01:53.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.643+0000 7f81e7835640 1 -- 172.21.15.76:0/3861398827 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e015f110 con 0x7f81e00cc4c0 2024-03-08T06:01:53.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.643+0000 7f81d5ffb640 1 -- 172.21.15.76:0/3861398827 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81d004b060 con 0x7f81e00cc4c0 2024-03-08T06:01:53.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.644+0000 7f81d5ffb640 1 --2- 172.21.15.76:0/3861398827 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f81c406b4d0 0x7f81c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:01:53.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.644+0000 7f81d5ffb640 1 -- 172.21.15.76:0/3861398827 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f81d00ac2d0 con 0x7f81e00cc4c0 2024-03-08T06:01:53.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.644+0000 7f81e55d1640 1 --2- 172.21.15.76:0/3861398827 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f81c406b4d0 0x7f81c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:01:53.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.644+0000 7f81e55d1640 1 --2- 172.21.15.76:0/3861398827 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f81c406b4d0 0x7f81c406d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f81e00c9ec0 tx=0x7f81dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:01:53.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.648+0000 7f81d5ffb640 1 -- 172.21.15.76:0/3861398827 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81e015f110 con 0x7f81e00cc4c0 2024-03-08T06:01:53.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.854+0000 7f81e7835640 1 -- 172.21.15.76:0/3861398827 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f81e004f030 con 0x7f81e00cc4c0 2024-03-08T06:01:53.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.854+0000 7f81d5ffb640 1 -- 172.21.15.76:0/3861398827 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f81d007afc0 con 0x7f81e00cc4c0 2024-03-08T06:01:53.855 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-08T06:01:53.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.857+0000 7f81e7835640 1 -- 172.21.15.76:0/3861398827 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f81c406b4d0 msgr2=0x7f81c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:53.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.857+0000 7f81e7835640 1 --2- 172.21.15.76:0/3861398827 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f81c406b4d0 0x7f81c406d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f81e00c9ec0 tx=0x7f81dc009040 comp rx=0 tx=0).stop 2024-03-08T06:01:53.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.857+0000 7f81e7835640 1 -- 172.21.15.76:0/3861398827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81e00cc4c0 msgr2=0x7f81e00b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:01:53.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.857+0000 7f81e7835640 1 --2- 172.21.15.76:0/3861398827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81e00cc4c0 0x7f81e00b23c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f81d0000c00 tx=0x7f81d00046c0 comp rx=0 tx=0).stop 2024-03-08T06:01:53.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.857+0000 7f81e7835640 1 -- 172.21.15.76:0/3861398827 shutdown_connections 2024-03-08T06:01:53.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.857+0000 7f81e7835640 1 --2- 172.21.15.76:0/3861398827 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f81c406b4d0 0x7f81c406d990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:53.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.857+0000 7f81e7835640 1 --2- 172.21.15.76:0/3861398827 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e015bb70 0x7f81e00c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:53.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.857+0000 7f81e7835640 1 --2- 172.21.15.76:0/3861398827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81e00cc4c0 0x7f81e00b23c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:53.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.857+0000 7f81e7835640 1 --2- 172.21.15.76:0/3861398827 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f81e006ac80 0x7f81e00b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:01:53.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.857+0000 7f81e7835640 1 -- 172.21.15.76:0/3861398827 >> 172.21.15.76:0/3861398827 conn(0x7f81e0099a60 msgr2=0x7f81e00cd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:01:53.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.857+0000 7f81e7835640 1 -- 172.21.15.76:0/3861398827 shutdown_connections 2024-03-08T06:01:53.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:01:53.857+0000 7f81e7835640 1 -- 172.21.15.76:0/3861398827 wait complete. 2024-03-08T06:01:53.868 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-08T06:01:53.868 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T06:01:53.868 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2024-03-08T06:01:53.868 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-08T06:01:53.868 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-08T06:01:53.868 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-08T06:01:53.869 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-08T06:01:53.869 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T06:01:53.869 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-08T06:01:53.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:53.869 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2024-03-08T06:01:53.889 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:01:53.889 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:01:53.889 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:01:53.889 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:53.889 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-03-08T06:01:53.889 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:53.889 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:53.889 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:53.889 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2024-03-08T06:01:53.890 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:53.890 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:53.890 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2024-03-08T06:01:53.890 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:01:53.890 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:53.890 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T06:01:53.890 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:53.890 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-08T06:01:53.890 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42634sec preferred_lft 42634sec 2024-03-08T06:01:53.890 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2024-03-08T06:01:53.890 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:53.891 INFO:teuthology.orchestra.run.smithi081.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T06:01:53.891 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 52:ef:1d:a5:da:97 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:01:53.891 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-08T06:01:53.891 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:53.891 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::f0d7:8dff:fe91:22e0/64 scope link 2024-03-08T06:01:53.891 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:53.891 INFO:teuthology.orchestra.run.smithi081.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T06:01:53.891 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 52:ef:1d:a5:da:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-08T06:01:53.891 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::50ef:1dff:fea5:da97/64 scope link 2024-03-08T06:01:53.891 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:53.891 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T06:01:53.891 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 62:d8:34:21:ae:a7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-08T06:01:53.891 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::60d8:34ff:fe21:aea7/64 scope link 2024-03-08T06:01:53.892 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:01:53.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:53.892 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T06:01:53.959 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-03-08T06:01:53.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:53.961 DEBUG:teuthology.orchestra.run.smithi081:> route 2024-03-08T06:01:54.019 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2024-03-08T06:01:54.020 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T06:01:54.020 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-08T06:01:54.020 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-08T06:01:54.020 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T06:01:54.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:54.020 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-08T06:01:54.020 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-08T06:01:54.020 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-08T06:01:54.021 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-08T06:01:54.021 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-08T06:01:54.021 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-08T06:01:54.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:54.118 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-03-08T06:01:54.134 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-08T06:01:54.134 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-08T06:01:54.134 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-08T06:01:54.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:54.135 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2024-03-08T06:01:54.202 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:01:54.203 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-08T06:01:54.258 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-08T06:01:54.258 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:01:54.258 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-08T06:01:54.313 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-08T06:01:54.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:54.314 DEBUG:teuthology.orchestra.run.smithi081:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:54.398 INFO:teuthology.orchestra.run.smithi081.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:54.399 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T06:01:54.406 INFO:teuthology.orchestra.run.smithi081.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T06:01:54.406 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T06:01:54.406 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: resolved to: "172.21.15.76:3300,172.21.15.81:3300,172.21.15.81:3301" 2024-03-08T06:01:54.406 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: trying mount with new device syntax: 1@8a42235b-b7c3-45b2-aa98-336e6b1846d1.cephfs=/ 2024-03-08T06:01:54.406 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.76:3300/172.21.15.81:3300/172.21.15.81:3301" will pass to kernel 2024-03-08T06:01:54.407 INFO:teuthology.orchestra.run.smithi081.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-08T06:01:54.407 INFO:teuthology.orchestra.run.smithi081.stdout: You just mounted a file system that supports labels which does not 2024-03-08T06:01:54.407 INFO:teuthology.orchestra.run.smithi081.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T06:01:54.407 INFO:teuthology.orchestra.run.smithi081.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T06:01:54.407 INFO:teuthology.orchestra.run.smithi081.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T06:01:54.409 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T06:01:54.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:54.409 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-08T06:01:54.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:01:54.436 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:01:54.492 INFO:teuthology.orchestra.run.smithi081.stdout:client5550 2024-03-08T06:01:54.493 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:01:54.493 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:02:04.549 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T06:02:04.549 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 counter dump 2024-03-08T06:02:04.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.662+0000 7f457c9d7640 1 Processor -- start 2024-03-08T06:02:04.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.662+0000 7f457c9d7640 1 -- start start 2024-03-08T06:02:04.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f457c9d7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4578156070 0x7f457815cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f457c9d7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f457806fb60 0x7f457806e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f457c9d7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4578071750 0x7f4578071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f457c9d7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4578072100 con 0x7f4578071750 2024-03-08T06:02:04.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f457c9d7640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f457806ffe0 con 0x7f457806fb60 2024-03-08T06:02:04.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f457c9d7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457806f6d0 con 0x7f4578156070 2024-03-08T06:02:04.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f4576d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4578071750 0x7f4578071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f4576d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4578071750 0x7f4578071bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60670/0 (socket says 172.21.15.76:60670) 2024-03-08T06:02:04.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f4576d9d640 1 -- 172.21.15.76:0/1384716962 learned_addr learned my addr 172.21.15.76:0/1384716962 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:04.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f4575d9b640 1 --2- 172.21.15.76:0/1384716962 >> v2:172.21.15.81:3300/0 conn(0x7f457806fb60 0x7f457806e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f457659c640 1 --2- 172.21.15.76:0/1384716962 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4578156070 0x7f457815cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f4576d9d640 1 -- 172.21.15.76:0/1384716962 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4578156070 msgr2=0x7f457815cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:04.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f4576d9d640 1 --2- 172.21.15.76:0/1384716962 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4578156070 0x7f457815cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f4576d9d640 1 -- 172.21.15.76:0/1384716962 >> v2:172.21.15.81:3300/0 conn(0x7f457806fb60 msgr2=0x7f457806e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:04.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f4576d9d640 1 --2- 172.21.15.76:0/1384716962 >> v2:172.21.15.81:3300/0 conn(0x7f457806fb60 0x7f457806e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f4576d9d640 1 -- 172.21.15.76:0/1384716962 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457806eac0 con 0x7f4578071750 2024-03-08T06:02:04.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.663+0000 7f4575d9b640 1 --2- 172.21.15.76:0/1384716962 >> v2:172.21.15.81:3300/0 conn(0x7f457806fb60 0x7f457806e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:04.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.664+0000 7f4576d9d640 1 --2- 172.21.15.76:0/1384716962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4578071750 0x7f4578071bc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f45780cfa70 tx=0x7f457002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5758402ae859ed9f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:04.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.664+0000 7f457559a640 1 -- 172.21.15.76:0/1384716962 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457002ca70 con 0x7f4578071750 2024-03-08T06:02:04.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.664+0000 7f457559a640 1 -- 172.21.15.76:0/1384716962 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457002cc10 con 0x7f4578071750 2024-03-08T06:02:04.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.664+0000 7f457559a640 1 -- 172.21.15.76:0/1384716962 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45700343f0 con 0x7f4578071750 2024-03-08T06:02:04.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.665+0000 7f457c9d7640 1 -- 172.21.15.76:0/1384716962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4578071750 msgr2=0x7f4578071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:04.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.665+0000 7f457c9d7640 1 --2- 172.21.15.76:0/1384716962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4578071750 0x7f4578071bc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f45780cfa70 tx=0x7f457002bed0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.665+0000 7f457c9d7640 1 -- 172.21.15.76:0/1384716962 shutdown_connections 2024-03-08T06:02:04.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.665+0000 7f457c9d7640 1 --2- 172.21.15.76:0/1384716962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4578071750 0x7f4578071bc0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.665+0000 7f457c9d7640 1 --2- 172.21.15.76:0/1384716962 >> v2:172.21.15.81:3300/0 conn(0x7f457806fb60 0x7f457806e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.665+0000 7f457c9d7640 1 --2- 172.21.15.76:0/1384716962 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4578156070 0x7f457815cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.665+0000 7f457c9d7640 1 -- 172.21.15.76:0/1384716962 >> 172.21.15.76:0/1384716962 conn(0x7f45780cc600 msgr2=0x7f45780cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:04.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.665+0000 7f457c9d7640 1 -- 172.21.15.76:0/1384716962 shutdown_connections 2024-03-08T06:02:04.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.665+0000 7f457c9d7640 1 -- 172.21.15.76:0/1384716962 wait complete. 2024-03-08T06:02:04.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.666+0000 7f457c9d7640 1 Processor -- start 2024-03-08T06:02:04.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.666+0000 7f457c9d7640 1 -- start start 2024-03-08T06:02:04.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f457c9d7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f457806fb60 0x7f4578144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f457c9d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4578071750 0x7f4578142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f457c9d7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4578156070 0x7f4578142930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f457c9d7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457806f360 con 0x7f4578071750 2024-03-08T06:02:04.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f457c9d7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457806f080 con 0x7f457806fb60 2024-03-08T06:02:04.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f457c9d7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457806e760 con 0x7f4578156070 2024-03-08T06:02:04.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f4575d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4578071750 0x7f4578142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f4575d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4578071750 0x7f4578142180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60686/0 (socket says 172.21.15.76:60686) 2024-03-08T06:02:04.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f4575d9b640 1 -- 172.21.15.76:0/3575311940 learned_addr learned my addr 172.21.15.76:0/3575311940 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:04.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f4576d9d640 1 --2- 172.21.15.76:0/3575311940 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4578156070 0x7f4578142930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f457659c640 1 --2- 172.21.15.76:0/3575311940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f457806fb60 0x7f4578144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f4575d9b640 1 -- 172.21.15.76:0/3575311940 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4578156070 msgr2=0x7f4578142930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:04.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f4575d9b640 1 --2- 172.21.15.76:0/3575311940 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4578156070 0x7f4578142930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f4575d9b640 1 -- 172.21.15.76:0/3575311940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f457806fb60 msgr2=0x7f4578144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:04.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f4575d9b640 1 --2- 172.21.15.76:0/3575311940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f457806fb60 0x7f4578144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f4575d9b640 1 -- 172.21.15.76:0/3575311940 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4578143280 con 0x7f4578071750 2024-03-08T06:02:04.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f4576d9d640 1 --2- 172.21.15.76:0/3575311940 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4578156070 0x7f4578142930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:04.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f457659c640 1 --2- 172.21.15.76:0/3575311940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f457806fb60 0x7f4578144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:04.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.667+0000 7f4575d9b640 1 --2- 172.21.15.76:0/3575311940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4578071750 0x7f4578142180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f456c00e810 tx=0x7f456c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:04.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.668+0000 7f4566ffd640 1 -- 172.21.15.76:0/3575311940 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c00f040 con 0x7f4578071750 2024-03-08T06:02:04.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.668+0000 7f4566ffd640 1 -- 172.21.15.76:0/3575311940 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c004030 con 0x7f4578071750 2024-03-08T06:02:04.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.668+0000 7f4566ffd640 1 -- 172.21.15.76:0/3575311940 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c004400 con 0x7f4578071750 2024-03-08T06:02:04.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.668+0000 7f457c9d7640 1 -- 172.21.15.76:0/3575311940 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4578143560 con 0x7f4578071750 2024-03-08T06:02:04.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.668+0000 7f457c9d7640 1 -- 172.21.15.76:0/3575311940 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4578143b20 con 0x7f4578071750 2024-03-08T06:02:04.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.669+0000 7f4566ffd640 1 -- 172.21.15.76:0/3575311940 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f456c00c6d0 con 0x7f4578071750 2024-03-08T06:02:04.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.669+0000 7f4566ffd640 1 --2- 172.21.15.76:0/3575311940 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f455806b4d0 0x7f455806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.670+0000 7f4566ffd640 1 -- 172.21.15.76:0/3575311940 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f456c08a0a0 con 0x7f4578071750 2024-03-08T06:02:04.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.670+0000 7f457659c640 1 --2- 172.21.15.76:0/3575311940 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f455806b4d0 0x7f455806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.670+0000 7f457659c640 1 --2- 172.21.15.76:0/3575311940 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f455806b4d0 0x7f455806d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4568006fd0 tx=0x7f4568008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:04.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.671+0000 7f457f5b3740 1 Processor -- start 2024-03-08T06:02:04.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.671+0000 7f457f5b3740 1 -- start start 2024-03-08T06:02:04.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.671+0000 7f457f5b3740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8c28a0 0x559bbf8c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.671+0000 7f457f5b3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8c68f0 0x559bbf8c8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.671+0000 7f457f5b3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8f2630 0x559bbf8f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.671+0000 7f457f5b3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbf82ce30 con 0x559bbf8c68f0 2024-03-08T06:02:04.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.671+0000 7f457f5b3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbf7d5d20 con 0x559bbf8f2630 2024-03-08T06:02:04.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.671+0000 7f457f5b3740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x559bbf828ed0 con 0x559bbf8c28a0 2024-03-08T06:02:04.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.672+0000 7f4575d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8c68f0 0x559bbf8c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.672+0000 7f4575d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8c68f0 0x559bbf8c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60692/0 (socket says 172.21.15.76:60692) 2024-03-08T06:02:04.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.672+0000 7f4575d9b640 1 -- 172.21.15.76:0/666305801 learned_addr learned my addr 172.21.15.76:0/666305801 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:04.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.672+0000 7f4576d9d640 1 --2- 172.21.15.76:0/666305801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8f2630 0x559bbf8f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.672+0000 7f457659c640 1 --2- 172.21.15.76:0/666305801 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8c28a0 0x559bbf8c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.672+0000 7f4575d9b640 1 -- 172.21.15.76:0/666305801 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8c28a0 msgr2=0x559bbf8c2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:04.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.672+0000 7f4575d9b640 1 --2- 172.21.15.76:0/666305801 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8c28a0 0x559bbf8c2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.672+0000 7f4575d9b640 1 -- 172.21.15.76:0/666305801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8f2630 msgr2=0x559bbf8f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:04.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.672+0000 7f4575d9b640 1 --2- 172.21.15.76:0/666305801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8f2630 0x559bbf8f2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.672+0000 7f4575d9b640 1 -- 172.21.15.76:0/666305801 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bbf5413f0 con 0x559bbf8c68f0 2024-03-08T06:02:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.672+0000 7f4575d9b640 1 --2- 172.21.15.76:0/666305801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8c68f0 0x559bbf8c8390 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f456c060870 tx=0x7f456c058750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02b5fb5de8732a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.672+0000 7f4557fff640 1 -- 172.21.15.76:0/666305801 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c020040 con 0x559bbf8c68f0 2024-03-08T06:02:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.672+0000 7f4557fff640 1 -- 172.21.15.76:0/666305801 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c0701d0 con 0x559bbf8c68f0 2024-03-08T06:02:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.672+0000 7f4557fff640 1 -- 172.21.15.76:0/666305801 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c054d00 con 0x559bbf8c68f0 2024-03-08T06:02:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.673+0000 7f457f5b3740 1 -- 172.21.15.76:0/666305801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8c68f0 msgr2=0x559bbf8c8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.673+0000 7f457f5b3740 1 --2- 172.21.15.76:0/666305801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8c68f0 0x559bbf8c8390 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f456c060870 tx=0x7f456c058750 comp rx=0 tx=0).stop 2024-03-08T06:02:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.673+0000 7f457f5b3740 1 -- 172.21.15.76:0/666305801 shutdown_connections 2024-03-08T06:02:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.673+0000 7f457f5b3740 1 --2- 172.21.15.76:0/666305801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8f2630 0x559bbf8f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.673+0000 7f457f5b3740 1 --2- 172.21.15.76:0/666305801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8c68f0 0x559bbf8c8390 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.673+0000 7f457f5b3740 1 --2- 172.21.15.76:0/666305801 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8c28a0 0x559bbf8c2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.673+0000 7f457f5b3740 1 -- 172.21.15.76:0/666305801 >> 172.21.15.76:0/666305801 conn(0x559bbf8c4920 msgr2=0x559bbf8c61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.673+0000 7f457f5b3740 1 -- 172.21.15.76:0/666305801 shutdown_connections 2024-03-08T06:02:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.673+0000 7f457f5b3740 1 -- 172.21.15.76:0/666305801 wait complete. 2024-03-08T06:02:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.674+0000 7f457f5b3740 1 -- start start 2024-03-08T06:02:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.674+0000 7f457f5b3740 1 Processor -- start 2024-03-08T06:02:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.674+0000 7f457f5b3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8f2630 0x559bbf90ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.674+0000 7f457f5b3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8e6c70 0x559bbf8ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.674+0000 7f457f5b3740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8eea10 0x559bbf8ead10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f457f5b3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbf8c6d80 con 0x559bbf8f2630 2024-03-08T06:02:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f457f5b3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbf7cb390 con 0x559bbf8e6c70 2024-03-08T06:02:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f457f5b3740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x559bbf8c7060 con 0x559bbf8eea10 2024-03-08T06:02:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f457f5b3740 10 client.0 _ll_register_callbacks cb 0x559bbf7e8c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:02:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f457659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8f2630 0x559bbf90ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f457659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8f2630 0x559bbf90ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60704/0 (socket says 172.21.15.76:60704) 2024-03-08T06:02:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f457659c640 1 -- 172.21.15.76:0/4008385996 learned_addr learned my addr 172.21.15.76:0/4008385996 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f4576d9d640 1 --2- 172.21.15.76:0/4008385996 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8eea10 0x559bbf8ead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f4575d9b640 1 --2- 172.21.15.76:0/4008385996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8e6c70 0x559bbf8ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f457659c640 1 -- 172.21.15.76:0/4008385996 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8eea10 msgr2=0x559bbf8ead10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:04.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f457659c640 1 --2- 172.21.15.76:0/4008385996 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8eea10 0x559bbf8ead10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f457659c640 1 -- 172.21.15.76:0/4008385996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8e6c70 msgr2=0x559bbf8ee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:04.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f457659c640 1 --2- 172.21.15.76:0/4008385996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8e6c70 0x559bbf8ee4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f457659c640 1 -- 172.21.15.76:0/4008385996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bbf60b7d0 con 0x559bbf8f2630 2024-03-08T06:02:04.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f4576d9d640 1 --2- 172.21.15.76:0/4008385996 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8eea10 0x559bbf8ead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:04.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f4575d9b640 1 --2- 172.21.15.76:0/4008385996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8e6c70 0x559bbf8ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:04.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f457659c640 1 --2- 172.21.15.76:0/4008385996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8f2630 0x559bbf90ab80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f45680144b0 tx=0x7f4568004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cf2c3a89880fc server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:04.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f45557fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-08T06:02:04.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f45557fa640 1 -- 172.21.15.76:0/4008385996 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568015820 con 0x559bbf8f2630 2024-03-08T06:02:04.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f45557fa640 1 -- 172.21.15.76:0/4008385996 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f45680159c0 con 0x559bbf8f2630 2024-03-08T06:02:04.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f457f5b3740 1 -- 172.21.15.76:0/4008385996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f456c0701d0 con 0x559bbf8f2630 2024-03-08T06:02:04.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f45557fa640 1 -- 172.21.15.76:0/4008385996 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4568015bc0 con 0x559bbf8f2630 2024-03-08T06:02:04.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f457f5b3740 10 client.5562 fetch_fsmap learned FSMap version 15 2024-03-08T06:02:04.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.675+0000 7f457f5b3740 1 -- 172.21.15.76:0/4008385996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x559bbf8ea510 con 0x559bbf8f2630 2024-03-08T06:02:04.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.676+0000 7f45557fa640 1 -- 172.21.15.76:0/4008385996 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c0701d0 con 0x559bbf8f2630 2024-03-08T06:02:04.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.676+0000 7f45557fa640 1 -- 172.21.15.76:0/4008385996 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4568031080 con 0x559bbf8f2630 2024-03-08T06:02:04.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.676+0000 7f457f5b3740 10 client.5562 fetch_fsmap finished waiting for FSMap version 15 2024-03-08T06:02:04.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.676+0000 7f457f5b3740 10 client.5562 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-03-08T06:02:04.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.676+0000 7f457f5b3740 20 client.5562 populate_metadata read hostname 'smithi076' 2024-03-08T06:02:04.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.676+0000 7f457f5b3740 1 --2- 172.21.15.76:0/4008385996 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x559bbf8eb720 0x559bbf8c7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.676+0000 7f457f5b3740 4 client.5562 mds_command: new command op to 5358 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T06:02:04.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.676+0000 7f457f5b3740 1 -- 172.21.15.76:0/4008385996 --> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559bbf83a370 con 0x559bbf8eb720 2024-03-08T06:02:04.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.676+0000 7f4575d9b640 1 --2- 172.21.15.76:0/4008385996 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x559bbf8eb720 0x559bbf8c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.676+0000 7f45557fa640 10 client.5562 ms_handle_connect on v2:172.21.15.81:6832/1503895376 2024-03-08T06:02:04.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.676+0000 7f4575d9b640 1 --2- 172.21.15.76:0/4008385996 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x559bbf8eb720 0x559bbf8c7240 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=850e086a245f00aa server_cookie=e378ddc51a6fa569 in_seq=0 out_seq=0 2024-03-08T06:02:04.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.677+0000 7f45557fa640 1 -- 172.21.15.76:0/4008385996 <== mds.0 v2:172.21.15.81:6832/1503895376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559bbf83a370 con 0x559bbf8eb720 2024-03-08T06:02:04.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.677+0000 7f45557fa640 10 client.5562 handle_command_reply: tid=0 2024-03-08T06:02:04.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.677+0000 7f457f5b3740 1 client.5562 shutdown 2024-03-08T06:02:04.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.677+0000 7f457f5b3740 1 -- 172.21.15.76:0/4008385996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8f2630 msgr2=0x559bbf90ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:04.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.677+0000 7f457f5b3740 1 --2- 172.21.15.76:0/4008385996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8f2630 0x559bbf90ab80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f45680144b0 tx=0x7f4568004730 comp rx=0 tx=0).stop 2024-03-08T06:02:04.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.677+0000 7f457f5b3740 1 -- 172.21.15.76:0/4008385996 shutdown_connections 2024-03-08T06:02:04.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.677+0000 7f457f5b3740 1 --2- 172.21.15.76:0/4008385996 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x559bbf8eb720 0x559bbf8c7240 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.677+0000 7f457f5b3740 1 --2- 172.21.15.76:0/4008385996 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8eea10 0x559bbf8ead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.677+0000 7f457f5b3740 1 --2- 172.21.15.76:0/4008385996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8e6c70 0x559bbf8ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.677+0000 7f45557fa640 0 client.5562 ms_handle_reset on v2:172.21.15.81:6832/1503895376 2024-03-08T06:02:04.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.677+0000 7f457f5b3740 1 --2- 172.21.15.76:0/4008385996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8f2630 0x559bbf90ab80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.677+0000 7f457f5b3740 1 -- 172.21.15.76:0/4008385996 >> 172.21.15.76:0/4008385996 conn(0x559bbf8c28a0 msgr2=0x559bbf8c15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:04.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.677+0000 7f457f5b3740 1 -- 172.21.15.76:0/4008385996 shutdown_connections 2024-03-08T06:02:04.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.677+0000 7f457f5b3740 1 -- 172.21.15.76:0/4008385996 wait complete. 2024-03-08T06:02:04.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.678+0000 7f457f5b3740 20 client.5562 trim_cache size 0 max 16384 2024-03-08T06:02:04.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.689+0000 7f457f5b3740 1 Processor -- start 2024-03-08T06:02:04.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.689+0000 7f457f5b3740 1 -- start start 2024-03-08T06:02:04.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.690+0000 7f457f5b3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8e6c70 0x559bbf8e8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.690+0000 7f457f5b3740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8eea10 0x559bbf911260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.690+0000 7f457f5b3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8f2630 0x559bbf8f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.690+0000 7f457f5b3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45781fe290 con 0x559bbf8f2630 2024-03-08T06:02:04.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.690+0000 7f457f5b3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45781fd4a0 con 0x559bbf8e6c70 2024-03-08T06:02:04.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.690+0000 7f457f5b3740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45780b3610 con 0x559bbf8eea10 2024-03-08T06:02:04.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.690+0000 7f4576d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8f2630 0x559bbf8f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.690+0000 7f4576d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8f2630 0x559bbf8f5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60708/0 (socket says 172.21.15.76:60708) 2024-03-08T06:02:04.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.690+0000 7f4576d9d640 1 -- 172.21.15.76:0/2077084587 learned_addr learned my addr 172.21.15.76:0/2077084587 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:04.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.690+0000 7f457659c640 1 --2- 172.21.15.76:0/2077084587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8e6c70 0x559bbf8e8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.690+0000 7f4575d9b640 1 --2- 172.21.15.76:0/2077084587 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8eea10 0x559bbf911260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.690+0000 7f4576d9d640 1 -- 172.21.15.76:0/2077084587 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8eea10 msgr2=0x559bbf911260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:04.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.690+0000 7f4576d9d640 1 --2- 172.21.15.76:0/2077084587 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8eea10 0x559bbf911260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.690+0000 7f4576d9d640 1 -- 172.21.15.76:0/2077084587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8e6c70 msgr2=0x559bbf8e8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:04.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.690+0000 7f4576d9d640 1 --2- 172.21.15.76:0/2077084587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8e6c70 0x559bbf8e8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.690+0000 7f4576d9d640 1 -- 172.21.15.76:0/2077084587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bbf8e7df0 con 0x559bbf8f2630 2024-03-08T06:02:04.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.691+0000 7f4576d9d640 1 --2- 172.21.15.76:0/2077084587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8f2630 0x559bbf8f5c60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4570036040 tx=0x7f457002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4003f93289ac285c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:04.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.691+0000 7f45577fe640 1 -- 172.21.15.76:0/2077084587 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457002ca70 con 0x559bbf8f2630 2024-03-08T06:02:04.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.691+0000 7f45577fe640 1 -- 172.21.15.76:0/2077084587 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457002cc10 con 0x559bbf8f2630 2024-03-08T06:02:04.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.691+0000 7f45577fe640 1 -- 172.21.15.76:0/2077084587 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45700496a0 con 0x559bbf8f2630 2024-03-08T06:02:04.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.691+0000 7f457f5b3740 1 -- 172.21.15.76:0/2077084587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8f2630 msgr2=0x559bbf8f5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:04.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.691+0000 7f457f5b3740 1 --2- 172.21.15.76:0/2077084587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8f2630 0x559bbf8f5c60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4570036040 tx=0x7f457002e000 comp rx=0 tx=0).stop 2024-03-08T06:02:04.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.691+0000 7f457f5b3740 1 -- 172.21.15.76:0/2077084587 shutdown_connections 2024-03-08T06:02:04.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.691+0000 7f457f5b3740 1 --2- 172.21.15.76:0/2077084587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8f2630 0x559bbf8f5c60 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.691+0000 7f457f5b3740 1 --2- 172.21.15.76:0/2077084587 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8eea10 0x559bbf911260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.691+0000 7f457f5b3740 1 --2- 172.21.15.76:0/2077084587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8e6c70 0x559bbf8e8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.691+0000 7f457f5b3740 1 -- 172.21.15.76:0/2077084587 >> 172.21.15.76:0/2077084587 conn(0x559bbf8c28a0 msgr2=0x559bbf90a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:04.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.691+0000 7f457f5b3740 1 -- 172.21.15.76:0/2077084587 shutdown_connections 2024-03-08T06:02:04.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.691+0000 7f457f5b3740 1 -- 172.21.15.76:0/2077084587 wait complete. 2024-03-08T06:02:04.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.692+0000 7f457f5b3740 1 -- start start 2024-03-08T06:02:04.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.692+0000 7f457f5b3740 1 Processor -- start 2024-03-08T06:02:04.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f457f5b3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8e6c70 0x559bbf977a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f457f5b3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8eea10 0x559bbf8c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f457f5b3740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8f2630 0x559bbf8c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f457f5b3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbf8e5510 con 0x559bbf8eea10 2024-03-08T06:02:04.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f457f5b3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45780c8890 con 0x559bbf8e6c70 2024-03-08T06:02:04.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f457f5b3740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x559bbf85ffe0 con 0x559bbf8f2630 2024-03-08T06:02:04.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f457f5b3740 10 client.0 _ll_register_callbacks cb 0x7f456800b200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:02:04.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f4575d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8eea10 0x559bbf8c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f4575d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8eea10 0x559bbf8c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60714/0 (socket says 172.21.15.76:60714) 2024-03-08T06:02:04.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f4575d9b640 1 -- 172.21.15.76:0/2785791536 learned_addr learned my addr 172.21.15.76:0/2785791536 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:04.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f457659c640 1 --2- 172.21.15.76:0/2785791536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8e6c70 0x559bbf977a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f4576d9d640 1 --2- 172.21.15.76:0/2785791536 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8f2630 0x559bbf8c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f4575d9b640 1 -- 172.21.15.76:0/2785791536 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8f2630 msgr2=0x559bbf8c74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:04.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f4575d9b640 1 --2- 172.21.15.76:0/2785791536 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8f2630 0x559bbf8c74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f4575d9b640 1 -- 172.21.15.76:0/2785791536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8e6c70 msgr2=0x559bbf977a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:04.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f4575d9b640 1 --2- 172.21.15.76:0/2785791536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8e6c70 0x559bbf977a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f4575d9b640 1 -- 172.21.15.76:0/2785791536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bbf8f1550 con 0x559bbf8eea10 2024-03-08T06:02:04.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f4575d9b640 1 --2- 172.21.15.76:0/2785791536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8eea10 0x559bbf8c6f90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f456c060870 tx=0x7f456c01d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd70674dc7777c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:04.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f45557fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-08T06:02:04.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f45557fa640 1 -- 172.21.15.76:0/2785791536 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c075060 con 0x559bbf8eea10 2024-03-08T06:02:04.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f45557fa640 1 -- 172.21.15.76:0/2785791536 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f456c057ae0 con 0x559bbf8eea10 2024-03-08T06:02:04.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.693+0000 7f457f5b3740 1 -- 172.21.15.76:0/2785791536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f457002cc10 con 0x559bbf8eea10 2024-03-08T06:02:04.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.694+0000 7f45557fa640 1 -- 172.21.15.76:0/2785791536 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f456c07c050 con 0x559bbf8eea10 2024-03-08T06:02:04.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.694+0000 7f457f5b3740 10 client.5568 fetch_fsmap learned FSMap version 15 2024-03-08T06:02:04.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.694+0000 7f457f5b3740 1 -- 172.21.15.76:0/2785791536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x559bbf8f1210 con 0x559bbf8eea10 2024-03-08T06:02:04.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.694+0000 7f45557fa640 1 -- 172.21.15.76:0/2785791536 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457002cc10 con 0x559bbf8eea10 2024-03-08T06:02:04.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.694+0000 7f45557fa640 1 -- 172.21.15.76:0/2785791536 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f456c012050 con 0x559bbf8eea10 2024-03-08T06:02:04.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.694+0000 7f457f5b3740 10 client.5568 fetch_fsmap finished waiting for FSMap version 15 2024-03-08T06:02:04.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.694+0000 7f457f5b3740 10 client.5568 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-03-08T06:02:04.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.694+0000 7f457f5b3740 20 client.5568 populate_metadata read hostname 'smithi076' 2024-03-08T06:02:04.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.694+0000 7f457f5b3740 1 --2- 172.21.15.76:0/2785791536 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x559bbf8e7080 0x559bbf97b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:04.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.694+0000 7f457f5b3740 4 client.5568 mds_command: new command op to 5358 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-08T06:02:04.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.694+0000 7f457f5b3740 1 -- 172.21.15.76:0/2785791536 --> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x559bbf8346c0 con 0x559bbf8e7080 2024-03-08T06:02:04.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.694+0000 7f457659c640 1 --2- 172.21.15.76:0/2785791536 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x559bbf8e7080 0x559bbf97b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:04.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.694+0000 7f45557fa640 10 client.5568 ms_handle_connect on v2:172.21.15.81:6832/1503895376 2024-03-08T06:02:04.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.694+0000 7f457659c640 1 --2- 172.21.15.76:0/2785791536 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x559bbf8e7080 0x559bbf97b6c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb7556f7bb9301a server_cookie=60df72895834e236 in_seq=0 out_seq=0 2024-03-08T06:02:04.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.695+0000 7f45557fa640 1 -- 172.21.15.76:0/2785791536 <== mds.0 v2:172.21.15.81:6832/1503895376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26751 (crc 0 0 0) 0x559bbf8346c0 con 0x559bbf8e7080 2024-03-08T06:02:04.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.695+0000 7f45557fa640 10 client.5568 handle_command_reply: tid=0 2024-03-08T06:02:04.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.695+0000 7f457f5b3740 1 client.5568 shutdown 2024-03-08T06:02:04.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.695+0000 7f457f5b3740 1 -- 172.21.15.76:0/2785791536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8eea10 msgr2=0x559bbf8c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:04.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.695+0000 7f457f5b3740 1 --2- 172.21.15.76:0/2785791536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8eea10 0x559bbf8c6f90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f456c060870 tx=0x7f456c01d070 comp rx=0 tx=0).stop 2024-03-08T06:02:04.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.695+0000 7f457f5b3740 1 -- 172.21.15.76:0/2785791536 shutdown_connections 2024-03-08T06:02:04.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.695+0000 7f457f5b3740 1 --2- 172.21.15.76:0/2785791536 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x559bbf8e7080 0x559bbf97b6c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.695+0000 7f457f5b3740 1 --2- 172.21.15.76:0/2785791536 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x559bbf8f2630 0x559bbf8c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.695+0000 7f457f5b3740 1 --2- 172.21.15.76:0/2785791536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559bbf8eea10 0x559bbf8c6f90 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.695+0000 7f457f5b3740 1 --2- 172.21.15.76:0/2785791536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bbf8e6c70 0x559bbf977a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.695+0000 7f45557fa640 0 client.5568 ms_handle_reset on v2:172.21.15.81:6832/1503895376 2024-03-08T06:02:04.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.695+0000 7f457f5b3740 1 -- 172.21.15.76:0/2785791536 >> 172.21.15.76:0/2785791536 conn(0x559bbf8c28a0 msgr2=0x559bbf8e81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:04.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.695+0000 7f457f5b3740 1 -- 172.21.15.76:0/2785791536 shutdown_connections 2024-03-08T06:02:04.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.695+0000 7f457f5b3740 1 -- 172.21.15.76:0/2785791536 wait complete. 2024-03-08T06:02:04.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.696+0000 7f457f5b3740 20 client.5568 trim_cache size 0 max 16384 2024-03-08T06:02:04.703 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-08T06:02:04.703 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker": [ 2024-03-08T06:02:04.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.703 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": { 2024-03-08T06:02:04.703 INFO:teuthology.orchestra.run.smithi076.stdout: "id": "0" 2024-03-08T06:02:04.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.703 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.703 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T06:02:04.703 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T06:02:04.703 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.704 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": { 2024-03-08T06:02:04.704 INFO:teuthology.orchestra.run.smithi076.stdout: "id": "1" 2024-03-08T06:02:04.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.704 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.704 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T06:02:04.704 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T06:02:04.704 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.704 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": { 2024-03-08T06:02:04.704 INFO:teuthology.orchestra.run.smithi076.stdout: "id": "2" 2024-03-08T06:02:04.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.704 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.704 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T06:02:04.704 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T06:02:04.705 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.705 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.705 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.705 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": [ 2024-03-08T06:02:04.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.705 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.705 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.705 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 37, 2024-03-08T06:02:04.705 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 29, 2024-03-08T06:02:04.705 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 11356, 2024-03-08T06:02:04.705 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 20104, 2024-03-08T06:02:04.705 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 8, 2024-03-08T06:02:04.705 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 3, 2024-03-08T06:02:04.705 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.009048935, 2024-03-08T06:02:04.705 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.001017636, 2024-03-08T06:02:04.706 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.005633043, 2024-03-08T06:02:04.706 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.001129594, 2024-03-08T06:02:04.706 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T06:02:04.706 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 27, 2024-03-08T06:02:04.706 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.128522068, 2024-03-08T06:02:04.706 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.004760076 2024-03-08T06:02:04.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.706 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-08T06:02:04.706 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 17, 2024-03-08T06:02:04.706 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000003157, 2024-03-08T06:02:04.706 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000185 2024-03-08T06:02:04.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.706 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-08T06:02:04.706 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 0 2024-03-08T06:02:04.706 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.707 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.707 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.707 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": [ 2024-03-08T06:02:04.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.707 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.707 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.707 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 73, 2024-03-08T06:02:04.707 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 66, 2024-03-08T06:02:04.707 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 116476, 2024-03-08T06:02:04.707 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 36513, 2024-03-08T06:02:04.707 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 7, 2024-03-08T06:02:04.707 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 5, 2024-03-08T06:02:04.707 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.012178772, 2024-03-08T06:02:04.707 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.001578921, 2024-03-08T06:02:04.707 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.015482461, 2024-03-08T06:02:04.708 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000452295, 2024-03-08T06:02:04.708 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T06:02:04.708 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 66, 2024-03-08T06:02:04.708 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.186144071, 2024-03-08T06:02:04.708 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002820364 2024-03-08T06:02:04.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.708 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-08T06:02:04.708 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2, 2024-03-08T06:02:04.708 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000148, 2024-03-08T06:02:04.708 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000074 2024-03-08T06:02:04.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.708 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 113472, 2024-03-08T06:02:04.708 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 12688 2024-03-08T06:02:04.708 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.708 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.708 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.709 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": [ 2024-03-08T06:02:04.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.709 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.709 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.709 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 46, 2024-03-08T06:02:04.709 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 42, 2024-03-08T06:02:04.709 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 13296, 2024-03-08T06:02:04.709 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 127931, 2024-03-08T06:02:04.709 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 8, 2024-03-08T06:02:04.709 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 4, 2024-03-08T06:02:04.709 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.011586672, 2024-03-08T06:02:04.709 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.001440443, 2024-03-08T06:02:04.709 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.008872378, 2024-03-08T06:02:04.709 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000877110, 2024-03-08T06:02:04.709 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T06:02:04.710 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 41, 2024-03-08T06:02:04.710 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.061260517, 2024-03-08T06:02:04.710 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001494158 2024-03-08T06:02:04.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.710 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-08T06:02:04.710 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 59, 2024-03-08T06:02:04.710 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000008042, 2024-03-08T06:02:04.710 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000136 2024-03-08T06:02:04.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.710 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-08T06:02:04.710 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 92496 2024-03-08T06:02:04.710 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.710 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.711 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.711 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": [ 2024-03-08T06:02:04.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.711 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.711 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.711 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-03-08T06:02:04.711 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-03-08T06:02:04.711 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.711 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.711 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.711 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": [ 2024-03-08T06:02:04.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.711 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.711 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.712 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-03-08T06:02:04.712 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-03-08T06:02:04.712 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 48, 2024-03-08T06:02:04.712 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.002436695, 2024-03-08T06:02:04.712 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000050764 2024-03-08T06:02:04.712 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.712 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.712 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.712 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.712 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": [ 2024-03-08T06:02:04.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.712 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.712 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.713 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-03-08T06:02:04.713 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-03-08T06:02:04.713 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2, 2024-03-08T06:02:04.713 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000011902, 2024-03-08T06:02:04.713 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000005951 2024-03-08T06:02:04.713 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.713 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.713 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.713 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.713 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-03-08T06:02:04.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.713 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.713 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.713 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 25, 2024-03-08T06:02:04.714 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 25, 2024-03-08T06:02:04.714 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-03-08T06:02:04.714 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 25, 2024-03-08T06:02:04.714 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.005572633, 2024-03-08T06:02:04.714 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000222905 2024-03-08T06:02:04.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.714 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-03-08T06:02:04.714 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 0, 2024-03-08T06:02:04.714 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 0, 2024-03-08T06:02:04.714 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 0, 2024-03-08T06:02:04.714 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 12, 2024-03-08T06:02:04.714 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 0, 2024-03-08T06:02:04.714 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 0, 2024-03-08T06:02:04.714 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 10, 2024-03-08T06:02:04.715 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 0, 2024-03-08T06:02:04.715 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 10, 2024-03-08T06:02:04.715 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 0, 2024-03-08T06:02:04.715 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 10, 2024-03-08T06:02:04.715 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 0, 2024-03-08T06:02:04.715 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 1, 2024-03-08T06:02:04.715 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 2, 2024-03-08T06:02:04.715 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 2, 2024-03-08T06:02:04.715 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 49, 2024-03-08T06:02:04.715 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 49, 2024-03-08T06:02:04.715 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 0, 2024-03-08T06:02:04.715 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 0, 2024-03-08T06:02:04.715 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 0, 2024-03-08T06:02:04.715 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-03-08T06:02:04.716 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 0, 2024-03-08T06:02:04.716 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 0, 2024-03-08T06:02:04.716 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 0, 2024-03-08T06:02:04.716 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 0, 2024-03-08T06:02:04.716 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 0, 2024-03-08T06:02:04.716 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 0, 2024-03-08T06:02:04.716 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 0, 2024-03-08T06:02:04.716 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-03-08T06:02:04.716 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 0, 2024-03-08T06:02:04.716 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-03-08T06:02:04.716 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 0, 2024-03-08T06:02:04.716 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 0, 2024-03-08T06:02:04.716 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 0, 2024-03-08T06:02:04.716 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 0, 2024-03-08T06:02:04.716 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-03-08T06:02:04.717 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-03-08T06:02:04.717 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-03-08T06:02:04.717 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 0, 2024-03-08T06:02:04.717 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 0, 2024-03-08T06:02:04.717 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 0, 2024-03-08T06:02:04.717 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 0, 2024-03-08T06:02:04.717 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 0, 2024-03-08T06:02:04.717 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 0, 2024-03-08T06:02:04.717 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 0, 2024-03-08T06:02:04.717 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 0, 2024-03-08T06:02:04.717 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-08T06:02:04.717 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-08T06:02:04.717 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 0, 2024-03-08T06:02:04.717 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 0, 2024-03-08T06:02:04.717 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 0, 2024-03-08T06:02:04.718 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 1 2024-03-08T06:02:04.718 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.718 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.718 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.718 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": [ 2024-03-08T06:02:04.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.718 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.718 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.718 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 0, 2024-03-08T06:02:04.718 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 0, 2024-03-08T06:02:04.718 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 0, 2024-03-08T06:02:04.718 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 0, 2024-03-08T06:02:04.718 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 0, 2024-03-08T06:02:04.718 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 0, 2024-03-08T06:02:04.719 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 0, 2024-03-08T06:02:04.719 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-03-08T06:02:04.719 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 0, 2024-03-08T06:02:04.719 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 0, 2024-03-08T06:02:04.719 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-03-08T06:02:04.719 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-03-08T06:02:04.719 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-03-08T06:02:04.719 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-03-08T06:02:04.719 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-03-08T06:02:04.719 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-03-08T06:02:04.719 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-03-08T06:02:04.719 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 0, 2024-03-08T06:02:04.719 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 0, 2024-03-08T06:02:04.719 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-03-08T06:02:04.719 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 0, 2024-03-08T06:02:04.720 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-03-08T06:02:04.720 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-03-08T06:02:04.720 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.720 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.720 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.720 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_client_metrics": [ 2024-03-08T06:02:04.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.720 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": { 2024-03-08T06:02:04.720 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-03-08T06:02:04.720 INFO:teuthology.orchestra.run.smithi076.stdout: "id": "5358" 2024-03-08T06:02:04.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.720 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.720 INFO:teuthology.orchestra.run.smithi076.stdout: "num_clients": 2 2024-03-08T06:02:04.720 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.721 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.721 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_client_metrics-cephfs": [ 2024-03-08T06:02:04.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.721 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": { 2024-03-08T06:02:04.721 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.5541", 2024-03-08T06:02:04.721 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": "0" 2024-03-08T06:02:04.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.721 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.721 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_hits": 5, 2024-03-08T06:02:04.721 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_miss": 18, 2024-03-08T06:02:04.721 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_read_latency": 0.000000000, 2024-03-08T06:02:04.721 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_write_latency": 0.000000000, 2024-03-08T06:02:04.721 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_metadata_latency": 0.000000000, 2024-03-08T06:02:04.721 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_lease_hits": 0, 2024-03-08T06:02:04.722 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_lease_miss": 0, 2024-03-08T06:02:04.722 INFO:teuthology.orchestra.run.smithi076.stdout: "opened_files": 0, 2024-03-08T06:02:04.722 INFO:teuthology.orchestra.run.smithi076.stdout: "opened_inodes": 1, 2024-03-08T06:02:04.722 INFO:teuthology.orchestra.run.smithi076.stdout: "pinned_icaps": 1, 2024-03-08T06:02:04.722 INFO:teuthology.orchestra.run.smithi076.stdout: "total_inodes": 1, 2024-03-08T06:02:04.722 INFO:teuthology.orchestra.run.smithi076.stdout: "total_read_ops": 0, 2024-03-08T06:02:04.722 INFO:teuthology.orchestra.run.smithi076.stdout: "total_read_size": 0, 2024-03-08T06:02:04.722 INFO:teuthology.orchestra.run.smithi076.stdout: "total_write_ops": 0, 2024-03-08T06:02:04.722 INFO:teuthology.orchestra.run.smithi076.stdout: "total_write_size": 0 2024-03-08T06:02:04.722 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.722 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": { 2024-03-08T06:02:04.722 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.5550", 2024-03-08T06:02:04.723 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": "0" 2024-03-08T06:02:04.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.723 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.723 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_hits": 23, 2024-03-08T06:02:04.723 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_miss": 0, 2024-03-08T06:02:04.723 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_read_latency": 0.000000000, 2024-03-08T06:02:04.723 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_write_latency": 0.000000000, 2024-03-08T06:02:04.723 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_metadata_latency": 0.000000000, 2024-03-08T06:02:04.723 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_lease_hits": 0, 2024-03-08T06:02:04.723 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_lease_miss": 0, 2024-03-08T06:02:04.723 INFO:teuthology.orchestra.run.smithi076.stdout: "opened_files": 0, 2024-03-08T06:02:04.723 INFO:teuthology.orchestra.run.smithi076.stdout: "opened_inodes": 1, 2024-03-08T06:02:04.723 INFO:teuthology.orchestra.run.smithi076.stdout: "pinned_icaps": 1, 2024-03-08T06:02:04.723 INFO:teuthology.orchestra.run.smithi076.stdout: "total_inodes": 1, 2024-03-08T06:02:04.723 INFO:teuthology.orchestra.run.smithi076.stdout: "total_read_ops": 0, 2024-03-08T06:02:04.724 INFO:teuthology.orchestra.run.smithi076.stdout: "total_read_size": 0, 2024-03-08T06:02:04.724 INFO:teuthology.orchestra.run.smithi076.stdout: "total_write_ops": 0, 2024-03-08T06:02:04.724 INFO:teuthology.orchestra.run.smithi076.stdout: "total_write_size": 0 2024-03-08T06:02:04.724 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.724 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.724 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.724 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": [ 2024-03-08T06:02:04.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.724 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.724 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.724 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 0, 2024-03-08T06:02:04.724 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 6, 2024-03-08T06:02:04.724 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 0, 2024-03-08T06:02:04.724 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 0, 2024-03-08T06:02:04.724 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 6, 2024-03-08T06:02:04.725 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-03-08T06:02:04.725 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 0, 2024-03-08T06:02:04.725 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 2, 2024-03-08T06:02:04.725 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 0, 2024-03-08T06:02:04.725 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 0, 2024-03-08T06:02:04.725 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 2, 2024-03-08T06:02:04.725 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 2, 2024-03-08T06:02:04.725 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-03-08T06:02:04.725 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 0, 2024-03-08T06:02:04.725 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 4194304, 2024-03-08T06:02:04.725 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 4196743, 2024-03-08T06:02:04.725 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 0, 2024-03-08T06:02:04.725 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-03-08T06:02:04.725 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 3, 2024-03-08T06:02:04.726 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.010951892, 2024-03-08T06:02:04.726 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003650630 2024-03-08T06:02:04.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.726 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 0 2024-03-08T06:02:04.726 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.726 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.726 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.726 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": [ 2024-03-08T06:02:04.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.726 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.726 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.726 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 13, 2024-03-08T06:02:04.726 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 13, 2024-03-08T06:02:04.726 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 0, 2024-03-08T06:02:04.726 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 12, 2024-03-08T06:02:04.727 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 12, 2024-03-08T06:02:04.727 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 0, 2024-03-08T06:02:04.727 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 10, 2024-03-08T06:02:04.727 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 10, 2024-03-08T06:02:04.727 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 0, 2024-03-08T06:02:04.727 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 2, 2024-03-08T06:02:04.727 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 2, 2024-03-08T06:02:04.727 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 0, 2024-03-08T06:02:04.727 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 42840, 2024-03-08T06:02:04.727 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 215392 2024-03-08T06:02:04.727 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.727 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.727 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.727 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": [ 2024-03-08T06:02:04.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.728 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.728 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.728 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 25, 2024-03-08T06:02:04.728 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 0, 2024-03-08T06:02:04.728 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 25, 2024-03-08T06:02:04.728 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 2, 2024-03-08T06:02:04.728 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 0, 2024-03-08T06:02:04.728 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-03-08T06:02:04.728 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.728 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.728 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.728 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-03-08T06:02:04.728 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 24, 2024-03-08T06:02:04.728 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.005026633, 2024-03-08T06:02:04.729 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000209443 2024-03-08T06:02:04.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.729 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-03-08T06:02:04.729 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.729 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.729 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.729 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-03-08T06:02:04.729 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.729 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.729 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.729 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-03-08T06:02:04.729 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.729 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.729 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.730 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-03-08T06:02:04.730 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.730 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.730 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.730 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-03-08T06:02:04.730 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.730 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.730 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.730 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-03-08T06:02:04.730 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.730 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.731 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.731 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-03-08T06:02:04.731 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.731 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.731 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.731 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-03-08T06:02:04.731 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.731 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.731 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.731 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-03-08T06:02:04.731 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.731 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.732 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.732 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-03-08T06:02:04.732 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.732 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.732 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.732 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-03-08T06:02:04.732 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.732 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.732 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.732 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-03-08T06:02:04.732 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.732 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.733 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.733 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-03-08T06:02:04.733 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.733 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.733 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.733 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-03-08T06:02:04.733 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.733 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.733 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.733 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-03-08T06:02:04.733 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.733 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.734 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.734 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-03-08T06:02:04.734 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.734 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.734 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.734 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-03-08T06:02:04.734 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.734 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.734 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.734 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-03-08T06:02:04.734 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.735 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.735 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.735 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-03-08T06:02:04.735 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.735 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.735 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.735 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-03-08T06:02:04.735 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.735 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.735 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.735 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-03-08T06:02:04.736 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1, 2024-03-08T06:02:04.736 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000546000, 2024-03-08T06:02:04.736 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000546000 2024-03-08T06:02:04.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.736 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-03-08T06:02:04.736 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.736 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.736 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.736 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-03-08T06:02:04.736 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.736 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.736 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.736 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-03-08T06:02:04.737 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.737 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.737 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.737 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-03-08T06:02:04.737 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.737 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.737 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.737 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-03-08T06:02:04.737 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.737 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.737 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.737 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-03-08T06:02:04.738 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.738 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.738 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.738 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-03-08T06:02:04.738 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-03-08T06:02:04.738 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-03-08T06:02:04.738 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.738 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.738 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.738 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.738 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.738 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.738 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.739 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": [ 2024-03-08T06:02:04.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.739 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.739 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.739 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 2, 2024-03-08T06:02:04.739 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 2, 2024-03-08T06:02:04.739 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 0, 2024-03-08T06:02:04.739 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 2, 2024-03-08T06:02:04.739 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-03-08T06:02:04.739 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 24, 2024-03-08T06:02:04.739 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 12, 2024-03-08T06:02:04.739 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 7, 2024-03-08T06:02:04.739 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-03-08T06:02:04.739 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.740 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.740 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.740 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": [ 2024-03-08T06:02:04.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.740 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.740 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.740 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-03-08T06:02:04.740 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-03-08T06:02:04.740 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-03-08T06:02:04.740 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-03-08T06:02:04.740 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-03-08T06:02:04.740 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-03-08T06:02:04.740 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-08T06:02:04.740 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-03-08T06:02:04.740 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-08T06:02:04.741 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-03-08T06:02:04.741 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-03-08T06:02:04.741 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-03-08T06:02:04.741 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-08T06:02:04.741 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-03-08T06:02:04.741 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-03-08T06:02:04.741 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-03-08T06:02:04.741 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-03-08T06:02:04.741 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-03-08T06:02:04.741 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-08T06:02:04.741 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-03-08T06:02:04.741 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-08T06:02:04.741 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-03-08T06:02:04.741 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-03-08T06:02:04.741 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-03-08T06:02:04.742 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-03-08T06:02:04.742 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-03-08T06:02:04.742 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-08T06:02:04.742 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-03-08T06:02:04.742 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-03-08T06:02:04.742 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-03-08T06:02:04.742 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-03-08T06:02:04.742 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-03-08T06:02:04.742 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-03-08T06:02:04.742 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-03-08T06:02:04.742 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-03-08T06:02:04.742 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-03-08T06:02:04.742 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 232675, 2024-03-08T06:02:04.742 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 91, 2024-03-08T06:02:04.743 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-03-08T06:02:04.743 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-03-08T06:02:04.743 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-03-08T06:02:04.743 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-03-08T06:02:04.743 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-03-08T06:02:04.743 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-03-08T06:02:04.743 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-03-08T06:02:04.743 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-03-08T06:02:04.743 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 5424, 2024-03-08T06:02:04.743 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 76, 2024-03-08T06:02:04.743 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-03-08T06:02:04.743 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-03-08T06:02:04.743 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-03-08T06:02:04.743 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-03-08T06:02:04.743 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 52240, 2024-03-08T06:02:04.744 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 213, 2024-03-08T06:02:04.744 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-03-08T06:02:04.744 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-03-08T06:02:04.744 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-03-08T06:02:04.744 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-03-08T06:02:04.744 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.744 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.744 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.744 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": [ 2024-03-08T06:02:04.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.744 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.744 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.744 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-03-08T06:02:04.744 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-03-08T06:02:04.744 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 41, 2024-03-08T06:02:04.745 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 13559, 2024-03-08T06:02:04.745 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 37, 2024-03-08T06:02:04.745 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 41, 2024-03-08T06:02:04.745 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-03-08T06:02:04.745 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 41, 2024-03-08T06:02:04.745 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.843983057, 2024-03-08T06:02:04.745 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.020584952 2024-03-08T06:02:04.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.745 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-03-08T06:02:04.745 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-03-08T06:02:04.745 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 41, 2024-03-08T06:02:04.745 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 65 2024-03-08T06:02:04.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:04.745 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 41, 2024-03-08T06:02:04.746 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 0, 2024-03-08T06:02:04.746 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 41, 2024-03-08T06:02:04.746 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-03-08T06:02:04.746 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-03-08T06:02:04.746 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 0, 2024-03-08T06:02:04.746 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 11, 2024-03-08T06:02:04.746 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 0, 2024-03-08T06:02:04.746 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 3, 2024-03-08T06:02:04.746 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 7, 2024-03-08T06:02:04.746 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-03-08T06:02:04.746 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-03-08T06:02:04.746 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 0, 2024-03-08T06:02:04.746 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 0, 2024-03-08T06:02:04.746 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 6, 2024-03-08T06:02:04.746 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-03-08T06:02:04.747 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-03-08T06:02:04.747 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-03-08T06:02:04.747 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 0, 2024-03-08T06:02:04.747 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 22, 2024-03-08T06:02:04.747 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-03-08T06:02:04.747 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-03-08T06:02:04.747 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-03-08T06:02:04.747 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-03-08T06:02:04.747 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-03-08T06:02:04.747 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-03-08T06:02:04.747 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-03-08T06:02:04.747 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-03-08T06:02:04.747 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-03-08T06:02:04.747 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 0, 2024-03-08T06:02:04.748 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-03-08T06:02:04.748 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-03-08T06:02:04.748 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-03-08T06:02:04.748 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-03-08T06:02:04.748 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-03-08T06:02:04.748 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-03-08T06:02:04.748 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-03-08T06:02:04.748 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-03-08T06:02:04.748 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-03-08T06:02:04.748 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-03-08T06:02:04.748 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-03-08T06:02:04.748 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-03-08T06:02:04.748 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-03-08T06:02:04.749 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-03-08T06:02:04.749 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-03-08T06:02:04.749 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-03-08T06:02:04.749 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 40, 2024-03-08T06:02:04.749 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-03-08T06:02:04.749 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 1, 2024-03-08T06:02:04.749 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 8, 2024-03-08T06:02:04.749 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 8, 2024-03-08T06:02:04.749 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 0, 2024-03-08T06:02:04.749 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-03-08T06:02:04.749 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 15, 2024-03-08T06:02:04.749 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 0, 2024-03-08T06:02:04.749 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 1 2024-03-08T06:02:04.750 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.750 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.750 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.750 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": [ 2024-03-08T06:02:04.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.750 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.750 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.750 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 1, 2024-03-08T06:02:04.750 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 0, 2024-03-08T06:02:04.750 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 0, 2024-03-08T06:02:04.750 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-03-08T06:02:04.750 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.750 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.751 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.751 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": [ 2024-03-08T06:02:04.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.751 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.751 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.751 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 0 2024-03-08T06:02:04.751 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.751 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.751 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.751 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": [ 2024-03-08T06:02:04.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.751 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.751 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.751 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-03-08T06:02:04.752 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 0, 2024-03-08T06:02:04.752 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-03-08T06:02:04.752 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 0, 2024-03-08T06:02:04.752 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 0, 2024-03-08T06:02:04.752 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 0, 2024-03-08T06:02:04.752 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-03-08T06:02:04.752 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.752 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.752 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.752 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-08T06:02:04.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.752 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.753 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.753 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-03-08T06:02:04.753 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-03-08T06:02:04.753 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-03-08T06:02:04.753 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 151, 2024-03-08T06:02:04.753 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 123138, 2024-03-08T06:02:04.753 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-03-08T06:02:04.753 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 151, 2024-03-08T06:02:04.753 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-03-08T06:02:04.753 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-03-08T06:02:04.753 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 151, 2024-03-08T06:02:04.753 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 123138, 2024-03-08T06:02:04.753 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-03-08T06:02:04.754 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.754 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.754 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.754 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.754 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.754 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.754 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.754 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": [ 2024-03-08T06:02:04.754 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.754 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.754 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.754 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-03-08T06:02:04.754 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-03-08T06:02:04.754 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-03-08T06:02:04.755 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-03-08T06:02:04.755 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-03-08T06:02:04.755 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-03-08T06:02:04.755 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-03-08T06:02:04.755 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 41, 2024-03-08T06:02:04.755 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 13559, 2024-03-08T06:02:04.755 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 41, 2024-03-08T06:02:04.755 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 13559, 2024-03-08T06:02:04.755 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-03-08T06:02:04.755 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.755 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.755 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.756 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.756 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.756 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.756 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.756 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": [ 2024-03-08T06:02:04.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.756 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.756 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.756 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-03-08T06:02:04.756 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-03-08T06:02:04.756 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-03-08T06:02:04.756 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-03-08T06:02:04.756 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-03-08T06:02:04.756 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-03-08T06:02:04.757 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-03-08T06:02:04.757 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 41, 2024-03-08T06:02:04.757 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 41, 2024-03-08T06:02:04.757 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 41, 2024-03-08T06:02:04.757 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 41, 2024-03-08T06:02:04.757 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-03-08T06:02:04.757 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.757 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.757 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.757 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.757 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.757 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.757 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.757 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": [ 2024-03-08T06:02:04.757 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.758 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.758 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.758 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-03-08T06:02:04.758 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-03-08T06:02:04.758 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-03-08T06:02:04.758 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-03-08T06:02:04.758 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-03-08T06:02:04.758 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-03-08T06:02:04.758 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-03-08T06:02:04.758 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-03-08T06:02:04.758 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-03-08T06:02:04.758 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 0, 2024-03-08T06:02:04.758 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 0, 2024-03-08T06:02:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-03-08T06:02:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x55c65ff9a6e0": [ 2024-03-08T06:02:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-03-08T06:02:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-03-08T06:02:04.760 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-03-08T06:02:04.760 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 6, 2024-03-08T06:02:04.760 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 2439, 2024-03-08T06:02:04.760 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-03-08T06:02:04.760 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 6, 2024-03-08T06:02:04.760 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-03-08T06:02:04.760 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-03-08T06:02:04.760 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 3, 2024-03-08T06:02:04.760 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 2439, 2024-03-08T06:02:04.760 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-03-08T06:02:04.760 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:04.760 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:04.760 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:04.760 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.761 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.761 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:04.761 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-03-08T06:02:04.761 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-08T06:02:04.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.696+0000 7f457c9d7640 1 -- 172.21.15.76:0/3575311940 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f455806b4d0 msgr2=0x7f455806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:04.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.696+0000 7f457c9d7640 1 --2- 172.21.15.76:0/3575311940 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f455806b4d0 0x7f455806d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4568006fd0 tx=0x7f4568008040 comp rx=0 tx=0).stop 2024-03-08T06:02:04.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.696+0000 7f457c9d7640 1 -- 172.21.15.76:0/3575311940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4578071750 msgr2=0x7f4578142180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:04.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.696+0000 7f457c9d7640 1 --2- 172.21.15.76:0/3575311940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4578071750 0x7f4578142180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f456c00e810 tx=0x7f456c00ecd0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.696+0000 7f457c9d7640 1 -- 172.21.15.76:0/3575311940 shutdown_connections 2024-03-08T06:02:04.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.696+0000 7f457c9d7640 1 --2- 172.21.15.76:0/3575311940 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f455806b4d0 0x7f455806d990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.696+0000 7f457c9d7640 1 --2- 172.21.15.76:0/3575311940 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4578156070 0x7f4578142930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.696+0000 7f457c9d7640 1 --2- 172.21.15.76:0/3575311940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4578071750 0x7f4578142180 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.696+0000 7f457c9d7640 1 --2- 172.21.15.76:0/3575311940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f457806fb60 0x7f4578144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:04.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.696+0000 7f457c9d7640 1 -- 172.21.15.76:0/3575311940 >> 172.21.15.76:0/3575311940 conn(0x7f45780cc600 msgr2=0x7f45780cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:04.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.696+0000 7f457c9d7640 1 -- 172.21.15.76:0/3575311940 shutdown_connections 2024-03-08T06:02:04.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:04.696+0000 7f457c9d7640 1 -- 172.21.15.76:0/3575311940 wait complete. 2024-03-08T06:02:04.764 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:02:04.764 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:02:04.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:04.779 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:02:04.835 INFO:teuthology.orchestra.run.smithi076.stdout:client5541 2024-03-08T06:02:04.835 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:02:04.836 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:02:04.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:04.854 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:02:04.913 INFO:teuthology.orchestra.run.smithi081.stdout:client5550 2024-03-08T06:02:04.914 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:02:04.914 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:02:04.929 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-03-08T06:02:04.929 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-03-08T06:02:04.929 DEBUG:teuthology.orchestra.run.smithi076:> import uuid 2024-03-08T06:02:04.929 DEBUG:teuthology.orchestra.run.smithi076:> 2024-03-08T06:02:04.929 DEBUG:teuthology.orchestra.run.smithi076:> n = 1000 2024-03-08T06:02:04.929 DEBUG:teuthology.orchestra.run.smithi076:> create_hard_links = False 2024-03-08T06:02:04.929 DEBUG:teuthology.orchestra.run.smithi076:> if 0 > 0: 2024-03-08T06:02:04.930 DEBUG:teuthology.orchestra.run.smithi076:> create_hard_links = True 2024-03-08T06:02:04.930 DEBUG:teuthology.orchestra.run.smithi076:> path = "/home/ubuntu/cephtest/mnt.0/test_dir/test_file" 2024-03-08T06:02:04.930 DEBUG:teuthology.orchestra.run.smithi076:> 2024-03-08T06:02:04.930 DEBUG:teuthology.orchestra.run.smithi076:> dpath = os.path.dirname(path) 2024-03-08T06:02:04.930 DEBUG:teuthology.orchestra.run.smithi076:> fnameprefix = os.path.basename(path) 2024-03-08T06:02:04.930 DEBUG:teuthology.orchestra.run.smithi076:> os.makedirs(dpath, exist_ok=True) 2024-03-08T06:02:04.930 DEBUG:teuthology.orchestra.run.smithi076:> 2024-03-08T06:02:04.930 DEBUG:teuthology.orchestra.run.smithi076:> try: 2024-03-08T06:02:04.930 DEBUG:teuthology.orchestra.run.smithi076:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-03-08T06:02:04.930 DEBUG:teuthology.orchestra.run.smithi076:> 2024-03-08T06:02:04.930 DEBUG:teuthology.orchestra.run.smithi076:> for i in range(n): 2024-03-08T06:02:04.930 DEBUG:teuthology.orchestra.run.smithi076:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-03-08T06:02:04.930 DEBUG:teuthology.orchestra.run.smithi076:> with open(fpath, '"'"'w'"'"') as f: 2024-03-08T06:02:04.930 DEBUG:teuthology.orchestra.run.smithi076:> f.write(f"{i}") 2024-03-08T06:02:04.931 DEBUG:teuthology.orchestra.run.smithi076:> if True: 2024-03-08T06:02:04.931 DEBUG:teuthology.orchestra.run.smithi076:> f.flush() 2024-03-08T06:02:04.931 DEBUG:teuthology.orchestra.run.smithi076:> os.fsync(f.fileno()) 2024-03-08T06:02:04.931 DEBUG:teuthology.orchestra.run.smithi076:> if False: 2024-03-08T06:02:04.931 DEBUG:teuthology.orchestra.run.smithi076:> os.unlink(fpath) 2024-03-08T06:02:04.931 DEBUG:teuthology.orchestra.run.smithi076:> if False: 2024-03-08T06:02:04.931 DEBUG:teuthology.orchestra.run.smithi076:> os.fsync(dirfd) 2024-03-08T06:02:04.931 DEBUG:teuthology.orchestra.run.smithi076:> if create_hard_links: 2024-03-08T06:02:04.931 DEBUG:teuthology.orchestra.run.smithi076:> for j in range(0): 2024-03-08T06:02:04.931 DEBUG:teuthology.orchestra.run.smithi076:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-03-08T06:02:04.931 DEBUG:teuthology.orchestra.run.smithi076:> if False: 2024-03-08T06:02:04.931 DEBUG:teuthology.orchestra.run.smithi076:> os.fsync(dirfd) 2024-03-08T06:02:04.931 DEBUG:teuthology.orchestra.run.smithi076:> finally: 2024-03-08T06:02:04.931 DEBUG:teuthology.orchestra.run.smithi076:> os.close(dirfd) 2024-03-08T06:02:04.932 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-03-08T06:02:05.817 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:02:05.820 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:02:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T06:02:08.521 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T06:02:08.521 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 counter dump 2024-03-08T06:02:08.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.624+0000 7fc8c7d3a640 1 Processor -- start 2024-03-08T06:02:08.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.624+0000 7fc8c7d3a640 1 -- start start 2024-03-08T06:02:08.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.625+0000 7fc8c7d3a640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc8c00aee10 0x7fc8c015dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.625+0000 7fc8c7d3a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc8c00b2e20 0x7fc8c015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.625+0000 7fc8c7d3a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c015b820 con 0x7fc8c015b260 2024-03-08T06:02:08.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.625+0000 7fc8c7d3a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c015e490 con 0x7fc8c00b2e20 2024-03-08T06:02:08.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.625+0000 7fc8c7d3a640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c015e770 con 0x7fc8c00aee10 2024-03-08T06:02:08.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.625+0000 7fc8c5ad6640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc8c00aee10 0x7fc8c015dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.625+0000 7fc8c62d7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc8c00b2e20 0x7fc8c015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.625+0000 7fc8c52d5640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc8c015b260 0x7fc8c00b2710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:56804/0 (socket says 172.21.15.76:56804) 2024-03-08T06:02:08.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.625+0000 7fc8c52d5640 1 -- 172.21.15.76:0/3389842118 learned_addr learned my addr 172.21.15.76:0/3389842118 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:08.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.625+0000 7fc8c62d7640 1 -- 172.21.15.76:0/3389842118 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc8c00aee10 msgr2=0x7fc8c015dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:08.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.625+0000 7fc8c62d7640 1 --2- 172.21.15.76:0/3389842118 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc8c00aee10 0x7fc8c015dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.625+0000 7fc8c62d7640 1 -- 172.21.15.76:0/3389842118 >> v1:172.21.15.81:6789/0 conn(0x7fc8c015b260 legacy=0x7fc8c00b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:08.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.625+0000 7fc8c62d7640 1 -- 172.21.15.76:0/3389842118 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c015ba90 con 0x7fc8c00b2e20 2024-03-08T06:02:08.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.625+0000 7fc8c5ad6640 1 --2- 172.21.15.76:0/3389842118 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc8c00aee10 0x7fc8c015dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:02:08.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.625+0000 7fc8c4ad4640 1 -- 172.21.15.76:0/3389842118 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914266749 0 0) 0x7fc8c015b820 con 0x7fc8c015b260 2024-03-08T06:02:08.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.626+0000 7fc8c62d7640 1 --2- 172.21.15.76:0/3389842118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8c00b2e20 0x7fc8c015bfb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009d90 tx=0x7fc8b802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c97fba521b4730e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:08.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.626+0000 7fc8c4ad4640 1 -- 172.21.15.76:0/3389842118 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8039070 con 0x7fc8c00b2e20 2024-03-08T06:02:08.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.626+0000 7fc8c4ad4640 1 -- 172.21.15.76:0/3389842118 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b802cbc0 con 0x7fc8c00b2e20 2024-03-08T06:02:08.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.626+0000 7fc8c4ad4640 1 -- 172.21.15.76:0/3389842118 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8034440 con 0x7fc8c00b2e20 2024-03-08T06:02:08.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.627+0000 7fc8c7d3a640 1 -- 172.21.15.76:0/3389842118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8c00b2e20 msgr2=0x7fc8c015bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:08.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.627+0000 7fc8c7d3a640 1 --2- 172.21.15.76:0/3389842118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8c00b2e20 0x7fc8c015bfb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009d90 tx=0x7fc8b802be10 comp rx=0 tx=0).stop 2024-03-08T06:02:08.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.627+0000 7fc8c7d3a640 1 -- 172.21.15.76:0/3389842118 shutdown_connections 2024-03-08T06:02:08.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.627+0000 7fc8c7d3a640 1 --2- 172.21.15.76:0/3389842118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8c00b2e20 0x7fc8c015bfb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.627+0000 7fc8c7d3a640 1 --2- 172.21.15.76:0/3389842118 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc8c00aee10 0x7fc8c015dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.627+0000 7fc8c7d3a640 1 -- 172.21.15.76:0/3389842118 >> 172.21.15.76:0/3389842118 conn(0x7fc8c006ac80 msgr2=0x7fc8c006b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:08.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.627+0000 7fc8c7d3a640 1 -- 172.21.15.76:0/3389842118 shutdown_connections 2024-03-08T06:02:08.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.627+0000 7fc8c7d3a640 1 -- 172.21.15.76:0/3389842118 wait complete. 2024-03-08T06:02:08.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.628+0000 7fc8c7d3a640 1 Processor -- start 2024-03-08T06:02:08.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.628+0000 7fc8c7d3a640 1 -- start start 2024-03-08T06:02:08.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.629+0000 7fc8c7d3a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8c00aee10 0x7fc8c01658c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.629+0000 7fc8c7d3a640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc8c00b2e20 0x7fc8c013c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.629+0000 7fc8c7d3a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8c015b260 0x7fc8c0132af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.629+0000 7fc8c7d3a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c015c4f0 con 0x7fc8c015b260 2024-03-08T06:02:08.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.629+0000 7fc8c7d3a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c015ea50 con 0x7fc8c00aee10 2024-03-08T06:02:08.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.629+0000 7fc8c7d3a640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c015c670 con 0x7fc8c00b2e20 2024-03-08T06:02:08.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.629+0000 7fc8c62d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8c015b260 0x7fc8c0132af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.629+0000 7fc8c62d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8c015b260 0x7fc8c0132af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51048/0 (socket says 172.21.15.76:51048) 2024-03-08T06:02:08.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.629+0000 7fc8c62d7640 1 -- 172.21.15.76:0/966421975 learned_addr learned my addr 172.21.15.76:0/966421975 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:08.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.629+0000 7fc8c52d5640 1 --2- 172.21.15.76:0/966421975 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc8c00b2e20 0x7fc8c013c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.629+0000 7fc8c5ad6640 1 --2- 172.21.15.76:0/966421975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8c00aee10 0x7fc8c01658c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.629+0000 7fc8c62d7640 1 -- 172.21.15.76:0/966421975 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc8c00b2e20 msgr2=0x7fc8c013c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:08.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.629+0000 7fc8c62d7640 1 --2- 172.21.15.76:0/966421975 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc8c00b2e20 0x7fc8c013c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.629+0000 7fc8c62d7640 1 -- 172.21.15.76:0/966421975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8c00aee10 msgr2=0x7fc8c01658c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:08.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.629+0000 7fc8c62d7640 1 --2- 172.21.15.76:0/966421975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8c00aee10 0x7fc8c01658c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.629+0000 7fc8c62d7640 1 -- 172.21.15.76:0/966421975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c013ce30 con 0x7fc8c015b260 2024-03-08T06:02:08.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.630+0000 7fc8c62d7640 1 --2- 172.21.15.76:0/966421975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8c015b260 0x7fc8c0132af0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009d90 tx=0x7fc8b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:08.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.630+0000 7fc8b67fc640 1 -- 172.21.15.76:0/966421975 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8044020 con 0x7fc8c015b260 2024-03-08T06:02:08.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.630+0000 7fc8b67fc640 1 -- 172.21.15.76:0/966421975 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b803a040 con 0x7fc8c015b260 2024-03-08T06:02:08.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.630+0000 7fc8c7d3a640 1 -- 172.21.15.76:0/966421975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c013d0d0 con 0x7fc8c015b260 2024-03-08T06:02:08.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.630+0000 7fc8b67fc640 1 -- 172.21.15.76:0/966421975 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8039050 con 0x7fc8c015b260 2024-03-08T06:02:08.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.630+0000 7fc8c7d3a640 1 -- 172.21.15.76:0/966421975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c013d690 con 0x7fc8c015b260 2024-03-08T06:02:08.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.631+0000 7fc8b67fc640 1 -- 172.21.15.76:0/966421975 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8b8003bb0 con 0x7fc8c015b260 2024-03-08T06:02:08.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.631+0000 7fc8b67fc640 1 --2- 172.21.15.76:0/966421975 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc8a406b4d0 0x7fc8a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.632+0000 7fc8b67fc640 1 -- 172.21.15.76:0/966421975 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc8b80ac820 con 0x7fc8c015b260 2024-03-08T06:02:08.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.632+0000 7fc8c5ad6640 1 --2- 172.21.15.76:0/966421975 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc8a406b4d0 0x7fc8a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.632+0000 7fc8c5ad6640 1 --2- 172.21.15.76:0/966421975 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc8a406b4d0 0x7fc8a406d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009560 tx=0x7fc8bc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:08.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.633+0000 7fc8ca9ee740 1 Processor -- start 2024-03-08T06:02:08.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.633+0000 7fc8ca9ee740 1 -- start start 2024-03-08T06:02:08.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.633+0000 7fc8ca9ee740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc669078a0 0x55cc66907c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.633+0000 7fc8ca9ee740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc6690b8f0 0x55cc6690d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.633+0000 7fc8ca9ee740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66937630 0x55cc66937ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.633+0000 7fc8ca9ee740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc66871e30 con 0x55cc66937630 2024-03-08T06:02:08.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.633+0000 7fc8ca9ee740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc6681ad20 con 0x55cc6690b8f0 2024-03-08T06:02:08.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.633+0000 7fc8ca9ee740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc6686ded0 con 0x55cc669078a0 2024-03-08T06:02:08.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.634+0000 7fc8c62d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66937630 0x55cc66937ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.634+0000 7fc8c62d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66937630 0x55cc66937ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51054/0 (socket says 172.21.15.76:51054) 2024-03-08T06:02:08.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.634+0000 7fc8c62d7640 1 -- 172.21.15.76:0/1118376166 learned_addr learned my addr 172.21.15.76:0/1118376166 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:08.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.634+0000 7fc8c5ad6640 1 --2- 172.21.15.76:0/1118376166 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc669078a0 0x55cc66907c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.634+0000 7fc8c52d5640 1 --2- 172.21.15.76:0/1118376166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc6690b8f0 0x55cc6690d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.634+0000 7fc8c62d7640 1 -- 172.21.15.76:0/1118376166 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc669078a0 msgr2=0x55cc66907c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:08.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.634+0000 7fc8c62d7640 1 --2- 172.21.15.76:0/1118376166 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc669078a0 0x55cc66907c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.634+0000 7fc8c62d7640 1 -- 172.21.15.76:0/1118376166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc6690b8f0 msgr2=0x55cc6690d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:08.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.634+0000 7fc8c62d7640 1 --2- 172.21.15.76:0/1118376166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc6690b8f0 0x55cc6690d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.634+0000 7fc8c62d7640 1 -- 172.21.15.76:0/1118376166 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc665863f0 con 0x55cc66937630 2024-03-08T06:02:08.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.634+0000 7fc8c52d5640 1 --2- 172.21.15.76:0/1118376166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc6690b8f0 0x55cc6690d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:08.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.634+0000 7fc8c62d7640 1 --2- 172.21.15.76:0/1118376166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66937630 0x55cc66937ab0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8037040 tx=0x7fc8b807aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931961d602692c4a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:08.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.634+0000 7fc8a37fe640 1 -- 172.21.15.76:0/1118376166 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b80a9070 con 0x55cc66937630 2024-03-08T06:02:08.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.634+0000 7fc8a37fe640 1 -- 172.21.15.76:0/1118376166 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b8083590 con 0x55cc66937630 2024-03-08T06:02:08.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.634+0000 7fc8a37fe640 1 -- 172.21.15.76:0/1118376166 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b80775a0 con 0x55cc66937630 2024-03-08T06:02:08.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.635+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/1118376166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66937630 msgr2=0x55cc66937ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:08.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.635+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/1118376166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66937630 0x55cc66937ab0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8037040 tx=0x7fc8b807aba0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.635+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/1118376166 shutdown_connections 2024-03-08T06:02:08.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.635+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/1118376166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66937630 0x55cc66937ab0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.635+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/1118376166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc6690b8f0 0x55cc6690d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.635+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/1118376166 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc669078a0 0x55cc66907c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.635+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/1118376166 >> 172.21.15.76:0/1118376166 conn(0x55cc66909920 msgr2=0x55cc6690b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:08.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.635+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/1118376166 shutdown_connections 2024-03-08T06:02:08.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.635+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/1118376166 wait complete. 2024-03-08T06:02:08.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.635+0000 7fc8ca9ee740 1 -- start start 2024-03-08T06:02:08.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.635+0000 7fc8ca9ee740 1 Processor -- start 2024-03-08T06:02:08.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.636+0000 7fc8ca9ee740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc66937630 0x55cc6692b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.636+0000 7fc8ca9ee740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66965e70 0x55cc66963080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.636+0000 7fc8ca9ee740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc66952f40 0x55cc6693aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.636+0000 7fc8ca9ee740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc66871e30 con 0x55cc66965e70 2024-03-08T06:02:08.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.636+0000 7fc8ca9ee740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc66810390 con 0x55cc66937630 2024-03-08T06:02:08.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.636+0000 7fc8ca9ee740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc6690bd80 con 0x55cc66952f40 2024-03-08T06:02:08.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.636+0000 7fc8ca9ee740 10 client.0 _ll_register_callbacks cb 0x55cc6682dc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:02:08.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.636+0000 7fc8c52d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66965e70 0x55cc66963080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8c5ad6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc66937630 0x55cc6692b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8c52d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66965e70 0x55cc66963080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51056/0 (socket says 172.21.15.76:51056) 2024-03-08T06:02:08.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8c52d5640 1 -- 172.21.15.76:0/3628437222 learned_addr learned my addr 172.21.15.76:0/3628437222 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:08.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8c62d7640 1 --2- 172.21.15.76:0/3628437222 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc66952f40 0x55cc6693aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8c52d5640 1 -- 172.21.15.76:0/3628437222 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc66952f40 msgr2=0x55cc6693aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:08.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8c52d5640 1 --2- 172.21.15.76:0/3628437222 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc66952f40 0x55cc6693aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8c52d5640 1 -- 172.21.15.76:0/3628437222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc66937630 msgr2=0x55cc6692b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:08.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8c52d5640 1 --2- 172.21.15.76:0/3628437222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc66937630 0x55cc6692b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8c52d5640 1 -- 172.21.15.76:0/3628437222 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc666507d0 con 0x55cc66965e70 2024-03-08T06:02:08.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8c52d5640 1 --2- 172.21.15.76:0/3628437222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66965e70 0x55cc66963080 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0004ba0 tx=0x7fc8b00015e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5954775f8b81c255 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:08.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-08T06:02:08.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8a0ff9640 1 -- 172.21.15.76:0/3628437222 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0014400 con 0x55cc66965e70 2024-03-08T06:02:08.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8a0ff9640 1 -- 172.21.15.76:0/3628437222 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc8b000c910 con 0x55cc66965e70 2024-03-08T06:02:08.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/3628437222 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8b8083590 con 0x55cc66965e70 2024-03-08T06:02:08.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8a0ff9640 1 -- 172.21.15.76:0/3628437222 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8b000cb10 con 0x55cc66965e70 2024-03-08T06:02:08.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8ca9ee740 10 client.5577 fetch_fsmap learned FSMap version 15 2024-03-08T06:02:08.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.637+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/3628437222 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55cc6693b480 con 0x55cc66965e70 2024-03-08T06:02:08.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.638+0000 7fc8a0ff9640 1 -- 172.21.15.76:0/3628437222 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b8083590 con 0x55cc66965e70 2024-03-08T06:02:08.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.638+0000 7fc8a0ff9640 1 -- 172.21.15.76:0/3628437222 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc8b000ccb0 con 0x55cc66965e70 2024-03-08T06:02:08.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.638+0000 7fc8ca9ee740 10 client.5577 fetch_fsmap finished waiting for FSMap version 15 2024-03-08T06:02:08.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.638+0000 7fc8ca9ee740 10 client.5577 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-03-08T06:02:08.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.638+0000 7fc8ca9ee740 20 client.5577 populate_metadata read hostname 'smithi076' 2024-03-08T06:02:08.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.638+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/3628437222 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x55cc66953320 0x55cc6692f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.638+0000 7fc8ca9ee740 4 client.5577 mds_command: new command op to 5358 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T06:02:08.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.638+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/3628437222 --> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc6687f370 con 0x55cc66953320 2024-03-08T06:02:08.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.638+0000 7fc8c5ad6640 1 --2- 172.21.15.76:0/3628437222 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x55cc66953320 0x55cc6692f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.638+0000 7fc8a0ff9640 10 client.5577 ms_handle_connect on v2:172.21.15.81:6832/1503895376 2024-03-08T06:02:08.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.638+0000 7fc8c5ad6640 1 --2- 172.21.15.76:0/3628437222 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x55cc66953320 0x55cc6692f5d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c788c686ab23938a server_cookie=7948ea6b5f0a8a3a in_seq=0 out_seq=0 2024-03-08T06:02:08.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.639+0000 7fc8a0ff9640 1 -- 172.21.15.76:0/3628437222 <== mds.0 v2:172.21.15.81:6832/1503895376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc6687f370 con 0x55cc66953320 2024-03-08T06:02:08.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.639+0000 7fc8a0ff9640 10 client.5577 handle_command_reply: tid=0 2024-03-08T06:02:08.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.639+0000 7fc8ca9ee740 1 client.5577 shutdown 2024-03-08T06:02:08.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.639+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/3628437222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66965e70 msgr2=0x55cc66963080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:08.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.639+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/3628437222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66965e70 0x55cc66963080 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0004ba0 tx=0x7fc8b00015e0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.639+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/3628437222 shutdown_connections 2024-03-08T06:02:08.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.639+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/3628437222 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x55cc66953320 0x55cc6692f5d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.639+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/3628437222 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc66952f40 0x55cc6693aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.639+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/3628437222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66965e70 0x55cc66963080 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.639+0000 7fc8a0ff9640 0 client.5577 ms_handle_reset on v2:172.21.15.81:6832/1503895376 2024-03-08T06:02:08.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.639+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/3628437222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc66937630 0x55cc6692b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.639+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/3628437222 >> 172.21.15.76:0/3628437222 conn(0x55cc669078a0 msgr2=0x55cc66953b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:08.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.639+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/3628437222 shutdown_connections 2024-03-08T06:02:08.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.639+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/3628437222 wait complete. 2024-03-08T06:02:08.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.639+0000 7fc8ca9ee740 20 client.5577 trim_cache size 0 max 16384 2024-03-08T06:02:08.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.651+0000 7fc8ca9ee740 1 Processor -- start 2024-03-08T06:02:08.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.651+0000 7fc8ca9ee740 1 -- start start 2024-03-08T06:02:08.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.651+0000 7fc8ca9ee740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc66937630 0x55cc66908060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.651+0000 7fc8ca9ee740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66952f40 0x55cc66953af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.651+0000 7fc8ca9ee740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc66965e70 0x55cc66952650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.651+0000 7fc8ca9ee740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01fe220 con 0x55cc66952f40 2024-03-08T06:02:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.651+0000 7fc8ca9ee740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01fd3a0 con 0x55cc66965e70 2024-03-08T06:02:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.651+0000 7fc8ca9ee740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c006dce0 con 0x55cc66937630 2024-03-08T06:02:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.651+0000 7fc8c52d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66952f40 0x55cc66953af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.652+0000 7fc8c52d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66952f40 0x55cc66953af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51060/0 (socket says 172.21.15.76:51060) 2024-03-08T06:02:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.652+0000 7fc8c52d5640 1 -- 172.21.15.76:0/1063474518 learned_addr learned my addr 172.21.15.76:0/1063474518 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.652+0000 7fc8c5ad6640 1 --2- 172.21.15.76:0/1063474518 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc66937630 0x55cc66908060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.652+0000 7fc8c62d7640 1 --2- 172.21.15.76:0/1063474518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc66965e70 0x55cc66952650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.652+0000 7fc8c52d5640 1 -- 172.21.15.76:0/1063474518 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc66937630 msgr2=0x55cc66908060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.652+0000 7fc8c52d5640 1 --2- 172.21.15.76:0/1063474518 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc66937630 0x55cc66908060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.652+0000 7fc8c52d5640 1 -- 172.21.15.76:0/1063474518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc66965e70 msgr2=0x55cc66952650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.652+0000 7fc8c52d5640 1 --2- 172.21.15.76:0/1063474518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc66965e70 0x55cc66952650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.652+0000 7fc8c52d5640 1 -- 172.21.15.76:0/1063474518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc66903630 con 0x55cc66952f40 2024-03-08T06:02:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.652+0000 7fc8c52d5640 1 --2- 172.21.15.76:0/1063474518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66952f40 0x55cc66953af0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0004860 tx=0x7fc8b000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17e8de831361407 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.652+0000 7fc8a2ffd640 1 -- 172.21.15.76:0/1063474518 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b002d070 con 0x55cc66952f40 2024-03-08T06:02:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.652+0000 7fc8a2ffd640 1 -- 172.21.15.76:0/1063474518 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b0019070 con 0x55cc66952f40 2024-03-08T06:02:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.652+0000 7fc8a2ffd640 1 -- 172.21.15.76:0/1063474518 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0007aa0 con 0x55cc66952f40 2024-03-08T06:02:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.653+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/1063474518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66952f40 msgr2=0x55cc66953af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.653+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/1063474518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66952f40 0x55cc66953af0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0004860 tx=0x7fc8b000e000 comp rx=0 tx=0).stop 2024-03-08T06:02:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.653+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/1063474518 shutdown_connections 2024-03-08T06:02:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.653+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/1063474518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc66965e70 0x55cc66952650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.653+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/1063474518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66952f40 0x55cc66953af0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.653+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/1063474518 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc66937630 0x55cc66908060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.653+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/1063474518 >> 172.21.15.76:0/1063474518 conn(0x55cc669078a0 msgr2=0x55cc66976650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.653+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/1063474518 shutdown_connections 2024-03-08T06:02:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.653+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/1063474518 wait complete. 2024-03-08T06:02:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.653+0000 7fc8ca9ee740 1 -- start start 2024-03-08T06:02:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.653+0000 7fc8ca9ee740 1 Processor -- start 2024-03-08T06:02:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.654+0000 7fc8ca9ee740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc66937630 0x55cc66934ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.654+0000 7fc8ca9ee740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66952f40 0x55cc669369b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.654+0000 7fc8ca9ee740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc66965e70 0x55cc6692e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.654+0000 7fc8ca9ee740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01fced0 con 0x55cc66952f40 2024-03-08T06:02:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.654+0000 7fc8ca9ee740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc669564a0 con 0x55cc66937630 2024-03-08T06:02:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.654+0000 7fc8ca9ee740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc6692ee20 con 0x55cc66965e70 2024-03-08T06:02:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.654+0000 7fc8ca9ee740 10 client.0 _ll_register_callbacks cb 0x55cc6693b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:02:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.654+0000 7fc8c52d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66952f40 0x55cc669369b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.654+0000 7fc8c62d7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc66965e70 0x55cc6692e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.654+0000 7fc8c52d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66952f40 0x55cc669369b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51072/0 (socket says 172.21.15.76:51072) 2024-03-08T06:02:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.654+0000 7fc8c5ad6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc66937630 0x55cc66934ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.654+0000 7fc8c52d5640 1 -- 172.21.15.76:0/2665291689 learned_addr learned my addr 172.21.15.76:0/2665291689 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8c52d5640 1 -- 172.21.15.76:0/2665291689 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc66965e70 msgr2=0x55cc6692e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8c52d5640 1 --2- 172.21.15.76:0/2665291689 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc66965e70 0x55cc6692e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8c52d5640 1 -- 172.21.15.76:0/2665291689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc66937630 msgr2=0x55cc66934ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8c52d5640 1 --2- 172.21.15.76:0/2665291689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc66937630 0x55cc66934ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8c52d5640 1 -- 172.21.15.76:0/2665291689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc66953c90 con 0x55cc66952f40 2024-03-08T06:02:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8c5ad6640 1 --2- 172.21.15.76:0/2665291689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc66937630 0x55cc66934ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:02:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8c52d5640 1 --2- 172.21.15.76:0/2665291689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66952f40 0x55cc669369b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc8b002b040 tx=0x7fc8b0001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ea6904df80f616 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-08T06:02:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8a0ff9640 1 -- 172.21.15.76:0/2665291689 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b002d040 con 0x55cc66952f40 2024-03-08T06:02:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8a0ff9640 1 -- 172.21.15.76:0/2665291689 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc8b0032ca0 con 0x55cc66952f40 2024-03-08T06:02:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/2665291689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8b0019070 con 0x55cc66952f40 2024-03-08T06:02:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8a0ff9640 1 -- 172.21.15.76:0/2665291689 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8b0032ea0 con 0x55cc66952f40 2024-03-08T06:02:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8ca9ee740 10 client.5583 fetch_fsmap learned FSMap version 15 2024-03-08T06:02:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/2665291689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55cc669083a0 con 0x55cc66952f40 2024-03-08T06:02:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8a0ff9640 1 -- 172.21.15.76:0/2665291689 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b0019070 con 0x55cc66952f40 2024-03-08T06:02:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8a0ff9640 1 -- 172.21.15.76:0/2665291689 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc8b0014740 con 0x55cc66952f40 2024-03-08T06:02:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8ca9ee740 10 client.5583 fetch_fsmap finished waiting for FSMap version 15 2024-03-08T06:02:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8ca9ee740 10 client.5583 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-03-08T06:02:08.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8ca9ee740 20 client.5583 populate_metadata read hostname 'smithi076' 2024-03-08T06:02:08.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/2665291689 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x55cc669c07e0 0x55cc66937a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:08.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8ca9ee740 4 client.5583 mds_command: new command op to 5358 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-08T06:02:08.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.655+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/2665291689 --> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x55cc6690a080 con 0x55cc669c07e0 2024-03-08T06:02:08.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.656+0000 7fc8c5ad6640 1 --2- 172.21.15.76:0/2665291689 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x55cc669c07e0 0x55cc66937a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:08.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.656+0000 7fc8a0ff9640 10 client.5583 ms_handle_connect on v2:172.21.15.81:6832/1503895376 2024-03-08T06:02:08.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.656+0000 7fc8c5ad6640 1 --2- 172.21.15.76:0/2665291689 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x55cc669c07e0 0x55cc66937a10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30350fbd96653a06 server_cookie=bc9759127657057c in_seq=0 out_seq=0 2024-03-08T06:02:08.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.656+0000 7fc8a0ff9640 1 -- 172.21.15.76:0/2665291689 <== mds.0 v2:172.21.15.81:6832/1503895376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26922 (crc 0 0 0) 0x55cc6690a080 con 0x55cc669c07e0 2024-03-08T06:02:08.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.656+0000 7fc8a0ff9640 10 client.5583 handle_command_reply: tid=0 2024-03-08T06:02:08.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.656+0000 7fc8ca9ee740 1 client.5583 shutdown 2024-03-08T06:02:08.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/2665291689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66952f40 msgr2=0x55cc669369b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:08.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/2665291689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66952f40 0x55cc669369b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc8b002b040 tx=0x7fc8b0001080 comp rx=0 tx=0).stop 2024-03-08T06:02:08.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/2665291689 shutdown_connections 2024-03-08T06:02:08.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/2665291689 >> [v2:172.21.15.81:6832/1503895376,v1:172.21.15.81:6834/1503895376] conn(0x55cc669c07e0 0x55cc66937a10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/2665291689 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x55cc66965e70 0x55cc6692e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8a0ff9640 0 client.5583 ms_handle_reset on v2:172.21.15.81:6832/1503895376 2024-03-08T06:02:08.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/2665291689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc66952f40 0x55cc669369b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8ca9ee740 1 --2- 172.21.15.76:0/2665291689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cc66937630 0x55cc66934ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/2665291689 >> 172.21.15.76:0/2665291689 conn(0x55cc669078a0 msgr2=0x55cc66906a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:08.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/2665291689 shutdown_connections 2024-03-08T06:02:08.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8ca9ee740 1 -- 172.21.15.76:0/2665291689 wait complete. 2024-03-08T06:02:08.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8ca9ee740 20 client.5583 trim_cache size 0 max 16384 2024-03-08T06:02:08.665 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-08T06:02:08.665 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker": [ 2024-03-08T06:02:08.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.666 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": { 2024-03-08T06:02:08.666 INFO:teuthology.orchestra.run.smithi076.stdout: "id": "0" 2024-03-08T06:02:08.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.666 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.666 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T06:02:08.666 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T06:02:08.666 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.666 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": { 2024-03-08T06:02:08.666 INFO:teuthology.orchestra.run.smithi076.stdout: "id": "1" 2024-03-08T06:02:08.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.666 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.667 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T06:02:08.667 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T06:02:08.667 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.667 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": { 2024-03-08T06:02:08.667 INFO:teuthology.orchestra.run.smithi076.stdout: "id": "2" 2024-03-08T06:02:08.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.667 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.667 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T06:02:08.667 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T06:02:08.667 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.667 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.667 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.668 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": [ 2024-03-08T06:02:08.668 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.668 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.668 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.668 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 1049, 2024-03-08T06:02:08.668 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 1037, 2024-03-08T06:02:08.668 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 261238, 2024-03-08T06:02:08.668 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 4586717, 2024-03-08T06:02:08.668 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 8, 2024-03-08T06:02:08.668 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 3, 2024-03-08T06:02:08.668 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.065116218, 2024-03-08T06:02:08.668 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.023617732, 2024-03-08T06:02:08.668 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.017278008, 2024-03-08T06:02:08.668 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.019116673, 2024-03-08T06:02:08.669 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T06:02:08.669 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1035, 2024-03-08T06:02:08.669 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.143037320, 2024-03-08T06:02:08.669 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000138200 2024-03-08T06:02:08.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.669 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-08T06:02:08.669 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 26, 2024-03-08T06:02:08.669 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000004967, 2024-03-08T06:02:08.669 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000191 2024-03-08T06:02:08.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.669 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-08T06:02:08.669 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 0 2024-03-08T06:02:08.669 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.669 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.669 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.670 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": [ 2024-03-08T06:02:08.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.670 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.670 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.670 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 76, 2024-03-08T06:02:08.670 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 69, 2024-03-08T06:02:08.670 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 117040, 2024-03-08T06:02:08.670 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 83157, 2024-03-08T06:02:08.670 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 9, 2024-03-08T06:02:08.670 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 7, 2024-03-08T06:02:08.670 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.012727151, 2024-03-08T06:02:08.670 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.001660869, 2024-03-08T06:02:08.670 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.015892762, 2024-03-08T06:02:08.671 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000469857, 2024-03-08T06:02:08.671 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T06:02:08.671 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 69, 2024-03-08T06:02:08.671 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.186180171, 2024-03-08T06:02:08.671 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002698263 2024-03-08T06:02:08.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.671 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-08T06:02:08.671 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 4, 2024-03-08T06:02:08.671 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000271, 2024-03-08T06:02:08.671 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000067 2024-03-08T06:02:08.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.671 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 113744, 2024-03-08T06:02:08.671 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 13312 2024-03-08T06:02:08.671 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.672 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.672 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.672 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": [ 2024-03-08T06:02:08.672 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.672 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.672 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.672 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 3054, 2024-03-08T06:02:08.672 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 4047, 2024-03-08T06:02:08.672 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 796051, 2024-03-08T06:02:08.672 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 1912106, 2024-03-08T06:02:08.672 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 8, 2024-03-08T06:02:08.672 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 4, 2024-03-08T06:02:08.672 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.138067828, 2024-03-08T06:02:08.672 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.061727008, 2024-03-08T06:02:08.672 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.097531338, 2024-03-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.001003187, 2024-03-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 4046, 2024-03-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.103254962, 2024-03-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000025520 2024-03-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 6541, 2024-03-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000811137, 2024-03-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000124 2024-03-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 94288 2024-03-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": [ 2024-03-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-03-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-03-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": [ 2024-03-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-03-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-03-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2054, 2024-03-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.407060466, 2024-03-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000198179 2024-03-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": [ 2024-03-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-03-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-03-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2, 2024-03-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000011902, 2024-03-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000005951 2024-03-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-03-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 1031, 2024-03-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 1031, 2024-03-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-03-08T06:02:08.677 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1031, 2024-03-08T06:02:08.677 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.562348534, 2024-03-08T06:02:08.677 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000545439 2024-03-08T06:02:08.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.677 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-03-08T06:02:08.677 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 0, 2024-03-08T06:02:08.677 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 0, 2024-03-08T06:02:08.677 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 0, 2024-03-08T06:02:08.677 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 12, 2024-03-08T06:02:08.677 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 0, 2024-03-08T06:02:08.677 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 0, 2024-03-08T06:02:08.677 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 1011, 2024-03-08T06:02:08.677 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 0, 2024-03-08T06:02:08.677 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 1011, 2024-03-08T06:02:08.678 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 0, 2024-03-08T06:02:08.678 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 1011, 2024-03-08T06:02:08.678 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 0, 2024-03-08T06:02:08.678 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 1002, 2024-03-08T06:02:08.678 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 1003, 2024-03-08T06:02:08.678 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 2, 2024-03-08T06:02:08.678 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 1061, 2024-03-08T06:02:08.678 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 1060, 2024-03-08T06:02:08.678 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 0, 2024-03-08T06:02:08.678 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 0, 2024-03-08T06:02:08.678 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 0, 2024-03-08T06:02:08.678 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-03-08T06:02:08.678 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 0, 2024-03-08T06:02:08.679 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 0, 2024-03-08T06:02:08.679 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 0, 2024-03-08T06:02:08.679 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 0, 2024-03-08T06:02:08.679 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 0, 2024-03-08T06:02:08.679 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 0, 2024-03-08T06:02:08.679 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 0, 2024-03-08T06:02:08.679 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-03-08T06:02:08.679 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 0, 2024-03-08T06:02:08.679 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-03-08T06:02:08.679 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 863, 2024-03-08T06:02:08.679 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 2476, 2024-03-08T06:02:08.679 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 0, 2024-03-08T06:02:08.679 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 0, 2024-03-08T06:02:08.679 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-03-08T06:02:08.679 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-03-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-03-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 0, 2024-03-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 0, 2024-03-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 0, 2024-03-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 0, 2024-03-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 0, 2024-03-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 1, 2024-03-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 1000, 2024-03-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 0, 2024-03-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 1001, 2024-03-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 1000, 2024-03-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 0, 2024-03-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 3 2024-03-08T06:02:08.681 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.681 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.681 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.681 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": [ 2024-03-08T06:02:08.681 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.681 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.681 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.681 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 0, 2024-03-08T06:02:08.681 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 0, 2024-03-08T06:02:08.681 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 0, 2024-03-08T06:02:08.681 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 0, 2024-03-08T06:02:08.681 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 0, 2024-03-08T06:02:08.681 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 0, 2024-03-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 0, 2024-03-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-03-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 0, 2024-03-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 0, 2024-03-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-03-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-03-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-03-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-03-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-03-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-03-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-03-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 0, 2024-03-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 0, 2024-03-08T06:02:08.683 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-03-08T06:02:08.683 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 0, 2024-03-08T06:02:08.683 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-03-08T06:02:08.683 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-03-08T06:02:08.683 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.683 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.683 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.683 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_client_metrics": [ 2024-03-08T06:02:08.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.683 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": { 2024-03-08T06:02:08.683 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-03-08T06:02:08.683 INFO:teuthology.orchestra.run.smithi076.stdout: "id": "5358" 2024-03-08T06:02:08.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.683 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi076.stdout: "num_clients": 2 2024-03-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_client_metrics-cephfs": [ 2024-03-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": { 2024-03-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.5541", 2024-03-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": "0" 2024-03-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_hits": 3742, 2024-03-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_miss": 19, 2024-03-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_read_latency": 0.000000000, 2024-03-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_write_latency": 0.000000000, 2024-03-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_metadata_latency": 0.000000000, 2024-03-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_lease_hits": 372, 2024-03-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_lease_miss": 0, 2024-03-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi076.stdout: "opened_files": 2, 2024-03-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi076.stdout: "opened_inodes": 373, 2024-03-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi076.stdout: "pinned_icaps": 373, 2024-03-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi076.stdout: "total_inodes": 373, 2024-03-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi076.stdout: "total_read_ops": 0, 2024-03-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi076.stdout: "total_read_size": 0, 2024-03-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi076.stdout: "total_write_ops": 371, 2024-03-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi076.stdout: "total_write_size": 1003 2024-03-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": { 2024-03-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.5550", 2024-03-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": "0" 2024-03-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_hits": 41, 2024-03-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_miss": 0, 2024-03-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_read_latency": 0.000000000, 2024-03-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_write_latency": 0.000000000, 2024-03-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_metadata_latency": 0.000000000, 2024-03-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_lease_hits": 0, 2024-03-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_lease_miss": 0, 2024-03-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi076.stdout: "opened_files": 0, 2024-03-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi076.stdout: "opened_inodes": 1, 2024-03-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi076.stdout: "pinned_icaps": 1, 2024-03-08T06:02:08.687 INFO:teuthology.orchestra.run.smithi076.stdout: "total_inodes": 1, 2024-03-08T06:02:08.687 INFO:teuthology.orchestra.run.smithi076.stdout: "total_read_ops": 0, 2024-03-08T06:02:08.687 INFO:teuthology.orchestra.run.smithi076.stdout: "total_read_size": 0, 2024-03-08T06:02:08.687 INFO:teuthology.orchestra.run.smithi076.stdout: "total_write_ops": 0, 2024-03-08T06:02:08.687 INFO:teuthology.orchestra.run.smithi076.stdout: "total_write_size": 0 2024-03-08T06:02:08.687 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.687 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.687 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.687 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": [ 2024-03-08T06:02:08.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.687 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.687 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.687 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 0, 2024-03-08T06:02:08.687 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 2008, 2024-03-08T06:02:08.687 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 0, 2024-03-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 0, 2024-03-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 2008, 2024-03-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-03-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 0, 2024-03-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 3, 2024-03-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 0, 2024-03-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 0, 2024-03-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 3, 2024-03-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 2, 2024-03-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-03-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 0, 2024-03-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 4194304, 2024-03-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 8387435, 2024-03-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 0, 2024-03-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-03-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1004, 2024-03-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 1.473092014, 2024-03-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001467223 2024-03-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 0 2024-03-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": [ 2024-03-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1014, 2024-03-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 1014, 2024-03-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 0, 2024-03-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 13, 2024-03-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 13, 2024-03-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 0, 2024-03-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 1011, 2024-03-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 1011, 2024-03-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 0, 2024-03-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 1003, 2024-03-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 1003, 2024-03-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 0, 2024-03-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 54360, 2024-03-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 215392 2024-03-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": [ 2024-03-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 1032, 2024-03-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 0, 2024-03-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 1031, 2024-03-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 1002, 2024-03-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 0, 2024-03-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-03-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1000, 2024-03-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.552556493, 2024-03-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000552556 2024-03-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-03-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 29, 2024-03-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.005759988, 2024-03-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000198620 2024-03-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-03-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-03-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-03-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-03-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-03-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.694 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-03-08T06:02:08.694 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.694 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.694 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.694 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-03-08T06:02:08.694 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.694 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.694 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.694 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-03-08T06:02:08.694 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.694 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.695 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.695 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-03-08T06:02:08.695 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.695 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.695 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.695 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-03-08T06:02:08.695 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1, 2024-03-08T06:02:08.695 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.003486053, 2024-03-08T06:02:08.695 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003486053 2024-03-08T06:02:08.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.695 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-03-08T06:02:08.695 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.695 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.696 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.696 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-03-08T06:02:08.696 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.696 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.696 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.696 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-03-08T06:02:08.696 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.696 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.696 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.696 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-03-08T06:02:08.696 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.696 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.697 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.697 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-03-08T06:02:08.697 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.697 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.697 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.697 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-03-08T06:02:08.697 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.697 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.697 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.697 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-03-08T06:02:08.697 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.697 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.698 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.698 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-03-08T06:02:08.698 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.698 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.698 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.698 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-03-08T06:02:08.698 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.698 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.698 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.698 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-03-08T06:02:08.699 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.699 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.699 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.699 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-03-08T06:02:08.699 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1, 2024-03-08T06:02:08.699 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000546000, 2024-03-08T06:02:08.699 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000546000 2024-03-08T06:02:08.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.699 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-03-08T06:02:08.699 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.699 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.699 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.700 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-03-08T06:02:08.700 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.700 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.700 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.700 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-03-08T06:02:08.700 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.700 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.700 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.700 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-03-08T06:02:08.700 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.700 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.700 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.701 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-03-08T06:02:08.701 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.701 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.701 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.701 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-03-08T06:02:08.701 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.701 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.701 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.701 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-03-08T06:02:08.701 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-03-08T06:02:08.701 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-03-08T06:02:08.701 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.702 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.702 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.702 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.702 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.702 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.702 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.702 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": [ 2024-03-08T06:02:08.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.702 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.702 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.702 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 2, 2024-03-08T06:02:08.702 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 2, 2024-03-08T06:02:08.702 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 0, 2024-03-08T06:02:08.702 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 2, 2024-03-08T06:02:08.703 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-03-08T06:02:08.703 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 1007, 2024-03-08T06:02:08.703 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 503, 2024-03-08T06:02:08.703 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 12, 2024-03-08T06:02:08.703 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-03-08T06:02:08.703 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.703 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.703 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.703 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": [ 2024-03-08T06:02:08.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.703 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.703 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.703 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-03-08T06:02:08.703 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-03-08T06:02:08.703 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-03-08T06:02:08.703 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-03-08T06:02:08.704 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-03-08T06:02:08.704 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-03-08T06:02:08.704 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-08T06:02:08.704 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-03-08T06:02:08.704 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-08T06:02:08.704 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-03-08T06:02:08.704 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-03-08T06:02:08.704 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-03-08T06:02:08.704 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-08T06:02:08.704 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-03-08T06:02:08.704 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-03-08T06:02:08.704 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-03-08T06:02:08.704 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-03-08T06:02:08.704 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-03-08T06:02:08.704 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-08T06:02:08.705 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-03-08T06:02:08.705 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-08T06:02:08.705 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-03-08T06:02:08.705 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-03-08T06:02:08.705 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-03-08T06:02:08.705 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-03-08T06:02:08.705 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-03-08T06:02:08.705 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-08T06:02:08.705 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-03-08T06:02:08.705 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-03-08T06:02:08.705 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-03-08T06:02:08.705 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-03-08T06:02:08.705 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-03-08T06:02:08.705 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-03-08T06:02:08.705 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-03-08T06:02:08.706 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-03-08T06:02:08.706 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-03-08T06:02:08.706 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 279577, 2024-03-08T06:02:08.706 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 367, 2024-03-08T06:02:08.706 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 1760, 2024-03-08T06:02:08.706 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 20, 2024-03-08T06:02:08.706 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-03-08T06:02:08.706 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-03-08T06:02:08.706 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-03-08T06:02:08.706 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-03-08T06:02:08.706 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-03-08T06:02:08.706 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-03-08T06:02:08.706 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 5424, 2024-03-08T06:02:08.706 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 76, 2024-03-08T06:02:08.707 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-03-08T06:02:08.707 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-03-08T06:02:08.707 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-03-08T06:02:08.707 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-03-08T06:02:08.707 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 915330, 2024-03-08T06:02:08.707 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 10231, 2024-03-08T06:02:08.707 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-03-08T06:02:08.707 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-03-08T06:02:08.707 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-03-08T06:02:08.707 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-03-08T06:02:08.707 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.707 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.707 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.707 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": [ 2024-03-08T06:02:08.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.707 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.708 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.708 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-03-08T06:02:08.708 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-03-08T06:02:08.708 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 1046, 2024-03-08T06:02:08.708 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 4259116, 2024-03-08T06:02:08.708 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 37, 2024-03-08T06:02:08.708 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 1046, 2024-03-08T06:02:08.708 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-03-08T06:02:08.708 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1046, 2024-03-08T06:02:08.708 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 2.311953605, 2024-03-08T06:02:08.708 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002210280 2024-03-08T06:02:08.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.708 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-03-08T06:02:08.708 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-03-08T06:02:08.708 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1046, 2024-03-08T06:02:08.708 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 1073 2024-03-08T06:02:08.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-08T06:02:08.709 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 1046, 2024-03-08T06:02:08.709 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 0, 2024-03-08T06:02:08.709 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 1046, 2024-03-08T06:02:08.709 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-03-08T06:02:08.709 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-03-08T06:02:08.709 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 0, 2024-03-08T06:02:08.709 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 11, 2024-03-08T06:02:08.709 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 0, 2024-03-08T06:02:08.709 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 1004, 2024-03-08T06:02:08.709 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 8, 2024-03-08T06:02:08.709 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-03-08T06:02:08.709 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-03-08T06:02:08.709 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 0, 2024-03-08T06:02:08.709 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 0, 2024-03-08T06:02:08.709 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 6, 2024-03-08T06:02:08.710 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-03-08T06:02:08.710 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-03-08T06:02:08.710 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-03-08T06:02:08.710 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 0, 2024-03-08T06:02:08.710 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 22, 2024-03-08T06:02:08.710 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-03-08T06:02:08.710 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-03-08T06:02:08.710 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-03-08T06:02:08.710 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-03-08T06:02:08.710 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-03-08T06:02:08.710 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-03-08T06:02:08.710 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-03-08T06:02:08.710 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-03-08T06:02:08.710 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-03-08T06:02:08.711 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 0, 2024-03-08T06:02:08.711 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-03-08T06:02:08.711 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-03-08T06:02:08.711 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-03-08T06:02:08.711 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-03-08T06:02:08.711 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-03-08T06:02:08.711 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-03-08T06:02:08.711 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-03-08T06:02:08.711 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-03-08T06:02:08.711 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-03-08T06:02:08.711 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-03-08T06:02:08.711 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-03-08T06:02:08.711 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-03-08T06:02:08.711 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-03-08T06:02:08.711 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-03-08T06:02:08.711 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-03-08T06:02:08.712 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-03-08T06:02:08.712 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 40, 2024-03-08T06:02:08.712 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-03-08T06:02:08.712 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 1, 2024-03-08T06:02:08.712 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 8, 2024-03-08T06:02:08.712 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 8, 2024-03-08T06:02:08.712 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 0, 2024-03-08T06:02:08.712 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-03-08T06:02:08.712 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 21, 2024-03-08T06:02:08.712 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 0, 2024-03-08T06:02:08.712 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 1 2024-03-08T06:02:08.712 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.712 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.712 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.713 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": [ 2024-03-08T06:02:08.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.713 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.713 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.713 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 1, 2024-03-08T06:02:08.713 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 0, 2024-03-08T06:02:08.713 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 0, 2024-03-08T06:02:08.713 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-03-08T06:02:08.713 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.713 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.713 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.713 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": [ 2024-03-08T06:02:08.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.713 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 0 2024-03-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": [ 2024-03-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-03-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 0, 2024-03-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-03-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 0, 2024-03-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 0, 2024-03-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 0, 2024-03-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-03-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-03-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-03-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-03-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 4174, 2024-03-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 794217, 2024-03-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-03-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 4174, 2024-03-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-03-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-03-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 4174, 2024-03-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 794217, 2024-03-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-03-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": [ 2024-03-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-03-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-03-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-03-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-03-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-03-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-03-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-03-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 1046, 2024-03-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 4259116, 2024-03-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 1046, 2024-03-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 4259116, 2024-03-08T06:02:08.718 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-03-08T06:02:08.718 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.718 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.718 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.718 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.718 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.718 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.718 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.718 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": [ 2024-03-08T06:02:08.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.718 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.718 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.718 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-03-08T06:02:08.718 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-03-08T06:02:08.718 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-03-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-03-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-03-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-03-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-03-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 1046, 2024-03-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 1046, 2024-03-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 1046, 2024-03-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1046, 2024-03-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-03-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": [ 2024-03-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-03-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-03-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-03-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-03-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-03-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-03-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-03-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-03-08T06:02:08.721 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-03-08T06:02:08.721 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 0, 2024-03-08T06:02:08.721 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 0, 2024-03-08T06:02:08.721 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-03-08T06:02:08.721 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.721 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.721 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.721 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-08T06:02:08.721 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x55c65ff9a6e0": [ 2024-03-08T06:02:08.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-08T06:02:08.721 INFO:teuthology.orchestra.run.smithi076.stdout: "labels": {}, 2024-03-08T06:02:08.722 INFO:teuthology.orchestra.run.smithi076.stdout: "counters": { 2024-03-08T06:02:08.722 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-03-08T06:02:08.722 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-03-08T06:02:08.722 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-03-08T06:02:08.722 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 2008, 2024-03-08T06:02:08.722 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 4193131, 2024-03-08T06:02:08.722 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-03-08T06:02:08.722 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 2008, 2024-03-08T06:02:08.722 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-03-08T06:02:08.722 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-03-08T06:02:08.722 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 1004, 2024-03-08T06:02:08.722 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 4193131, 2024-03-08T06:02:08.722 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-03-08T06:02:08.722 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-08T06:02:08.723 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-08T06:02:08.723 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-08T06:02:08.723 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.723 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.723 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-08T06:02:08.723 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-03-08T06:02:08.723 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-08T06:02:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8c7d3a640 1 -- 172.21.15.76:0/966421975 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc8a406b4d0 msgr2=0x7fc8a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8c7d3a640 1 --2- 172.21.15.76:0/966421975 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc8a406b4d0 0x7fc8a406d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009560 tx=0x7fc8bc009190 comp rx=0 tx=0).stop 2024-03-08T06:02:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8c7d3a640 1 -- 172.21.15.76:0/966421975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8c015b260 msgr2=0x7fc8c0132af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8c7d3a640 1 --2- 172.21.15.76:0/966421975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8c015b260 0x7fc8c0132af0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009d90 tx=0x7fc8b8004a40 comp rx=0 tx=0).stop 2024-03-08T06:02:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8c7d3a640 1 -- 172.21.15.76:0/966421975 shutdown_connections 2024-03-08T06:02:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8c7d3a640 1 --2- 172.21.15.76:0/966421975 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc8a406b4d0 0x7fc8a406d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8c7d3a640 1 --2- 172.21.15.76:0/966421975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8c015b260 0x7fc8c0132af0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8c7d3a640 1 --2- 172.21.15.76:0/966421975 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc8c00b2e20 0x7fc8c013c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8c7d3a640 1 --2- 172.21.15.76:0/966421975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8c00aee10 0x7fc8c01658c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8c7d3a640 1 -- 172.21.15.76:0/966421975 >> 172.21.15.76:0/966421975 conn(0x7fc8c006ac80 msgr2=0x7fc8c006b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8c7d3a640 1 -- 172.21.15.76:0/966421975 shutdown_connections 2024-03-08T06:02:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:08.657+0000 7fc8c7d3a640 1 -- 172.21.15.76:0/966421975 wait complete. 2024-03-08T06:02:35.975 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:02:35.979 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:02:37.742 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-08T06:02:37.742 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-08T06:02:37.783 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-08T06:02:37.829 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-08T06:02:37.829 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:02:37.829 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:02:37.846 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:02:37.846 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:02:37.903 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:02:38.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.049+0000 7fa10e981640 1 Processor -- start 2024-03-08T06:02:38.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.049+0000 7fa10e981640 1 -- start start 2024-03-08T06:02:38.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.049+0000 7fa10e981640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1080d0b90 0x7fa108168bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.049+0000 7fa10e981640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1080cf180 con 0x7fa10815dde0 2024-03-08T06:02:38.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.049+0000 7fa10e981640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1080d1ed0 con 0x7fa1080b6d00 2024-03-08T06:02:38.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.049+0000 7fa10e981640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1080d21b0 con 0x7fa1080d0b90 2024-03-08T06:02:38.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.050+0000 7fa107fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa10815dde0 0x7fa108162250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40432/0 (socket says 172.21.15.76:40432) 2024-03-08T06:02:38.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.050+0000 7fa107fff640 1 -- 172.21.15.76:0/732309976 learned_addr learned my addr 172.21.15.76:0/732309976 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:38.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.050+0000 7fa10cf1e640 1 --2- 172.21.15.76:0/732309976 >> v2:172.21.15.76:3300/0 conn(0x7fa1080d0b90 0x7fa108168bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.050+0000 7fa106ffd640 1 -- 172.21.15.76:0/732309976 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807823295 0 0) 0x7fa1080cf180 con 0x7fa10815dde0 2024-03-08T06:02:38.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.050+0000 7fa106ffd640 1 -- 172.21.15.76:0/732309976 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0f0003520 con 0x7fa10815dde0 2024-03-08T06:02:38.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.050+0000 7fa10cf1e640 1 -- 172.21.15.76:0/732309976 >> v1:172.21.15.81:6789/0 conn(0x7fa1080b6d00 legacy=0x7fa1080d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.050+0000 7fa10cf1e640 1 -- 172.21.15.76:0/732309976 >> v1:172.21.15.76:6789/0 conn(0x7fa10815dde0 legacy=0x7fa108162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.050+0000 7fa10cf1e640 1 -- 172.21.15.76:0/732309976 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1080cec10 con 0x7fa1080d0b90 2024-03-08T06:02:38.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.051+0000 7fa10cf1e640 1 --2- 172.21.15.76:0/732309976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1080d0b90 0x7fa108168bd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa100009d90 tx=0x7fa10002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e43542453b8794 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:38.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.051+0000 7fa106ffd640 1 -- 172.21.15.76:0/732309976 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100039070 con 0x7fa1080d0b90 2024-03-08T06:02:38.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.051+0000 7fa106ffd640 1 -- 172.21.15.76:0/732309976 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10002cc60 con 0x7fa1080d0b90 2024-03-08T06:02:38.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.051+0000 7fa106ffd640 1 -- 172.21.15.76:0/732309976 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100034660 con 0x7fa1080d0b90 2024-03-08T06:02:38.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.051+0000 7fa10e981640 1 -- 172.21.15.76:0/732309976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1080d0b90 msgr2=0x7fa108168bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.051+0000 7fa10e981640 1 --2- 172.21.15.76:0/732309976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1080d0b90 0x7fa108168bd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa100009d90 tx=0x7fa10002beb0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.051+0000 7fa10e981640 1 -- 172.21.15.76:0/732309976 shutdown_connections 2024-03-08T06:02:38.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.051+0000 7fa10e981640 1 --2- 172.21.15.76:0/732309976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1080d0b90 0x7fa108168bd0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.051+0000 7fa10e981640 1 -- 172.21.15.76:0/732309976 >> 172.21.15.76:0/732309976 conn(0x7fa1080b56e0 msgr2=0x7fa1080b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:38.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.051+0000 7fa10e981640 1 -- 172.21.15.76:0/732309976 shutdown_connections 2024-03-08T06:02:38.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.051+0000 7fa10e981640 1 -- 172.21.15.76:0/732309976 wait complete. 2024-03-08T06:02:38.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.053+0000 7fa10e981640 1 Processor -- start 2024-03-08T06:02:38.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.053+0000 7fa10e981640 1 -- start start 2024-03-08T06:02:38.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa10e981640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa1080b6d00 0x7fa1081522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa10e981640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1080d0b90 0x7fa108157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa10e981640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10815dde0 0x7fa1080b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa10e981640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1081694a0 con 0x7fa10815dde0 2024-03-08T06:02:38.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa10e981640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa108169320 con 0x7fa1080d0b90 2024-03-08T06:02:38.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa10e981640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa108169620 con 0x7fa1080b6d00 2024-03-08T06:02:38.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa10cf1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10815dde0 0x7fa1080b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa10cf1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10815dde0 0x7fa1080b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47720/0 (socket says 172.21.15.76:47720) 2024-03-08T06:02:38.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa10cf1e640 1 -- 172.21.15.76:0/1969066115 learned_addr learned my addr 172.21.15.76:0/1969066115 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:38.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa107fff640 1 --2- 172.21.15.76:0/1969066115 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa1080b6d00 0x7fa1081522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa1077fe640 1 --2- 172.21.15.76:0/1969066115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1080d0b90 0x7fa108157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa10cf1e640 1 -- 172.21.15.76:0/1969066115 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa1080b6d00 msgr2=0x7fa1081522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa10cf1e640 1 --2- 172.21.15.76:0/1969066115 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa1080b6d00 0x7fa1081522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa10cf1e640 1 -- 172.21.15.76:0/1969066115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1080d0b90 msgr2=0x7fa108157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa10cf1e640 1 --2- 172.21.15.76:0/1969066115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1080d0b90 0x7fa108157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa10cf1e640 1 -- 172.21.15.76:0/1969066115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa108157520 con 0x7fa10815dde0 2024-03-08T06:02:38.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.054+0000 7fa10cf1e640 1 --2- 172.21.15.76:0/1969066115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10815dde0 0x7fa1080b3550 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa100009eb0 tx=0x7fa100004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:38.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.055+0000 7fa104ff9640 1 -- 172.21.15.76:0/1969066115 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100039050 con 0x7fa10815dde0 2024-03-08T06:02:38.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.055+0000 7fa104ff9640 1 -- 172.21.15.76:0/1969066115 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa100045070 con 0x7fa10815dde0 2024-03-08T06:02:38.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.055+0000 7fa10e981640 1 -- 172.21.15.76:0/1969066115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1080caca0 con 0x7fa10815dde0 2024-03-08T06:02:38.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.055+0000 7fa104ff9640 1 -- 172.21.15.76:0/1969066115 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100009450 con 0x7fa10815dde0 2024-03-08T06:02:38.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.055+0000 7fa10e981640 1 -- 172.21.15.76:0/1969066115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1080cb260 con 0x7fa10815dde0 2024-03-08T06:02:38.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.056+0000 7fa10e981640 1 -- 172.21.15.76:0/1969066115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10804f030 con 0x7fa10815dde0 2024-03-08T06:02:38.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.057+0000 7fa104ff9640 1 -- 172.21.15.76:0/1969066115 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa10003e850 con 0x7fa10815dde0 2024-03-08T06:02:38.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.057+0000 7fa104ff9640 1 --2- 172.21.15.76:0/1969066115 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa0f406b4d0 0x7fa0f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.057+0000 7fa104ff9640 1 -- 172.21.15.76:0/1969066115 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fa1000ad280 con 0x7fa10815dde0 2024-03-08T06:02:38.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.057+0000 7fa107fff640 1 --2- 172.21.15.76:0/1969066115 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa0f406b4d0 0x7fa0f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.058+0000 7fa107fff640 1 --2- 172.21.15.76:0/1969066115 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa0f406b4d0 0x7fa0f406d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80011e0 tx=0x7fa0f8007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:38.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.062+0000 7fa104ff9640 1 -- 172.21.15.76:0/1969066115 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa10007ba20 con 0x7fa10815dde0 2024-03-08T06:02:38.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.232+0000 7fa10e981640 1 -- 172.21.15.76:0/1969066115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa108159140 con 0x7fa10815dde0 2024-03-08T06:02:38.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.233+0000 7fa104ff9640 1 -- 172.21.15.76:0/1969066115 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16850 (secure 0 0 0) 0x7fa10007fac0 con 0x7fa10815dde0 2024-03-08T06:02:38.234 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:02:38.234 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":42,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:02:18.021974+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:01:45.628991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-08T06:01:46.195100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:02:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.235+0000 7fa10e981640 1 -- 172.21.15.76:0/1969066115 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa0f406b4d0 msgr2=0x7fa0f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.235+0000 7fa10e981640 1 --2- 172.21.15.76:0/1969066115 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa0f406b4d0 0x7fa0f406d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80011e0 tx=0x7fa0f8007d30 comp rx=0 tx=0).stop 2024-03-08T06:02:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.235+0000 7fa10e981640 1 -- 172.21.15.76:0/1969066115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10815dde0 msgr2=0x7fa1080b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.235+0000 7fa10e981640 1 --2- 172.21.15.76:0/1969066115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10815dde0 0x7fa1080b3550 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa100009eb0 tx=0x7fa100004a40 comp rx=0 tx=0).stop 2024-03-08T06:02:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.235+0000 7fa10e981640 1 -- 172.21.15.76:0/1969066115 shutdown_connections 2024-03-08T06:02:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.235+0000 7fa10e981640 1 --2- 172.21.15.76:0/1969066115 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa0f406b4d0 0x7fa0f406d990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.235+0000 7fa10e981640 1 --2- 172.21.15.76:0/1969066115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa10815dde0 0x7fa1080b3550 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.235+0000 7fa10e981640 1 --2- 172.21.15.76:0/1969066115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1080d0b90 0x7fa108157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.235+0000 7fa10e981640 1 --2- 172.21.15.76:0/1969066115 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa1080b6d00 0x7fa1081522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.235+0000 7fa10e981640 1 -- 172.21.15.76:0/1969066115 >> 172.21.15.76:0/1969066115 conn(0x7fa1080b56e0 msgr2=0x7fa1081617d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.236+0000 7fa10e981640 1 -- 172.21.15.76:0/1969066115 shutdown_connections 2024-03-08T06:02:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.236+0000 7fa10e981640 1 -- 172.21.15.76:0/1969066115 wait complete. 2024-03-08T06:02:38.247 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:01:45.628991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:01:46.195100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:02:38.247 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:02:38.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.350+0000 7f894480e640 1 Processor -- start 2024-03-08T06:02:38.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.350+0000 7f894480e640 1 -- start start 2024-03-08T06:02:38.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.351+0000 7f894480e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8940154df0 0x7f89400b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.351+0000 7f894480e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f89400cc7e0 0x7f894015af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.351+0000 7f894480e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f894015f450 con 0x7f89400b3640 2024-03-08T06:02:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.351+0000 7f894480e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89400af1e0 con 0x7f89400cc7e0 2024-03-08T06:02:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.351+0000 7f894480e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89400b3c80 con 0x7f8940154df0 2024-03-08T06:02:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.351+0000 7f893ed9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f89400cc7e0 0x7f894015af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.351+0000 7f893ed9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f89400cc7e0 0x7f894015af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47724/0 (socket says 172.21.15.76:47724) 2024-03-08T06:02:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.351+0000 7f893ed9d640 1 -- 172.21.15.76:0/1467975709 learned_addr learned my addr 172.21.15.76:0/1467975709 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.351+0000 7f893e59c640 1 --2- 172.21.15.76:0/1467975709 >> v2:172.21.15.81:3300/0 conn(0x7f8940154df0 0x7f89400b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.351+0000 7f893ed9d640 1 -- 172.21.15.76:0/1467975709 >> v2:172.21.15.81:3300/0 conn(0x7f8940154df0 msgr2=0x7f89400b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.351+0000 7f893ed9d640 1 --2- 172.21.15.76:0/1467975709 >> v2:172.21.15.81:3300/0 conn(0x7f8940154df0 0x7f89400b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.351+0000 7f893ed9d640 1 -- 172.21.15.76:0/1467975709 >> v1:172.21.15.81:6789/0 conn(0x7f89400b3640 legacy=0x7f89400cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.351+0000 7f893ed9d640 1 -- 172.21.15.76:0/1467975709 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894015e690 con 0x7f89400cc7e0 2024-03-08T06:02:38.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.352+0000 7f893ed9d640 1 --2- 172.21.15.76:0/1467975709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89400cc7e0 0x7f894015af70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f894006ba20 tx=0x7f893002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b6a4817efb7f6f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:38.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.352+0000 7f893d59a640 1 -- 172.21.15.76:0/1467975709 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893002c9f0 con 0x7f89400cc7e0 2024-03-08T06:02:38.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.352+0000 7f893d59a640 1 -- 172.21.15.76:0/1467975709 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893002cb90 con 0x7f89400cc7e0 2024-03-08T06:02:38.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.352+0000 7f893d59a640 1 -- 172.21.15.76:0/1467975709 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89300343f0 con 0x7f89400cc7e0 2024-03-08T06:02:38.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.352+0000 7f894480e640 1 -- 172.21.15.76:0/1467975709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89400cc7e0 msgr2=0x7f894015af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.352+0000 7f894480e640 1 --2- 172.21.15.76:0/1467975709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89400cc7e0 0x7f894015af70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f894006ba20 tx=0x7f893002be50 comp rx=0 tx=0).stop 2024-03-08T06:02:38.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.352+0000 7f894480e640 1 -- 172.21.15.76:0/1467975709 shutdown_connections 2024-03-08T06:02:38.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.352+0000 7f894480e640 1 --2- 172.21.15.76:0/1467975709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89400cc7e0 0x7f894015af70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.352+0000 7f894480e640 1 --2- 172.21.15.76:0/1467975709 >> v2:172.21.15.81:3300/0 conn(0x7f8940154df0 0x7f89400b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.352+0000 7f894480e640 1 -- 172.21.15.76:0/1467975709 >> 172.21.15.76:0/1467975709 conn(0x7f894006ac80 msgr2=0x7f894006b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:38.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.352+0000 7f894480e640 1 -- 172.21.15.76:0/1467975709 shutdown_connections 2024-03-08T06:02:38.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.352+0000 7f894480e640 1 -- 172.21.15.76:0/1467975709 wait complete. 2024-03-08T06:02:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.354+0000 7f894480e640 1 Processor -- start 2024-03-08T06:02:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.354+0000 7f894480e640 1 -- start start 2024-03-08T06:02:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.354+0000 7f894480e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89400b3640 0x7f89400b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.354+0000 7f894480e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f89400cc7e0 0x7f8940147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.354+0000 7f894480e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8940154df0 0x7f8940159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.354+0000 7f894480e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894015ef30 con 0x7f89400b3640 2024-03-08T06:02:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.354+0000 7f894480e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894015ec50 con 0x7f8940154df0 2024-03-08T06:02:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.354+0000 7f894480e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89401551d0 con 0x7f89400cc7e0 2024-03-08T06:02:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.354+0000 7f893e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89400b3640 0x7f89400b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.355+0000 7f893e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89400b3640 0x7f89400b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47734/0 (socket says 172.21.15.76:47734) 2024-03-08T06:02:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.355+0000 7f893e59c640 1 -- 172.21.15.76:0/1708110064 learned_addr learned my addr 172.21.15.76:0/1708110064 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.355+0000 7f893e59c640 1 -- 172.21.15.76:0/1708110064 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f89400cc7e0 msgr2=0x7f8940147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.355+0000 7f893e59c640 1 --2- 172.21.15.76:0/1708110064 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f89400cc7e0 0x7f8940147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.355+0000 7f893e59c640 1 -- 172.21.15.76:0/1708110064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8940154df0 msgr2=0x7f8940159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.355+0000 7f893e59c640 1 --2- 172.21.15.76:0/1708110064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8940154df0 0x7f8940159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.355+0000 7f893e59c640 1 -- 172.21.15.76:0/1708110064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894015bf80 con 0x7f89400b3640 2024-03-08T06:02:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.355+0000 7f893e59c640 1 --2- 172.21.15.76:0/1708110064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89400b3640 0x7f89400b0970 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f893800d620 tx=0x7f893800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.355+0000 7f8936ffd640 1 -- 172.21.15.76:0/1708110064 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89380125f0 con 0x7f89400b3640 2024-03-08T06:02:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.355+0000 7f8936ffd640 1 -- 172.21.15.76:0/1708110064 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8938019070 con 0x7f89400b3640 2024-03-08T06:02:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.355+0000 7f894480e640 1 -- 172.21.15.76:0/1708110064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89400b0eb0 con 0x7f89400b3640 2024-03-08T06:02:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.355+0000 7f8936ffd640 1 -- 172.21.15.76:0/1708110064 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89380129d0 con 0x7f89400b3640 2024-03-08T06:02:38.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.355+0000 7f894480e640 1 -- 172.21.15.76:0/1708110064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894015b6e0 con 0x7f89400b3640 2024-03-08T06:02:38.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.356+0000 7f894480e640 1 -- 172.21.15.76:0/1708110064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f894004f030 con 0x7f89400b3640 2024-03-08T06:02:38.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.356+0000 7f8936ffd640 1 -- 172.21.15.76:0/1708110064 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8938012b70 con 0x7f89400b3640 2024-03-08T06:02:38.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.357+0000 7f8936ffd640 1 --2- 172.21.15.76:0/1708110064 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f892006b4d0 0x7f892006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.357+0000 7f8936ffd640 1 -- 172.21.15.76:0/1708110064 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f893808a7a0 con 0x7f89400b3640 2024-03-08T06:02:38.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.357+0000 7f893dd9b640 1 --2- 172.21.15.76:0/1708110064 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f892006b4d0 0x7f892006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.357+0000 7f893dd9b640 1 --2- 172.21.15.76:0/1708110064 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f892006b4d0 0x7f892006d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f894006e370 tx=0x7f89280079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:38.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.361+0000 7f8936ffd640 1 -- 172.21.15.76:0/1708110064 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8938059470 con 0x7f89400b3640 2024-03-08T06:02:38.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.533+0000 7f894480e640 1 -- 172.21.15.76:0/1708110064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f894006c300 con 0x7f89400b3640 2024-03-08T06:02:38.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.534+0000 7f8936ffd640 1 -- 172.21.15.76:0/1708110064 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8938058d70 con 0x7f89400b3640 2024-03-08T06:02:38.534 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:02:38.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.536+0000 7f894480e640 1 -- 172.21.15.76:0/1708110064 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f892006b4d0 msgr2=0x7f892006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.536+0000 7f894480e640 1 --2- 172.21.15.76:0/1708110064 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f892006b4d0 0x7f892006d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f894006e370 tx=0x7f89280079b0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.536+0000 7f894480e640 1 -- 172.21.15.76:0/1708110064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89400b3640 msgr2=0x7f89400b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.536+0000 7f894480e640 1 --2- 172.21.15.76:0/1708110064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89400b3640 0x7f89400b0970 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f893800d620 tx=0x7f893800dae0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.536+0000 7f894480e640 1 -- 172.21.15.76:0/1708110064 shutdown_connections 2024-03-08T06:02:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.536+0000 7f894480e640 1 --2- 172.21.15.76:0/1708110064 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f892006b4d0 0x7f892006d990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.536+0000 7f894480e640 1 --2- 172.21.15.76:0/1708110064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8940154df0 0x7f8940159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.536+0000 7f894480e640 1 --2- 172.21.15.76:0/1708110064 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f89400cc7e0 0x7f8940147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.536+0000 7f894480e640 1 --2- 172.21.15.76:0/1708110064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89400b3640 0x7f89400b0970 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.536+0000 7f894480e640 1 -- 172.21.15.76:0/1708110064 >> 172.21.15.76:0/1708110064 conn(0x7f894006ac80 msgr2=0x7f894006b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.536+0000 7f894480e640 1 -- 172.21.15.76:0/1708110064 shutdown_connections 2024-03-08T06:02:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.536+0000 7f894480e640 1 -- 172.21.15.76:0/1708110064 wait complete. 2024-03-08T06:02:38.547 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:02:38.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.649+0000 7f2d0d919640 1 Processor -- start 2024-03-08T06:02:38.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.649+0000 7f2d0d919640 1 -- start start 2024-03-08T06:02:38.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.650+0000 7f2d0d919640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2d08159770 0x7f2d0815f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.650+0000 7f2d0d919640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d08071d40 con 0x7f2d08071890 2024-03-08T06:02:38.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.650+0000 7f2d0d919640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0815fc10 con 0x7f2d0815c600 2024-03-08T06:02:38.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.650+0000 7f2d0d919640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0815c170 con 0x7f2d08159770 2024-03-08T06:02:38.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.650+0000 7f2d067fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2d08071890 0x7f2d080cf4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40392/0 (socket says 172.21.15.76:40392) 2024-03-08T06:02:38.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.650+0000 7f2d067fc640 1 -- 172.21.15.76:0/2343781370 learned_addr learned my addr 172.21.15.76:0/2343781370 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:38.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.650+0000 7f2d06ffd640 1 --2- 172.21.15.76:0/2343781370 >> v2:172.21.15.81:3300/0 conn(0x7f2d08159770 0x7f2d0815f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 -- 172.21.15.76:0/2343781370 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728527618 0 0) 0x7f2d08071d40 con 0x7f2d08071890 2024-03-08T06:02:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 -- 172.21.15.76:0/2343781370 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cf0003520 con 0x7f2d08071890 2024-03-08T06:02:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 -- 172.21.15.76:0/2343781370 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037012848 0 0) 0x7f2d0815fc10 con 0x7f2d0815c600 2024-03-08T06:02:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 -- 172.21.15.76:0/2343781370 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d08071d40 con 0x7f2d0815c600 2024-03-08T06:02:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 -- 172.21.15.76:0/2343781370 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 647644473 0 0) 0x7f2cf0003520 con 0x7f2d08071890 2024-03-08T06:02:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 -- 172.21.15.76:0/2343781370 >> v2:172.21.15.81:3300/0 conn(0x7f2d08159770 msgr2=0x7f2d0815f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 --2- 172.21.15.76:0/2343781370 >> v2:172.21.15.81:3300/0 conn(0x7f2d08159770 0x7f2d0815f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 -- 172.21.15.76:0/2343781370 >> v1:172.21.15.81:6789/0 conn(0x7f2d0815c600 legacy=0x7f2d080ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 -- 172.21.15.76:0/2343781370 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0815fef0 con 0x7f2d08071890 2024-03-08T06:02:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 -- 172.21.15.76:0/2343781370 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f2cfc001670 con 0x7f2d08071890 2024-03-08T06:02:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d06ffd640 1 --2- 172.21.15.76:0/2343781370 >> v2:172.21.15.81:3300/0 conn(0x7f2d08159770 0x7f2d0815f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:02:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 -- 172.21.15.76:0/2343781370 >> v1:172.21.15.76:6789/0 conn(0x7f2d08071890 legacy=0x7f2d080cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 --2- 172.21.15.76:0/2343781370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2cf0003bb0 0x7f2cf0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 --2- 172.21.15.76:0/2343781370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cf0004800 0x7f2cf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 --2- 172.21.15.76:0/2343781370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cf00051c0 0x7f2cf000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 -- 172.21.15.76:0/2343781370 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0815c170 con 0x7f2cf0004800 2024-03-08T06:02:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 -- 172.21.15.76:0/2343781370 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf000c0c0 con 0x7f2cf00051c0 2024-03-08T06:02:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d05ffb640 1 -- 172.21.15.76:0/2343781370 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf000c2c0 con 0x7f2cf0003bb0 2024-03-08T06:02:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d067fc640 1 --2- 172.21.15.76:0/2343781370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cf0004800 0x7f2cf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d06ffd640 1 --2- 172.21.15.76:0/2343781370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2cf0003bb0 0x7f2cf0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d077fe640 1 --2- 172.21.15.76:0/2343781370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cf00051c0 0x7f2cf000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d067fc640 1 -- 172.21.15.76:0/2343781370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2cf0003bb0 msgr2=0x7f2cf0003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d067fc640 1 --2- 172.21.15.76:0/2343781370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2cf0003bb0 0x7f2cf0003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d067fc640 1 -- 172.21.15.76:0/2343781370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cf00051c0 msgr2=0x7f2cf000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d067fc640 1 --2- 172.21.15.76:0/2343781370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cf00051c0 0x7f2cf000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d067fc640 1 -- 172.21.15.76:0/2343781370 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf0003520 con 0x7f2cf0004800 2024-03-08T06:02:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d077fe640 1 --2- 172.21.15.76:0/2343781370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cf00051c0 0x7f2cf000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.651+0000 7f2d077fe640 1 -- 172.21.15.76:0/2343781370 reap_dead start 2024-03-08T06:02:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.652+0000 7f2d067fc640 1 --2- 172.21.15.76:0/2343781370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cf0004800 0x7f2cf0004c80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00c190 tx=0x7f2cf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a5957d095e5ef server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.652+0000 7f2d05ffb640 1 -- 172.21.15.76:0/2343781370 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc007ce0 con 0x7f2cf0004800 2024-03-08T06:02:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.652+0000 7f2d05ffb640 1 -- 172.21.15.76:0/2343781370 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cfc030070 con 0x7f2cf0004800 2024-03-08T06:02:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.652+0000 7f2d05ffb640 1 -- 172.21.15.76:0/2343781370 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc005d90 con 0x7f2cf0004800 2024-03-08T06:02:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.653+0000 7f2d0d919640 1 -- 172.21.15.76:0/2343781370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cf0004800 msgr2=0x7f2cf0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.653+0000 7f2d0d919640 1 --2- 172.21.15.76:0/2343781370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cf0004800 0x7f2cf0004c80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00c190 tx=0x7f2cf0005830 comp rx=0 tx=0).stop 2024-03-08T06:02:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.653+0000 7f2d0d919640 1 -- 172.21.15.76:0/2343781370 shutdown_connections 2024-03-08T06:02:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.653+0000 7f2d0d919640 1 --2- 172.21.15.76:0/2343781370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cf0004800 0x7f2cf0004c80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.653+0000 7f2d0d919640 1 -- 172.21.15.76:0/2343781370 >> 172.21.15.76:0/2343781370 conn(0x7f2d0806f530 msgr2=0x7f2d0806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.653+0000 7f2d0d919640 1 -- 172.21.15.76:0/2343781370 shutdown_connections 2024-03-08T06:02:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.653+0000 7f2d0d919640 1 -- 172.21.15.76:0/2343781370 wait complete. 2024-03-08T06:02:38.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.654+0000 7f2d0d919640 1 Processor -- start 2024-03-08T06:02:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.654+0000 7f2d0d919640 1 -- start start 2024-03-08T06:02:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d0d919640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cf0004800 0x7f2d081516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d0d919640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2d0814d440 0x7f2d0815dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d0d919640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d0814d8c0 0x7f2d0815e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d0d919640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0815f5c0 con 0x7f2cf0004800 2024-03-08T06:02:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d0d919640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d081604b0 con 0x7f2d0814d8c0 2024-03-08T06:02:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d0d919640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0815f740 con 0x7f2d0814d440 2024-03-08T06:02:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d06ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cf0004800 0x7f2d081516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d06ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cf0004800 0x7f2d081516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36926/0 (socket says 172.21.15.76:36926) 2024-03-08T06:02:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d06ffd640 1 -- 172.21.15.76:0/2027931368 learned_addr learned my addr 172.21.15.76:0/2027931368 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d067fc640 1 --2- 172.21.15.76:0/2027931368 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2d0814d440 0x7f2d0815dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d077fe640 1 --2- 172.21.15.76:0/2027931368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d0814d8c0 0x7f2d0815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d06ffd640 1 -- 172.21.15.76:0/2027931368 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2d0814d440 msgr2=0x7f2d0815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d06ffd640 1 --2- 172.21.15.76:0/2027931368 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2d0814d440 0x7f2d0815dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d06ffd640 1 -- 172.21.15.76:0/2027931368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d0814d8c0 msgr2=0x7f2d0815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d06ffd640 1 --2- 172.21.15.76:0/2027931368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d0814d8c0 0x7f2d0815e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d06ffd640 1 -- 172.21.15.76:0/2027931368 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080b6950 con 0x7f2cf0004800 2024-03-08T06:02:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d06ffd640 1 --2- 172.21.15.76:0/2027931368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cf0004800 0x7f2d081516f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800f430 tx=0x7f2cf800f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2ceffff640 1 -- 172.21.15.76:0/2027931368 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8003b90 con 0x7f2cf0004800 2024-03-08T06:02:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2ceffff640 1 -- 172.21.15.76:0/2027931368 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf8003d50 con 0x7f2cf0004800 2024-03-08T06:02:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2ceffff640 1 -- 172.21.15.76:0/2027931368 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8004270 con 0x7f2cf0004800 2024-03-08T06:02:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.655+0000 7f2d0d919640 1 -- 172.21.15.76:0/2027931368 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d080b6bf0 con 0x7f2cf0004800 2024-03-08T06:02:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.656+0000 7f2d0d919640 1 -- 172.21.15.76:0/2027931368 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d080b71b0 con 0x7f2cf0004800 2024-03-08T06:02:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.657+0000 7f2d0d919640 1 -- 172.21.15.76:0/2027931368 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0804f030 con 0x7f2cf0004800 2024-03-08T06:02:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.657+0000 7f2ceffff640 1 -- 172.21.15.76:0/2027931368 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2cf8004410 con 0x7f2cf0004800 2024-03-08T06:02:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.657+0000 7f2ceffff640 1 --2- 172.21.15.76:0/2027931368 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2cf406b4d0 0x7f2cf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.657+0000 7f2ceffff640 1 -- 172.21.15.76:0/2027931368 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f2cf808b1f0 con 0x7f2cf0004800 2024-03-08T06:02:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.657+0000 7f2d067fc640 1 --2- 172.21.15.76:0/2027931368 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2cf406b4d0 0x7f2cf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.658+0000 7f2d067fc640 1 --2- 172.21.15.76:0/2027931368 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2cf406b4d0 0x7f2cf406d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc034000 tx=0x7f2cfc007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:38.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.662+0000 7f2ceffff640 1 -- 172.21.15.76:0/2027931368 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2cf8019070 con 0x7f2cf0004800 2024-03-08T06:02:38.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.831+0000 7f2d0d919640 1 -- 172.21.15.76:0/2027931368 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d080703a0 con 0x7f2cf0004800 2024-03-08T06:02:38.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.832+0000 7f2ceffff640 1 -- 172.21.15.76:0/2027931368 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2cf8010040 con 0x7f2cf0004800 2024-03-08T06:02:38.832 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:02:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.834+0000 7f2d0d919640 1 -- 172.21.15.76:0/2027931368 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2cf406b4d0 msgr2=0x7f2cf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.834+0000 7f2d0d919640 1 --2- 172.21.15.76:0/2027931368 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2cf406b4d0 0x7f2cf406d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc034000 tx=0x7f2cfc007000 comp rx=0 tx=0).stop 2024-03-08T06:02:38.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.834+0000 7f2d0d919640 1 -- 172.21.15.76:0/2027931368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cf0004800 msgr2=0x7f2d081516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.834+0000 7f2d0d919640 1 --2- 172.21.15.76:0/2027931368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cf0004800 0x7f2d081516f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800f430 tx=0x7f2cf800f830 comp rx=0 tx=0).stop 2024-03-08T06:02:38.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.834+0000 7f2d0d919640 1 -- 172.21.15.76:0/2027931368 shutdown_connections 2024-03-08T06:02:38.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.834+0000 7f2d0d919640 1 --2- 172.21.15.76:0/2027931368 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f2cf406b4d0 0x7f2cf406d990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.834+0000 7f2d0d919640 1 --2- 172.21.15.76:0/2027931368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d0814d8c0 0x7f2d0815e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.834+0000 7f2d0d919640 1 --2- 172.21.15.76:0/2027931368 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2d0814d440 0x7f2d0815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.834+0000 7f2d0d919640 1 --2- 172.21.15.76:0/2027931368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cf0004800 0x7f2d081516f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.834+0000 7f2d0d919640 1 -- 172.21.15.76:0/2027931368 >> 172.21.15.76:0/2027931368 conn(0x7f2d0806f530 msgr2=0x7f2d080d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:38.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.834+0000 7f2d0d919640 1 -- 172.21.15.76:0/2027931368 shutdown_connections 2024-03-08T06:02:38.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.834+0000 7f2d0d919640 1 -- 172.21.15.76:0/2027931368 wait complete. 2024-03-08T06:02:38.845 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:02:38.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.948+0000 7f28da1c9640 1 Processor -- start 2024-03-08T06:02:38.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.948+0000 7f28da1c9640 1 -- start start 2024-03-08T06:02:38.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.948+0000 7f28da1c9640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28d40ce5e0 0x7f28d406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.948+0000 7f28da1c9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f28d415dfb0 0x7f28d415e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.948+0000 7f28da1c9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d4161840 con 0x7f28d40ce9c0 2024-03-08T06:02:38.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.948+0000 7f28da1c9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d4161b90 con 0x7f28d415dfb0 2024-03-08T06:02:38.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.948+0000 7f28da1c9640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d415eb30 con 0x7f28d40ce5e0 2024-03-08T06:02:38.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.949+0000 7f28d2ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f28d40ce9c0 0x7f28d4162020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40396/0 (socket says 172.21.15.76:40396) 2024-03-08T06:02:38.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.949+0000 7f28d2ffd640 1 -- 172.21.15.76:0/471279142 learned_addr learned my addr 172.21.15.76:0/471279142 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:38.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.949+0000 7f28d37fe640 1 --2- 172.21.15.76:0/471279142 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28d40ce5e0 0x7f28d406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.949+0000 7f28d3fff640 1 --2- 172.21.15.76:0/471279142 >> v2:172.21.15.81:3300/0 conn(0x7f28d415dfb0 0x7f28d415e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.949+0000 7f28d27fc640 1 -- 172.21.15.76:0/471279142 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159916535 0 0) 0x7f28d4161840 con 0x7f28d40ce9c0 2024-03-08T06:02:38.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.949+0000 7f28d27fc640 1 -- 172.21.15.76:0/471279142 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28bc003520 con 0x7f28d40ce9c0 2024-03-08T06:02:38.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.949+0000 7f28d27fc640 1 -- 172.21.15.76:0/471279142 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 778430136 0 0) 0x7f28bc003520 con 0x7f28d40ce9c0 2024-03-08T06:02:38.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.949+0000 7f28d37fe640 1 -- 172.21.15.76:0/471279142 >> v2:172.21.15.81:3300/0 conn(0x7f28d415dfb0 msgr2=0x7f28d415e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:38.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.949+0000 7f28d37fe640 1 --2- 172.21.15.76:0/471279142 >> v2:172.21.15.81:3300/0 conn(0x7f28d415dfb0 0x7f28d415e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.949+0000 7f28d37fe640 1 -- 172.21.15.76:0/471279142 >> v1:172.21.15.76:6789/0 conn(0x7f28d40ce9c0 legacy=0x7f28d4162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.949+0000 7f28d37fe640 1 -- 172.21.15.76:0/471279142 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d415ee10 con 0x7f28d40ce5e0 2024-03-08T06:02:38.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.950+0000 7f28d37fe640 1 --2- 172.21.15.76:0/471279142 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28d40ce5e0 0x7f28d406d8a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f28c40097a0 tx=0x7f28c402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5ecfd89dc42b649 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:38.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.950+0000 7f28d27fc640 1 -- 172.21.15.76:0/471279142 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4038070 con 0x7f28d40ce5e0 2024-03-08T06:02:38.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.950+0000 7f28d27fc640 1 -- 172.21.15.76:0/471279142 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c4031500 con 0x7f28d40ce5e0 2024-03-08T06:02:38.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.950+0000 7f28d27fc640 1 -- 172.21.15.76:0/471279142 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c8001670 con 0x7f28d40ce5e0 2024-03-08T06:02:38.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.951+0000 7f28da1c9640 1 -- 172.21.15.76:0/471279142 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28d40ce5e0 msgr2=0x7f28d406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.951+0000 7f28da1c9640 1 --2- 172.21.15.76:0/471279142 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28d40ce5e0 0x7f28d406d8a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f28c40097a0 tx=0x7f28c402b820 comp rx=0 tx=0).stop 2024-03-08T06:02:38.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.951+0000 7f28da1c9640 1 -- 172.21.15.76:0/471279142 shutdown_connections 2024-03-08T06:02:38.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.951+0000 7f28da1c9640 1 --2- 172.21.15.76:0/471279142 >> v2:172.21.15.81:3300/0 conn(0x7f28d415dfb0 0x7f28d415e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.951+0000 7f28da1c9640 1 --2- 172.21.15.76:0/471279142 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28d40ce5e0 0x7f28d406d8a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.951+0000 7f28da1c9640 1 -- 172.21.15.76:0/471279142 >> 172.21.15.76:0/471279142 conn(0x7f28d406ae60 msgr2=0x7f28d406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:38.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.951+0000 7f28da1c9640 1 -- 172.21.15.76:0/471279142 shutdown_connections 2024-03-08T06:02:38.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.951+0000 7f28da1c9640 1 -- 172.21.15.76:0/471279142 wait complete. 2024-03-08T06:02:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.952+0000 7f28da1c9640 1 Processor -- start 2024-03-08T06:02:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.952+0000 7f28da1c9640 1 -- start start 2024-03-08T06:02:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.953+0000 7f28da1c9640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28d40ce5e0 0x7f28d40c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.953+0000 7f28da1c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28d40ce9c0 0x7f28d41e92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.953+0000 7f28da1c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28d415dfb0 0x7f28d40b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.953+0000 7f28da1c9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d40cee20 con 0x7f28d415dfb0 2024-03-08T06:02:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.953+0000 7f28da1c9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d4162730 con 0x7f28d40ce9c0 2024-03-08T06:02:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.953+0000 7f28da1c9640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d406bfa0 con 0x7f28d40ce5e0 2024-03-08T06:02:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.953+0000 7f28d3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28d415dfb0 0x7f28d40b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.953+0000 7f28d3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28d415dfb0 0x7f28d40b68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36938/0 (socket says 172.21.15.76:36938) 2024-03-08T06:02:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.953+0000 7f28d3fff640 1 -- 172.21.15.76:0/1867196711 learned_addr learned my addr 172.21.15.76:0/1867196711 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.953+0000 7f28d37fe640 1 --2- 172.21.15.76:0/1867196711 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28d40ce5e0 0x7f28d40c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.954+0000 7f28d2ffd640 1 --2- 172.21.15.76:0/1867196711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28d40ce9c0 0x7f28d41e92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.954+0000 7f28d3fff640 1 -- 172.21.15.76:0/1867196711 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28d40ce5e0 msgr2=0x7f28d40c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.954+0000 7f28d3fff640 1 --2- 172.21.15.76:0/1867196711 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28d40ce5e0 0x7f28d40c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.954+0000 7f28d3fff640 1 -- 172.21.15.76:0/1867196711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28d40ce9c0 msgr2=0x7f28d41e92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.954+0000 7f28d3fff640 1 --2- 172.21.15.76:0/1867196711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28d40ce9c0 0x7f28d41e92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.954+0000 7f28d3fff640 1 -- 172.21.15.76:0/1867196711 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d41e8a90 con 0x7f28d415dfb0 2024-03-08T06:02:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.954+0000 7f28d37fe640 1 --2- 172.21.15.76:0/1867196711 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28d40ce5e0 0x7f28d40c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.954+0000 7f28d2ffd640 1 --2- 172.21.15.76:0/1867196711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28d40ce9c0 0x7f28d41e92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.954+0000 7f28d3fff640 1 --2- 172.21.15.76:0/1867196711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28d415dfb0 0x7f28d40b68f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00efc0 tx=0x7f28cc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.954+0000 7f28bbfff640 1 -- 172.21.15.76:0/1867196711 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc018020 con 0x7f28d415dfb0 2024-03-08T06:02:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.954+0000 7f28bbfff640 1 -- 172.21.15.76:0/1867196711 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28cc014480 con 0x7f28d415dfb0 2024-03-08T06:02:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.954+0000 7f28da1c9640 1 -- 172.21.15.76:0/1867196711 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d41e84d0 con 0x7f28d415dfb0 2024-03-08T06:02:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.954+0000 7f28bbfff640 1 -- 172.21.15.76:0/1867196711 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc014870 con 0x7f28d415dfb0 2024-03-08T06:02:38.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.954+0000 7f28da1c9640 1 -- 172.21.15.76:0/1867196711 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d40b45f0 con 0x7f28d415dfb0 2024-03-08T06:02:38.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.955+0000 7f28da1c9640 1 -- 172.21.15.76:0/1867196711 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d4161840 con 0x7f28d415dfb0 2024-03-08T06:02:38.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.955+0000 7f28bbfff640 1 -- 172.21.15.76:0/1867196711 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28cc005d10 con 0x7f28d415dfb0 2024-03-08T06:02:38.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.956+0000 7f28bbfff640 1 --2- 172.21.15.76:0/1867196711 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f28c006b4d0 0x7f28c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:38.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.956+0000 7f28d37fe640 1 --2- 172.21.15.76:0/1867196711 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f28c006b4d0 0x7f28c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:38.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.956+0000 7f28bbfff640 1 -- 172.21.15.76:0/1867196711 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f28cc08ae00 con 0x7f28d415dfb0 2024-03-08T06:02:38.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.956+0000 7f28d37fe640 1 --2- 172.21.15.76:0/1867196711 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f28c006b4d0 0x7f28c006d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f28c4035000 tx=0x7f28c4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:38.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:38.960+0000 7f28bbfff640 1 -- 172.21.15.76:0/1867196711 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f28cc022070 con 0x7f28d415dfb0 2024-03-08T06:02:39.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.132+0000 7f28da1c9640 1 -- 172.21.15.76:0/1867196711 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28d40d0740 con 0x7f28d415dfb0 2024-03-08T06:02:39.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.132+0000 7f28bbfff640 1 -- 172.21.15.76:0/1867196711 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f28cc010040 con 0x7f28d415dfb0 2024-03-08T06:02:39.133 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:02:39.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.134+0000 7f28da1c9640 1 -- 172.21.15.76:0/1867196711 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f28c006b4d0 msgr2=0x7f28c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:39.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.134+0000 7f28da1c9640 1 --2- 172.21.15.76:0/1867196711 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f28c006b4d0 0x7f28c006d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f28c4035000 tx=0x7f28c4008030 comp rx=0 tx=0).stop 2024-03-08T06:02:39.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.134+0000 7f28da1c9640 1 -- 172.21.15.76:0/1867196711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28d415dfb0 msgr2=0x7f28d40b68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:39.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.134+0000 7f28da1c9640 1 --2- 172.21.15.76:0/1867196711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28d415dfb0 0x7f28d40b68f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00efc0 tx=0x7f28cc00c490 comp rx=0 tx=0).stop 2024-03-08T06:02:39.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.134+0000 7f28da1c9640 1 -- 172.21.15.76:0/1867196711 shutdown_connections 2024-03-08T06:02:39.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.134+0000 7f28da1c9640 1 --2- 172.21.15.76:0/1867196711 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f28c006b4d0 0x7f28c006d990 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.134+0000 7f28da1c9640 1 --2- 172.21.15.76:0/1867196711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28d415dfb0 0x7f28d40b68f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.134+0000 7f28da1c9640 1 --2- 172.21.15.76:0/1867196711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28d40ce9c0 0x7f28d41e92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.134+0000 7f28da1c9640 1 --2- 172.21.15.76:0/1867196711 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28d40ce5e0 0x7f28d40c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.134+0000 7f28da1c9640 1 -- 172.21.15.76:0/1867196711 >> 172.21.15.76:0/1867196711 conn(0x7f28d406ae60 msgr2=0x7f28d415bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:39.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.134+0000 7f28da1c9640 1 -- 172.21.15.76:0/1867196711 shutdown_connections 2024-03-08T06:02:39.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.134+0000 7f28da1c9640 1 -- 172.21.15.76:0/1867196711 wait complete. 2024-03-08T06:02:39.145 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T06:02:39.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.245+0000 7f997dc07640 1 Processor -- start 2024-03-08T06:02:39.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.245+0000 7f997dc07640 1 -- start start 2024-03-08T06:02:39.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f997dc07640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9978154df0 0x7f99780b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:39.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f997dc07640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f99780b3640 0x7f99780cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:39.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f997dc07640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997815f450 con 0x7f99780cc610 2024-03-08T06:02:39.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f997dc07640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99780af1e0 con 0x7f99780b3640 2024-03-08T06:02:39.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f997dc07640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99780b3c80 con 0x7f9978154df0 2024-03-08T06:02:39.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f9977fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f99780cc610 0x7f9978071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40408/0 (socket says 172.21.15.76:40408) 2024-03-08T06:02:39.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f9977fff640 1 -- 172.21.15.76:0/1143829632 learned_addr learned my addr 172.21.15.76:0/1143829632 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:39.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f99777fe640 1 --2- 172.21.15.76:0/1143829632 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9978154df0 0x7f99780b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:39.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f99767fc640 1 -- 172.21.15.76:0/1143829632 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847784692 0 0) 0x7f997815f450 con 0x7f99780cc610 2024-03-08T06:02:39.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f9976ffd640 1 --2- 172.21.15.76:0/1143829632 >> v2:172.21.15.81:3300/0 conn(0x7f99780b3640 0x7f99780cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:39.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f99767fc640 1 -- 172.21.15.76:0/1143829632 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9960003520 con 0x7f99780cc610 2024-03-08T06:02:39.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f99767fc640 1 -- 172.21.15.76:0/1143829632 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2898834196 0 0) 0x7f9960003520 con 0x7f99780cc610 2024-03-08T06:02:39.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f99767fc640 1 -- 172.21.15.76:0/1143829632 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9978154df0 msgr2=0x7f99780b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:39.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f99767fc640 1 --2- 172.21.15.76:0/1143829632 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9978154df0 0x7f99780b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f99767fc640 1 -- 172.21.15.76:0/1143829632 >> v2:172.21.15.81:3300/0 conn(0x7f99780b3640 msgr2=0x7f99780cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:39.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f99767fc640 1 --2- 172.21.15.76:0/1143829632 >> v2:172.21.15.81:3300/0 conn(0x7f99780b3640 0x7f99780cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f99767fc640 1 -- 172.21.15.76:0/1143829632 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99780cca30 con 0x7f99780cc610 2024-03-08T06:02:39.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.246+0000 7f99767fc640 1 -- 172.21.15.76:0/1143829632 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f99700019f0 con 0x7f99780cc610 2024-03-08T06:02:39.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.247+0000 7f99767fc640 1 -- 172.21.15.76:0/1143829632 >> v1:172.21.15.76:6789/0 conn(0x7f99780cc610 legacy=0x7f9978071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:39.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.247+0000 7f99767fc640 1 --2- 172.21.15.76:0/1143829632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9960003a20 0x7f9960003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:39.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.247+0000 7f99767fc640 1 --2- 172.21.15.76:0/1143829632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9960004800 0x7f9960004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:39.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.247+0000 7f99767fc640 1 --2- 172.21.15.76:0/1143829632 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9978154df0 0x7f99600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:39.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.247+0000 7f99767fc640 1 -- 172.21.15.76:0/1143829632 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99780af1e0 con 0x7f9960004800 2024-03-08T06:02:39.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.247+0000 7f99767fc640 1 -- 172.21.15.76:0/1143829632 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99780b3c80 con 0x7f9960003a20 2024-03-08T06:02:39.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.247+0000 7f99767fc640 1 -- 172.21.15.76:0/1143829632 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996000ba40 con 0x7f9978154df0 2024-03-08T06:02:39.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.247+0000 7f9976ffd640 1 --2- 172.21.15.76:0/1143829632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9960004800 0x7f9960004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:39.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.247+0000 7f99777fe640 1 --2- 172.21.15.76:0/1143829632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9960003a20 0x7f9960003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:39.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.247+0000 7f9977fff640 1 --2- 172.21.15.76:0/1143829632 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9978154df0 0x7f99600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:39.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.247+0000 7f9976ffd640 1 -- 172.21.15.76:0/1143829632 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9978154df0 msgr2=0x7f99600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:39.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.247+0000 7f9976ffd640 1 --2- 172.21.15.76:0/1143829632 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9978154df0 0x7f99600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.247+0000 7f9976ffd640 1 -- 172.21.15.76:0/1143829632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9960003a20 msgr2=0x7f9960003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:39.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.247+0000 7f9976ffd640 1 --2- 172.21.15.76:0/1143829632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9960003a20 0x7f9960003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.247+0000 7f9976ffd640 1 -- 172.21.15.76:0/1143829632 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9960003520 con 0x7f9960004800 2024-03-08T06:02:39.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.248+0000 7f9976ffd640 1 --2- 172.21.15.76:0/1143829632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9960004800 0x7f9960004c80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f996c009ac0 tx=0x7f9960005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0274a7f06c17f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:39.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.248+0000 7f99767fc640 1 -- 172.21.15.76:0/1143829632 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c039070 con 0x7f9960004800 2024-03-08T06:02:39.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.248+0000 7f99767fc640 1 -- 172.21.15.76:0/1143829632 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c02c710 con 0x7f9960004800 2024-03-08T06:02:39.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.248+0000 7f99767fc640 1 -- 172.21.15.76:0/1143829632 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c009690 con 0x7f9960004800 2024-03-08T06:02:39.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.248+0000 7f997dc07640 1 -- 172.21.15.76:0/1143829632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9960004800 msgr2=0x7f9960004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:39.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.248+0000 7f997dc07640 1 --2- 172.21.15.76:0/1143829632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9960004800 0x7f9960004c80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f996c009ac0 tx=0x7f9960005830 comp rx=0 tx=0).stop 2024-03-08T06:02:39.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.248+0000 7f9977fff640 1 -- 172.21.15.76:0/1143829632 reap_dead start 2024-03-08T06:02:39.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.248+0000 7f997dc07640 1 -- 172.21.15.76:0/1143829632 shutdown_connections 2024-03-08T06:02:39.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.248+0000 7f997dc07640 1 -- 172.21.15.76:0/1143829632 >> 172.21.15.76:0/1143829632 conn(0x7f997806ac80 msgr2=0x7f997806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:39.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.248+0000 7f997dc07640 1 -- 172.21.15.76:0/1143829632 shutdown_connections 2024-03-08T06:02:39.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.248+0000 7f997dc07640 1 -- 172.21.15.76:0/1143829632 wait complete. 2024-03-08T06:02:39.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.250+0000 7f997dc07640 1 Processor -- start 2024-03-08T06:02:39.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.250+0000 7f997dc07640 1 -- start start 2024-03-08T06:02:39.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.250+0000 7f997dc07640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9960004800 0x7f997814f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:39.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.250+0000 7f997dc07640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9978159cd0 0x7f9978156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:39.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.250+0000 7f997dc07640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9978157390 0x7f99780ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:39.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.250+0000 7f997dc07640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997815ec50 con 0x7f9960004800 2024-03-08T06:02:39.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.250+0000 7f997dc07640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997815e970 con 0x7f9978157390 2024-03-08T06:02:39.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.250+0000 7f997dc07640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997815ef30 con 0x7f9978159cd0 2024-03-08T06:02:39.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f99777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9960004800 0x7f997814f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:39.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f99777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9960004800 0x7f997814f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36950/0 (socket says 172.21.15.76:36950) 2024-03-08T06:02:39.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f99777fe640 1 -- 172.21.15.76:0/2992673679 learned_addr learned my addr 172.21.15.76:0/2992673679 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:39.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f9976ffd640 1 --2- 172.21.15.76:0/2992673679 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9978159cd0 0x7f9978156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:39.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f99777fe640 1 -- 172.21.15.76:0/2992673679 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9978159cd0 msgr2=0x7f9978156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:39.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f9977fff640 1 --2- 172.21.15.76:0/2992673679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9978157390 0x7f99780ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:39.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f99777fe640 1 --2- 172.21.15.76:0/2992673679 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9978159cd0 0x7f9978156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f99777fe640 1 -- 172.21.15.76:0/2992673679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9978157390 msgr2=0x7f99780ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:39.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f99777fe640 1 --2- 172.21.15.76:0/2992673679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9978157390 0x7f99780ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f99777fe640 1 -- 172.21.15.76:0/2992673679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997814f800 con 0x7f9960004800 2024-03-08T06:02:39.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f9976ffd640 1 --2- 172.21.15.76:0/2992673679 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9978159cd0 0x7f9978156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:39.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f99777fe640 1 --2- 172.21.15.76:0/2992673679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9960004800 0x7f997814f2c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9968011910 tx=0x7f9968011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:39.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f995ffff640 1 -- 172.21.15.76:0/2992673679 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996800faf0 con 0x7f9960004800 2024-03-08T06:02:39.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f995ffff640 1 -- 172.21.15.76:0/2992673679 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996800fc90 con 0x7f9960004800 2024-03-08T06:02:39.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f995ffff640 1 -- 172.21.15.76:0/2992673679 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968014440 con 0x7f9960004800 2024-03-08T06:02:39.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f997dc07640 1 -- 172.21.15.76:0/2992673679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9978158c30 con 0x7f9960004800 2024-03-08T06:02:39.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.251+0000 7f997dc07640 1 -- 172.21.15.76:0/2992673679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99781591f0 con 0x7f9960004800 2024-03-08T06:02:39.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.252+0000 7f997dc07640 1 -- 172.21.15.76:0/2992673679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997804f030 con 0x7f9960004800 2024-03-08T06:02:39.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.253+0000 7f995ffff640 1 -- 172.21.15.76:0/2992673679 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9968012040 con 0x7f9960004800 2024-03-08T06:02:39.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.253+0000 7f995ffff640 1 --2- 172.21.15.76:0/2992673679 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f996406b4d0 0x7f996406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:39.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.253+0000 7f995ffff640 1 -- 172.21.15.76:0/2992673679 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f996808a8e0 con 0x7f9960004800 2024-03-08T06:02:39.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.253+0000 7f9976ffd640 1 --2- 172.21.15.76:0/2992673679 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f996406b4d0 0x7f996406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:39.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.253+0000 7f9976ffd640 1 --2- 172.21.15.76:0/2992673679 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f996406b4d0 0x7f996406d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f996c036000 tx=0x7f996c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:39.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.257+0000 7f995ffff640 1 -- 172.21.15.76:0/2992673679 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f997814f800 con 0x7f9960004800 2024-03-08T06:02:39.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.438+0000 7f997dc07640 1 -- 172.21.15.76:0/2992673679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f99780cd9a0 con 0x7f9960004800 2024-03-08T06:02:39.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.438+0000 7f995ffff640 1 -- 172.21.15.76:0/2992673679 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9968019030 con 0x7f9960004800 2024-03-08T06:02:39.439 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-03-08T06:02:39.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.440+0000 7f997dc07640 1 -- 172.21.15.76:0/2992673679 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f996406b4d0 msgr2=0x7f996406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:39.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.440+0000 7f997dc07640 1 --2- 172.21.15.76:0/2992673679 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f996406b4d0 0x7f996406d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f996c036000 tx=0x7f996c030000 comp rx=0 tx=0).stop 2024-03-08T06:02:39.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.440+0000 7f997dc07640 1 -- 172.21.15.76:0/2992673679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9960004800 msgr2=0x7f997814f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:39.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.440+0000 7f997dc07640 1 --2- 172.21.15.76:0/2992673679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9960004800 0x7f997814f2c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9968011910 tx=0x7f9968011dd0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.440+0000 7f997dc07640 1 -- 172.21.15.76:0/2992673679 shutdown_connections 2024-03-08T06:02:39.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.440+0000 7f997dc07640 1 --2- 172.21.15.76:0/2992673679 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f996406b4d0 0x7f996406d990 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.440+0000 7f997dc07640 1 --2- 172.21.15.76:0/2992673679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9978157390 0x7f99780ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.440+0000 7f997dc07640 1 --2- 172.21.15.76:0/2992673679 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9978159cd0 0x7f9978156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.440+0000 7f997dc07640 1 --2- 172.21.15.76:0/2992673679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9960004800 0x7f997814f2c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.440+0000 7f997dc07640 1 -- 172.21.15.76:0/2992673679 >> 172.21.15.76:0/2992673679 conn(0x7f997806ac80 msgr2=0x7f99780af360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:39.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.440+0000 7f997dc07640 1 -- 172.21.15.76:0/2992673679 shutdown_connections 2024-03-08T06:02:39.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.440+0000 7f997dc07640 1 -- 172.21.15.76:0/2992673679 wait complete. 2024-03-08T06:02:39.451 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-08T06:02:39.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:39.452 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-08T06:02:39.806 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-03-08T06:02:39.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:39.806 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-08T06:02:39.821 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:02:39.821 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:02:39.836 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:02:39.836 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:02:39.891 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:02:39.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.992+0000 7f76563f4640 1 Processor -- start 2024-03-08T06:02:39.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.992+0000 7f76563f4640 1 -- start start 2024-03-08T06:02:39.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.992+0000 7f76563f4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f765015bb40 0x7f76500b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.992+0000 7f76563f4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f76500cf5b0 0x7f76500ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.992+0000 7f76563f4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76500b5000 con 0x7f765015bf20 2024-03-08T06:02:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.992+0000 7f76563f4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76500cfb50 con 0x7f76500cf5b0 2024-03-08T06:02:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.992+0000 7f76563f4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76500cfe30 con 0x7f765015bb40 2024-03-08T06:02:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.993+0000 7f7654991640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f76500cf5b0 0x7f76500ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.993+0000 7f764f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f765015bf20 0x7f76500b48f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40422/0 (socket says 172.21.15.76:40422) 2024-03-08T06:02:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.993+0000 7f7654991640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f76500cf5b0 0x7f76500ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36956/0 (socket says 172.21.15.76:36956) 2024-03-08T06:02:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.993+0000 7f764f7fe640 1 -- 172.21.15.76:0/1566008427 learned_addr learned my addr 172.21.15.76:0/1566008427 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.993+0000 7f764ffff640 1 --2- 172.21.15.76:0/1566008427 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f765015bb40 0x7f76500b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.993+0000 7f7654991640 1 -- 172.21.15.76:0/1566008427 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f765015bb40 msgr2=0x7f76500b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.993+0000 7f7654991640 1 --2- 172.21.15.76:0/1566008427 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f765015bb40 0x7f76500b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.993+0000 7f7654991640 1 -- 172.21.15.76:0/1566008427 >> v1:172.21.15.76:6789/0 conn(0x7f765015bf20 legacy=0x7f76500b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.993+0000 7f7654991640 1 -- 172.21.15.76:0/1566008427 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76500d0110 con 0x7f76500cf5b0 2024-03-08T06:02:39.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.993+0000 7f764effd640 1 -- 172.21.15.76:0/1566008427 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561990752 0 0) 0x7f76500b5000 con 0x7f765015bf20 2024-03-08T06:02:39.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.994+0000 7f7654991640 1 --2- 172.21.15.76:0/1566008427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76500cf5b0 0x7f76500ce3b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7648009d90 tx=0x7f764802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad3fe7edc833d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:39.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.994+0000 7f764effd640 1 -- 172.21.15.76:0/1566008427 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648039070 con 0x7f76500cf5b0 2024-03-08T06:02:39.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.994+0000 7f764effd640 1 -- 172.21.15.76:0/1566008427 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764802cc50 con 0x7f76500cf5b0 2024-03-08T06:02:39.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.994+0000 7f764effd640 1 -- 172.21.15.76:0/1566008427 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648034440 con 0x7f76500cf5b0 2024-03-08T06:02:39.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.994+0000 7f76563f4640 1 -- 172.21.15.76:0/1566008427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76500cf5b0 msgr2=0x7f76500ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:39.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.994+0000 7f76563f4640 1 --2- 172.21.15.76:0/1566008427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76500cf5b0 0x7f76500ce3b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7648009d90 tx=0x7f764802bea0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.994+0000 7f76563f4640 1 -- 172.21.15.76:0/1566008427 shutdown_connections 2024-03-08T06:02:39.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.994+0000 7f76563f4640 1 --2- 172.21.15.76:0/1566008427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76500cf5b0 0x7f76500ce3b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.994+0000 7f76563f4640 1 --2- 172.21.15.76:0/1566008427 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f765015bb40 0x7f76500b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:39.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.994+0000 7f76563f4640 1 -- 172.21.15.76:0/1566008427 >> 172.21.15.76:0/1566008427 conn(0x7f765006f5e0 msgr2=0x7f765006f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:39.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.994+0000 7f76563f4640 1 -- 172.21.15.76:0/1566008427 shutdown_connections 2024-03-08T06:02:39.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.994+0000 7f76563f4640 1 -- 172.21.15.76:0/1566008427 wait complete. 2024-03-08T06:02:39.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.995+0000 7f76563f4640 1 Processor -- start 2024-03-08T06:02:39.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.996+0000 7f76563f4640 1 -- start start 2024-03-08T06:02:39.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.996+0000 7f76563f4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76500cf5b0 0x7f76500b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:39.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.996+0000 7f76563f4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f765015bb40 0x7f76500b6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:39.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.996+0000 7f76563f4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f765015bf20 0x7f76500b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:39.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.996+0000 7f76563f4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76500b11d0 con 0x7f76500cf5b0 2024-03-08T06:02:39.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.996+0000 7f76563f4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76500d06d0 con 0x7f765015bb40 2024-03-08T06:02:39.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.996+0000 7f76563f4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76500ce8f0 con 0x7f765015bf20 2024-03-08T06:02:39.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.996+0000 7f764ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76500cf5b0 0x7f76500b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:39.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.996+0000 7f764ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76500cf5b0 0x7f76500b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36958/0 (socket says 172.21.15.76:36958) 2024-03-08T06:02:39.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.996+0000 7f764ffff640 1 -- 172.21.15.76:0/3815340157 learned_addr learned my addr 172.21.15.76:0/3815340157 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:39.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.997+0000 7f7654991640 1 --2- 172.21.15.76:0/3815340157 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f765015bf20 0x7f76500b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.997+0000 7f764f7fe640 1 --2- 172.21.15.76:0/3815340157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f765015bb40 0x7f76500b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.997+0000 7f764ffff640 1 -- 172.21.15.76:0/3815340157 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f765015bf20 msgr2=0x7f76500b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.997+0000 7f764ffff640 1 --2- 172.21.15.76:0/3815340157 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f765015bf20 0x7f76500b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.997+0000 7f764ffff640 1 -- 172.21.15.76:0/3815340157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f765015bb40 msgr2=0x7f76500b6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.997+0000 7f764ffff640 1 --2- 172.21.15.76:0/3815340157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f765015bb40 0x7f76500b6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.997+0000 7f764ffff640 1 -- 172.21.15.76:0/3815340157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76500b4cb0 con 0x7f76500cf5b0 2024-03-08T06:02:40.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.997+0000 7f764f7fe640 1 --2- 172.21.15.76:0/3815340157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f765015bb40 0x7f76500b6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:40.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.997+0000 7f764ffff640 1 --2- 172.21.15.76:0/3815340157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76500cf5b0 0x7f76500b45b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f764400d800 tx=0x7f764400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:40.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.997+0000 7f764cff9640 1 -- 172.21.15.76:0/3815340157 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764400b6c0 con 0x7f76500cf5b0 2024-03-08T06:02:40.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.997+0000 7f764cff9640 1 -- 172.21.15.76:0/3815340157 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7644019070 con 0x7f76500cf5b0 2024-03-08T06:02:40.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.997+0000 7f76563f4640 1 -- 172.21.15.76:0/3815340157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76500b4f90 con 0x7f76500cf5b0 2024-03-08T06:02:40.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.997+0000 7f764cff9640 1 -- 172.21.15.76:0/3815340157 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764400ba80 con 0x7f76500cf5b0 2024-03-08T06:02:40.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.997+0000 7f76563f4640 1 -- 172.21.15.76:0/3815340157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7650154d20 con 0x7f76500cf5b0 2024-03-08T06:02:40.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.998+0000 7f76563f4640 1 -- 172.21.15.76:0/3815340157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76500b6a50 con 0x7f76500cf5b0 2024-03-08T06:02:40.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.999+0000 7f764cff9640 1 -- 172.21.15.76:0/3815340157 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f764400bc20 con 0x7f76500cf5b0 2024-03-08T06:02:40.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.999+0000 7f764cff9640 1 --2- 172.21.15.76:0/3815340157 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f763c06b500 0x7f763c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.999+0000 7f764cff9640 1 -- 172.21.15.76:0/3815340157 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f764408a730 con 0x7f76500cf5b0 2024-03-08T06:02:40.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.999+0000 7f764f7fe640 1 --2- 172.21.15.76:0/3815340157 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f763c06b500 0x7f763c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:39.999+0000 7f764f7fe640 1 --2- 172.21.15.76:0/3815340157 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f763c06b500 0x7f763c06d9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f76500b6c60 tx=0x7f7640007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:40.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.003+0000 7f764cff9640 1 -- 172.21.15.76:0/3815340157 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7644059490 con 0x7f76500cf5b0 2024-03-08T06:02:40.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.176+0000 7f76563f4640 1 -- 172.21.15.76:0/3815340157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f765004f030 con 0x7f76500cf5b0 2024-03-08T06:02:40.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.177+0000 7f764cff9640 1 -- 172.21.15.76:0/3815340157 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16850 (secure 0 0 0) 0x7f7644058d90 con 0x7f76500cf5b0 2024-03-08T06:02:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:02:40.178 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":42,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:02:18.021974+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:01:45.628991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-08T06:01:46.195100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:02:40.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.179+0000 7f76563f4640 1 -- 172.21.15.76:0/3815340157 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f763c06b500 msgr2=0x7f763c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.179+0000 7f76563f4640 1 --2- 172.21.15.76:0/3815340157 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f763c06b500 0x7f763c06d9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f76500b6c60 tx=0x7f7640007ca0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.179+0000 7f76563f4640 1 -- 172.21.15.76:0/3815340157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76500cf5b0 msgr2=0x7f76500b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.179+0000 7f76563f4640 1 --2- 172.21.15.76:0/3815340157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76500cf5b0 0x7f76500b45b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f764400d800 tx=0x7f764400dcc0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.179+0000 7f76563f4640 1 -- 172.21.15.76:0/3815340157 shutdown_connections 2024-03-08T06:02:40.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.179+0000 7f76563f4640 1 --2- 172.21.15.76:0/3815340157 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f763c06b500 0x7f763c06d9c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.179+0000 7f76563f4640 1 --2- 172.21.15.76:0/3815340157 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f765015bf20 0x7f76500b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.179+0000 7f76563f4640 1 --2- 172.21.15.76:0/3815340157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f765015bb40 0x7f76500b6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.179+0000 7f76563f4640 1 --2- 172.21.15.76:0/3815340157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76500cf5b0 0x7f76500b45b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.179+0000 7f76563f4640 1 -- 172.21.15.76:0/3815340157 >> 172.21.15.76:0/3815340157 conn(0x7f765006f5e0 msgr2=0x7f765015a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:40.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.179+0000 7f76563f4640 1 -- 172.21.15.76:0/3815340157 shutdown_connections 2024-03-08T06:02:40.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.179+0000 7f76563f4640 1 -- 172.21.15.76:0/3815340157 wait complete. 2024-03-08T06:02:40.190 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:01:45.628991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:01:46.195100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:02:40.190 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:02:40.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.293+0000 7f981fa40640 1 Processor -- start 2024-03-08T06:02:40.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.293+0000 7f981fa40640 1 -- start start 2024-03-08T06:02:40.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.294+0000 7f981fa40640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f981806d8a0 0x7f9818162020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.294+0000 7f981fa40640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9818161840 con 0x7f9818162560 2024-03-08T06:02:40.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.294+0000 7f981fa40640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981815eb30 con 0x7f98180ce5e0 2024-03-08T06:02:40.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.294+0000 7f981fa40640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9818161b90 con 0x7f981806d8a0 2024-03-08T06:02:40.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.294+0000 7f981cfdb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f981806d8a0 0x7f9818162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.294+0000 7f981dfdd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9818162560 0x7f981815dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40438/0 (socket says 172.21.15.76:40438) 2024-03-08T06:02:40.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.294+0000 7f981dfdd640 1 -- 172.21.15.76:0/2665876710 learned_addr learned my addr 172.21.15.76:0/2665876710 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:40.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.294+0000 7f981cfdb640 1 -- 172.21.15.76:0/2665876710 >> v1:172.21.15.81:6789/0 conn(0x7f98180ce5e0 legacy=0x7f98180ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:40.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.294+0000 7f981cfdb640 1 -- 172.21.15.76:0/2665876710 >> v1:172.21.15.76:6789/0 conn(0x7f9818162560 legacy=0x7f981815dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.294+0000 7f981cfdb640 1 -- 172.21.15.76:0/2665876710 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981815ee10 con 0x7f981806d8a0 2024-03-08T06:02:40.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.294+0000 7f980ffff640 1 -- 172.21.15.76:0/2665876710 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155060469 0 0) 0x7f9818161840 con 0x7f9818162560 2024-03-08T06:02:40.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.295+0000 7f981cfdb640 1 --2- 172.21.15.76:0/2665876710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f981806d8a0 0x7f9818162020 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f98080097a0 tx=0x7f980802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedbd1042786a43 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:40.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.295+0000 7f980ffff640 1 -- 172.21.15.76:0/2665876710 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808034070 con 0x7f981806d8a0 2024-03-08T06:02:40.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.295+0000 7f980ffff640 1 -- 172.21.15.76:0/2665876710 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9808030610 con 0x7f981806d8a0 2024-03-08T06:02:40.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.295+0000 7f980ffff640 1 -- 172.21.15.76:0/2665876710 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98080309e0 con 0x7f981806d8a0 2024-03-08T06:02:40.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.295+0000 7f981fa40640 1 -- 172.21.15.76:0/2665876710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f981806d8a0 msgr2=0x7f9818162020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.295+0000 7f981fa40640 1 --2- 172.21.15.76:0/2665876710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f981806d8a0 0x7f9818162020 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f98080097a0 tx=0x7f980802b960 comp rx=0 tx=0).stop 2024-03-08T06:02:40.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.295+0000 7f981fa40640 1 -- 172.21.15.76:0/2665876710 shutdown_connections 2024-03-08T06:02:40.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.295+0000 7f981fa40640 1 --2- 172.21.15.76:0/2665876710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f981806d8a0 0x7f9818162020 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.296+0000 7f981fa40640 1 -- 172.21.15.76:0/2665876710 >> 172.21.15.76:0/2665876710 conn(0x7f981806ae60 msgr2=0x7f981806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:40.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.296+0000 7f981fa40640 1 -- 172.21.15.76:0/2665876710 shutdown_connections 2024-03-08T06:02:40.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.296+0000 7f981fa40640 1 -- 172.21.15.76:0/2665876710 wait complete. 2024-03-08T06:02:40.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.297+0000 7f981fa40640 1 Processor -- start 2024-03-08T06:02:40.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.297+0000 7f981fa40640 1 -- start start 2024-03-08T06:02:40.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.297+0000 7f981fa40640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f981806d8a0 0x7f98181f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.297+0000 7f981fa40640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98180ce5e0 0x7f9818158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.297+0000 7f981fa40640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9818162560 0x7f98180b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.297+0000 7f981fa40640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981815e840 con 0x7f98180ce5e0 2024-03-08T06:02:40.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.297+0000 7f981fa40640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981815e6c0 con 0x7f981806d8a0 2024-03-08T06:02:40.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.297+0000 7f981fa40640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981806bfa0 con 0x7f9818162560 2024-03-08T06:02:40.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.297+0000 7f981cfdb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98180ce5e0 0x7f9818158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.298+0000 7f981cfdb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98180ce5e0 0x7f9818158ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36980/0 (socket says 172.21.15.76:36980) 2024-03-08T06:02:40.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.298+0000 7f981cfdb640 1 -- 172.21.15.76:0/2319554247 learned_addr learned my addr 172.21.15.76:0/2319554247 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:40.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.298+0000 7f981dfdd640 1 --2- 172.21.15.76:0/2319554247 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9818162560 0x7f98180b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.298+0000 7f981d7dc640 1 --2- 172.21.15.76:0/2319554247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f981806d8a0 0x7f98181f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.298+0000 7f981cfdb640 1 -- 172.21.15.76:0/2319554247 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9818162560 msgr2=0x7f98180b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.298+0000 7f981cfdb640 1 --2- 172.21.15.76:0/2319554247 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9818162560 0x7f98180b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.298+0000 7f981cfdb640 1 -- 172.21.15.76:0/2319554247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f981806d8a0 msgr2=0x7f98181f58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.298+0000 7f981cfdb640 1 --2- 172.21.15.76:0/2319554247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f981806d8a0 0x7f98181f58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.298+0000 7f981cfdb640 1 -- 172.21.15.76:0/2319554247 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98181e77a0 con 0x7f98180ce5e0 2024-03-08T06:02:40.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.298+0000 7f981d7dc640 1 --2- 172.21.15.76:0/2319554247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f981806d8a0 0x7f98181f58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:40.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.298+0000 7f981cfdb640 1 --2- 172.21.15.76:0/2319554247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98180ce5e0 0x7f9818158ef0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f980802bf70 tx=0x7f98080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:40.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.298+0000 7f980dffb640 1 -- 172.21.15.76:0/2319554247 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808034040 con 0x7f98180ce5e0 2024-03-08T06:02:40.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.298+0000 7f980dffb640 1 -- 172.21.15.76:0/2319554247 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9808039a10 con 0x7f98180ce5e0 2024-03-08T06:02:40.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.298+0000 7f981fa40640 1 -- 172.21.15.76:0/2319554247 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981815cd10 con 0x7f98180ce5e0 2024-03-08T06:02:40.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.298+0000 7f980dffb640 1 -- 172.21.15.76:0/2319554247 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808039e00 con 0x7f98180ce5e0 2024-03-08T06:02:40.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.298+0000 7f981fa40640 1 -- 172.21.15.76:0/2319554247 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98181e7d20 con 0x7f98180ce5e0 2024-03-08T06:02:40.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.300+0000 7f981fa40640 1 -- 172.21.15.76:0/2319554247 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981804f030 con 0x7f98180ce5e0 2024-03-08T06:02:40.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.300+0000 7f980dffb640 1 -- 172.21.15.76:0/2319554247 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9808037070 con 0x7f98180ce5e0 2024-03-08T06:02:40.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.300+0000 7f980dffb640 1 --2- 172.21.15.76:0/2319554247 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f97fc06b4d0 0x7f97fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.300+0000 7f980dffb640 1 -- 172.21.15.76:0/2319554247 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f98080ac490 con 0x7f98180ce5e0 2024-03-08T06:02:40.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.300+0000 7f981d7dc640 1 --2- 172.21.15.76:0/2319554247 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f97fc06b4d0 0x7f97fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.301+0000 7f981d7dc640 1 --2- 172.21.15.76:0/2319554247 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f97fc06b4d0 0x7f97fc06d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9814003eb0 tx=0x7f9814009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:40.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.305+0000 7f980dffb640 1 -- 172.21.15.76:0/2319554247 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f980807b160 con 0x7f98180ce5e0 2024-03-08T06:02:40.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.474+0000 7f981fa40640 1 -- 172.21.15.76:0/2319554247 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f981815fa40 con 0x7f98180ce5e0 2024-03-08T06:02:40.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.474+0000 7f980dffb640 1 -- 172.21.15.76:0/2319554247 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f980807aa60 con 0x7f98180ce5e0 2024-03-08T06:02:40.475 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:02:40.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.476+0000 7f981fa40640 1 -- 172.21.15.76:0/2319554247 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f97fc06b4d0 msgr2=0x7f97fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.476+0000 7f981fa40640 1 --2- 172.21.15.76:0/2319554247 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f97fc06b4d0 0x7f97fc06d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9814003eb0 tx=0x7f9814009040 comp rx=0 tx=0).stop 2024-03-08T06:02:40.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.476+0000 7f981fa40640 1 -- 172.21.15.76:0/2319554247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98180ce5e0 msgr2=0x7f9818158ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.476+0000 7f981fa40640 1 --2- 172.21.15.76:0/2319554247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98180ce5e0 0x7f9818158ef0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f980802bf70 tx=0x7f98080046c0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.476+0000 7f981fa40640 1 -- 172.21.15.76:0/2319554247 shutdown_connections 2024-03-08T06:02:40.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.476+0000 7f981fa40640 1 --2- 172.21.15.76:0/2319554247 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f97fc06b4d0 0x7f97fc06d990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.476+0000 7f981fa40640 1 --2- 172.21.15.76:0/2319554247 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9818162560 0x7f98180b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.476+0000 7f981fa40640 1 --2- 172.21.15.76:0/2319554247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98180ce5e0 0x7f9818158ef0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.476+0000 7f981fa40640 1 --2- 172.21.15.76:0/2319554247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f981806d8a0 0x7f98181f58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.476+0000 7f981fa40640 1 -- 172.21.15.76:0/2319554247 >> 172.21.15.76:0/2319554247 conn(0x7f981806ae60 msgr2=0x7f9818160ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:40.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.476+0000 7f981fa40640 1 -- 172.21.15.76:0/2319554247 shutdown_connections 2024-03-08T06:02:40.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.476+0000 7f981fa40640 1 -- 172.21.15.76:0/2319554247 wait complete. 2024-03-08T06:02:40.487 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:02:40.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.589+0000 7f7c87a23640 1 Processor -- start 2024-03-08T06:02:40.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.589+0000 7f7c87a23640 1 -- start start 2024-03-08T06:02:40.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.590+0000 7f7c87a23640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7c8015dbc0 0x7f7c800d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.590+0000 7f7c87a23640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7c800b5cf0 0x7f7c800b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.590+0000 7f7c87a23640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c800d27d0 con 0x7f7c800b9840 2024-03-08T06:02:40.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.590+0000 7f7c87a23640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c800d2ab0 con 0x7f7c800b5cf0 2024-03-08T06:02:40.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.590+0000 7f7c87a23640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800d2d90 con 0x7f7c8015dbc0 2024-03-08T06:02:40.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.590+0000 7f7c85fc0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7c800b5cf0 0x7f7c800b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.590+0000 7f7c84fbe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7c800b9840 0x7f7c800b55e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40446/0 (socket says 172.21.15.76:40446) 2024-03-08T06:02:40.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.590+0000 7f7c84fbe640 1 -- 172.21.15.76:0/2171184921 learned_addr learned my addr 172.21.15.76:0/2171184921 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:40.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.590+0000 7f7c857bf640 1 --2- 172.21.15.76:0/2171184921 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7c8015dbc0 0x7f7c800d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.591+0000 7f7c85fc0640 1 -- 172.21.15.76:0/2171184921 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7c8015dbc0 msgr2=0x7f7c800d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:40.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.591+0000 7f7c85fc0640 1 --2- 172.21.15.76:0/2171184921 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7c8015dbc0 0x7f7c800d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.591+0000 7f7c85fc0640 1 -- 172.21.15.76:0/2171184921 >> v1:172.21.15.76:6789/0 conn(0x7f7c800b9840 legacy=0x7f7c800b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.591+0000 7f7c77fff640 1 -- 172.21.15.76:0/2171184921 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256149742 0 0) 0x7f7c800d27d0 con 0x7f7c800b9840 2024-03-08T06:02:40.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.591+0000 7f7c85fc0640 1 -- 172.21.15.76:0/2171184921 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c800d3070 con 0x7f7c800b5cf0 2024-03-08T06:02:40.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.591+0000 7f7c85fc0640 1 --2- 172.21.15.76:0/2171184921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c800b5cf0 0x7f7c800b6c90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009d90 tx=0x7f7c7802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4436863f9f5572 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:40.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.591+0000 7f7c77fff640 1 -- 172.21.15.76:0/2171184921 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78039070 con 0x7f7c800b5cf0 2024-03-08T06:02:40.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.591+0000 7f7c77fff640 1 -- 172.21.15.76:0/2171184921 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7802cc50 con 0x7f7c800b5cf0 2024-03-08T06:02:40.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.591+0000 7f7c77fff640 1 -- 172.21.15.76:0/2171184921 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78034440 con 0x7f7c800b5cf0 2024-03-08T06:02:40.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.592+0000 7f7c87a23640 1 -- 172.21.15.76:0/2171184921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c800b5cf0 msgr2=0x7f7c800b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.592+0000 7f7c87a23640 1 --2- 172.21.15.76:0/2171184921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c800b5cf0 0x7f7c800b6c90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009d90 tx=0x7f7c7802bea0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.592+0000 7f7c87a23640 1 -- 172.21.15.76:0/2171184921 shutdown_connections 2024-03-08T06:02:40.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.592+0000 7f7c87a23640 1 --2- 172.21.15.76:0/2171184921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c800b5cf0 0x7f7c800b6c90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.592+0000 7f7c87a23640 1 --2- 172.21.15.76:0/2171184921 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7c8015dbc0 0x7f7c800d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.592+0000 7f7c87a23640 1 -- 172.21.15.76:0/2171184921 >> 172.21.15.76:0/2171184921 conn(0x7f7c800670d0 msgr2=0x7f7c800d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:40.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.592+0000 7f7c87a23640 1 -- 172.21.15.76:0/2171184921 shutdown_connections 2024-03-08T06:02:40.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.592+0000 7f7c87a23640 1 -- 172.21.15.76:0/2171184921 wait complete. 2024-03-08T06:02:40.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.593+0000 7f7c87a23640 1 Processor -- start 2024-03-08T06:02:40.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.593+0000 7f7c87a23640 1 -- start start 2024-03-08T06:02:40.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.594+0000 7f7c87a23640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c800b5cf0 0x7f7c80157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.594+0000 7f7c87a23640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7c800b9840 0x7f7c801f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.594+0000 7f7c87a23640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c8015dbc0 0x7f7c8014ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.594+0000 7f7c87a23640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800d25b0 con 0x7f7c800b5cf0 2024-03-08T06:02:40.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.594+0000 7f7c87a23640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800d2430 con 0x7f7c8015dbc0 2024-03-08T06:02:40.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.594+0000 7f7c87a23640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8015dfa0 con 0x7f7c800b9840 2024-03-08T06:02:40.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.594+0000 7f7c857bf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c800b5cf0 0x7f7c80157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.594+0000 7f7c857bf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c800b5cf0 0x7f7c80157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36998/0 (socket says 172.21.15.76:36998) 2024-03-08T06:02:40.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.594+0000 7f7c857bf640 1 -- 172.21.15.76:0/1047897960 learned_addr learned my addr 172.21.15.76:0/1047897960 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:40.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.594+0000 7f7c857bf640 1 -- 172.21.15.76:0/1047897960 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7c800b9840 msgr2=0x7f7c801f34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.594+0000 7f7c857bf640 1 --2- 172.21.15.76:0/1047897960 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7c800b9840 0x7f7c801f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.594+0000 7f7c857bf640 1 -- 172.21.15.76:0/1047897960 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c8015dbc0 msgr2=0x7f7c8014ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.594+0000 7f7c857bf640 1 --2- 172.21.15.76:0/1047897960 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c8015dbc0 0x7f7c8014ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.594+0000 7f7c857bf640 1 -- 172.21.15.76:0/1047897960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8014b380 con 0x7f7c800b5cf0 2024-03-08T06:02:40.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.595+0000 7f7c857bf640 1 --2- 172.21.15.76:0/1047897960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c800b5cf0 0x7f7c80157f40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00e870 tx=0x7f7c7c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:40.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.595+0000 7f7c75ffb640 1 -- 172.21.15.76:0/1047897960 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c017070 con 0x7f7c800b5cf0 2024-03-08T06:02:40.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.595+0000 7f7c75ffb640 1 -- 172.21.15.76:0/1047897960 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7c00c940 con 0x7f7c800b5cf0 2024-03-08T06:02:40.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.595+0000 7f7c75ffb640 1 -- 172.21.15.76:0/1047897960 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c00cd30 con 0x7f7c800b5cf0 2024-03-08T06:02:40.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.595+0000 7f7c87a23640 1 -- 172.21.15.76:0/1047897960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c800cc0f0 con 0x7f7c800b5cf0 2024-03-08T06:02:40.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.595+0000 7f7c87a23640 1 -- 172.21.15.76:0/1047897960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c800cc6b0 con 0x7f7c800b5cf0 2024-03-08T06:02:40.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.596+0000 7f7c87a23640 1 -- 172.21.15.76:0/1047897960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8004f030 con 0x7f7c800b5cf0 2024-03-08T06:02:40.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.596+0000 7f7c75ffb640 1 -- 172.21.15.76:0/1047897960 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c7c01a070 con 0x7f7c800b5cf0 2024-03-08T06:02:40.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.596+0000 7f7c75ffb640 1 --2- 172.21.15.76:0/1047897960 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7c6406b500 0x7f7c6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.597+0000 7f7c75ffb640 1 -- 172.21.15.76:0/1047897960 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f7c7c08a220 con 0x7f7c800b5cf0 2024-03-08T06:02:40.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.597+0000 7f7c84fbe640 1 --2- 172.21.15.76:0/1047897960 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7c6406b500 0x7f7c6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.597+0000 7f7c84fbe640 1 --2- 172.21.15.76:0/1047897960 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7c6406b500 0x7f7c6406d9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7c70007fd0 tx=0x7f7c70007900 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:40.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.601+0000 7f7c75ffb640 1 -- 172.21.15.76:0/1047897960 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c7c058ef0 con 0x7f7c800b5cf0 2024-03-08T06:02:40.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.769+0000 7f7c87a23640 1 -- 172.21.15.76:0/1047897960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7c800d4820 con 0x7f7c800b5cf0 2024-03-08T06:02:40.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.770+0000 7f7c75ffb640 1 -- 172.21.15.76:0/1047897960 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7c7c0587f0 con 0x7f7c800b5cf0 2024-03-08T06:02:40.771 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:02:40.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.772+0000 7f7c87a23640 1 -- 172.21.15.76:0/1047897960 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7c6406b500 msgr2=0x7f7c6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.772+0000 7f7c87a23640 1 --2- 172.21.15.76:0/1047897960 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7c6406b500 0x7f7c6406d9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7c70007fd0 tx=0x7f7c70007900 comp rx=0 tx=0).stop 2024-03-08T06:02:40.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.772+0000 7f7c87a23640 1 -- 172.21.15.76:0/1047897960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c800b5cf0 msgr2=0x7f7c80157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.772+0000 7f7c87a23640 1 --2- 172.21.15.76:0/1047897960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c800b5cf0 0x7f7c80157f40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00e870 tx=0x7f7c7c00ed30 comp rx=0 tx=0).stop 2024-03-08T06:02:40.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.772+0000 7f7c87a23640 1 -- 172.21.15.76:0/1047897960 shutdown_connections 2024-03-08T06:02:40.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.772+0000 7f7c87a23640 1 --2- 172.21.15.76:0/1047897960 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7c6406b500 0x7f7c6406d9c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.772+0000 7f7c87a23640 1 --2- 172.21.15.76:0/1047897960 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c8015dbc0 0x7f7c8014ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.772+0000 7f7c87a23640 1 --2- 172.21.15.76:0/1047897960 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7c800b9840 0x7f7c801f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.772+0000 7f7c87a23640 1 --2- 172.21.15.76:0/1047897960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c800b5cf0 0x7f7c80157f40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.772+0000 7f7c87a23640 1 -- 172.21.15.76:0/1047897960 >> 172.21.15.76:0/1047897960 conn(0x7f7c800670d0 msgr2=0x7f7c800c7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:40.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.772+0000 7f7c87a23640 1 -- 172.21.15.76:0/1047897960 shutdown_connections 2024-03-08T06:02:40.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.772+0000 7f7c87a23640 1 -- 172.21.15.76:0/1047897960 wait complete. 2024-03-08T06:02:40.783 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:02:40.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.886+0000 7f7fbd5a0640 1 Processor -- start 2024-03-08T06:02:40.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.886+0000 7f7fbd5a0640 1 -- start start 2024-03-08T06:02:40.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.887+0000 7f7fbd5a0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7fb80cef90 0x7f7fb80d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.887+0000 7f7fbd5a0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7fb80d1680 0x7f7fb80b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.887+0000 7f7fbd5a0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb80d1c00 con 0x7f7fb815de30 2024-03-08T06:02:40.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.887+0000 7f7fbd5a0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb80d3820 con 0x7f7fb80d1680 2024-03-08T06:02:40.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.887+0000 7f7fbd5a0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb80d2810 con 0x7f7fb80cef90 2024-03-08T06:02:40.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.887+0000 7f7fb77fe640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7fb80cef90 0x7f7fb80d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.887+0000 7f7fbcd9f640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7fb80d1680 0x7f7fb80b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.887+0000 7f7fb7fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f7fb815de30 0x7f7fb80ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:60814/0 (socket says 172.21.15.76:60814) 2024-03-08T06:02:40.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.887+0000 7f7fb7fff640 1 -- 172.21.15.76:0/1035296503 learned_addr learned my addr 172.21.15.76:0/1035296503 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:40.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.887+0000 7f7fbcd9f640 1 -- 172.21.15.76:0/1035296503 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7fb80cef90 msgr2=0x7f7fb80d2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:40.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.887+0000 7f7fbcd9f640 1 --2- 172.21.15.76:0/1035296503 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7fb80cef90 0x7f7fb80d2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.888+0000 7f7fbcd9f640 1 -- 172.21.15.76:0/1035296503 >> v1:172.21.15.81:6789/0 conn(0x7f7fb815de30 legacy=0x7f7fb80ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.888+0000 7f7fbcd9f640 1 -- 172.21.15.76:0/1035296503 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb80d1ee0 con 0x7f7fb80d1680 2024-03-08T06:02:40.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.888+0000 7f7fb6ffd640 1 -- 172.21.15.76:0/1035296503 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2525141892 0 0) 0x7f7fb80d1c00 con 0x7f7fb815de30 2024-03-08T06:02:40.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.888+0000 7f7fb77fe640 1 --2- 172.21.15.76:0/1035296503 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7fb80cef90 0x7f7fb80d2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:02:40.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.888+0000 7f7fbcd9f640 1 --2- 172.21.15.76:0/1035296503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fb80d1680 0x7f7fb80b7050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009d90 tx=0x7f7fb002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf6d6f6a55d5665 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:40.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.888+0000 7f7fb6ffd640 1 -- 172.21.15.76:0/1035296503 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0039070 con 0x7f7fb80d1680 2024-03-08T06:02:40.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.889+0000 7f7fb6ffd640 1 -- 172.21.15.76:0/1035296503 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb002cc50 con 0x7f7fb80d1680 2024-03-08T06:02:40.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.889+0000 7f7fb6ffd640 1 -- 172.21.15.76:0/1035296503 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0034440 con 0x7f7fb80d1680 2024-03-08T06:02:40.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.889+0000 7f7fbd5a0640 1 -- 172.21.15.76:0/1035296503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fb80d1680 msgr2=0x7f7fb80b7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.889+0000 7f7fbd5a0640 1 --2- 172.21.15.76:0/1035296503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fb80d1680 0x7f7fb80b7050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009d90 tx=0x7f7fb002bea0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.889+0000 7f7fbd5a0640 1 -- 172.21.15.76:0/1035296503 shutdown_connections 2024-03-08T06:02:40.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.889+0000 7f7fbd5a0640 1 --2- 172.21.15.76:0/1035296503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fb80d1680 0x7f7fb80b7050 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.889+0000 7f7fbd5a0640 1 --2- 172.21.15.76:0/1035296503 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7fb80cef90 0x7f7fb80d2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.889+0000 7f7fbd5a0640 1 -- 172.21.15.76:0/1035296503 >> 172.21.15.76:0/1035296503 conn(0x7f7fb8067070 msgr2=0x7f7fb80d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:40.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.889+0000 7f7fbd5a0640 1 -- 172.21.15.76:0/1035296503 shutdown_connections 2024-03-08T06:02:40.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.889+0000 7f7fbd5a0640 1 -- 172.21.15.76:0/1035296503 wait complete. 2024-03-08T06:02:40.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.890+0000 7f7fbd5a0640 1 Processor -- start 2024-03-08T06:02:40.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.890+0000 7f7fbd5a0640 1 -- start start 2024-03-08T06:02:40.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.891+0000 7f7fbd5a0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fb80cef90 0x7f7fb81504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.891+0000 7f7fbd5a0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fb80d1680 0x7f7fb80ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.891+0000 7f7fbd5a0640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7fb815de30 0x7f7fb80b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.891+0000 7f7fbd5a0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb80b7590 con 0x7f7fb80d1680 2024-03-08T06:02:40.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.891+0000 7f7fbd5a0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb80d24a0 con 0x7f7fb80cef90 2024-03-08T06:02:40.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.891+0000 7f7fbd5a0640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb80b7710 con 0x7f7fb815de30 2024-03-08T06:02:40.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.891+0000 7f7fb77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fb80d1680 0x7f7fb80ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.891+0000 7f7fb77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fb80d1680 0x7f7fb80ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37014/0 (socket says 172.21.15.76:37014) 2024-03-08T06:02:40.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.891+0000 7f7fb77fe640 1 -- 172.21.15.76:0/2969167481 learned_addr learned my addr 172.21.15.76:0/2969167481 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:40.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.891+0000 7f7fbcd9f640 1 --2- 172.21.15.76:0/2969167481 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7fb815de30 0x7f7fb80b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.891+0000 7f7fb7fff640 1 --2- 172.21.15.76:0/2969167481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fb80cef90 0x7f7fb81504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.892+0000 7f7fb77fe640 1 -- 172.21.15.76:0/2969167481 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7fb815de30 msgr2=0x7f7fb80b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.892+0000 7f7fb77fe640 1 --2- 172.21.15.76:0/2969167481 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7fb815de30 0x7f7fb80b8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.892+0000 7f7fb77fe640 1 -- 172.21.15.76:0/2969167481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fb80cef90 msgr2=0x7f7fb81504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:40.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.892+0000 7f7fb77fe640 1 --2- 172.21.15.76:0/2969167481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fb80cef90 0x7f7fb81504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:40.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.892+0000 7f7fb77fe640 1 -- 172.21.15.76:0/2969167481 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb8150a10 con 0x7f7fb80d1680 2024-03-08T06:02:40.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.892+0000 7f7fb77fe640 1 --2- 172.21.15.76:0/2969167481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fb80d1680 0x7f7fb80ccad0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00ebe0 tx=0x7f7fac00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:40.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.892+0000 7f7fb4ff9640 1 -- 172.21.15.76:0/2969167481 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac017070 con 0x7f7fb80d1680 2024-03-08T06:02:40.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.892+0000 7f7fb4ff9640 1 -- 172.21.15.76:0/2969167481 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fac00cc30 con 0x7f7fb80d1680 2024-03-08T06:02:40.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.892+0000 7f7fbd5a0640 1 -- 172.21.15.76:0/2969167481 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb80cb010 con 0x7f7fb80d1680 2024-03-08T06:02:40.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.892+0000 7f7fb4ff9640 1 -- 172.21.15.76:0/2969167481 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac013470 con 0x7f7fb80d1680 2024-03-08T06:02:40.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.892+0000 7f7fbd5a0640 1 -- 172.21.15.76:0/2969167481 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb80cb5d0 con 0x7f7fb80d1680 2024-03-08T06:02:40.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.893+0000 7f7fbd5a0640 1 -- 172.21.15.76:0/2969167481 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb806a1d0 con 0x7f7fb80d1680 2024-03-08T06:02:40.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.893+0000 7f7fb4ff9640 1 -- 172.21.15.76:0/2969167481 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7fac01a070 con 0x7f7fb80d1680 2024-03-08T06:02:40.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.894+0000 7f7fb4ff9640 1 --2- 172.21.15.76:0/2969167481 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7fa406b500 0x7f7fa406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:40.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.894+0000 7f7fb4ff9640 1 -- 172.21.15.76:0/2969167481 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f7fac08a610 con 0x7f7fb80d1680 2024-03-08T06:02:40.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.894+0000 7f7fb7fff640 1 --2- 172.21.15.76:0/2969167481 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7fa406b500 0x7f7fa406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:40.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.894+0000 7f7fb7fff640 1 --2- 172.21.15.76:0/2969167481 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7fa406b500 0x7f7fa406d9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8007fd0 tx=0x7f7fa8007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:40.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:40.898+0000 7f7fb4ff9640 1 -- 172.21.15.76:0/2969167481 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7fb8150a10 con 0x7f7fb80d1680 2024-03-08T06:02:41.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.067+0000 7f7fbd5a0640 1 -- 172.21.15.76:0/2969167481 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7fb80b3680 con 0x7f7fb80d1680 2024-03-08T06:02:41.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.067+0000 7f7fb4ff9640 1 -- 172.21.15.76:0/2969167481 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7fac058ca0 con 0x7f7fb80d1680 2024-03-08T06:02:41.068 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:02:41.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.069+0000 7f7fbd5a0640 1 -- 172.21.15.76:0/2969167481 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7fa406b500 msgr2=0x7f7fa406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.069+0000 7f7fbd5a0640 1 --2- 172.21.15.76:0/2969167481 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7fa406b500 0x7f7fa406d9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8007fd0 tx=0x7f7fa8007de0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.069+0000 7f7fbd5a0640 1 -- 172.21.15.76:0/2969167481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fb80d1680 msgr2=0x7f7fb80ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.069+0000 7f7fbd5a0640 1 --2- 172.21.15.76:0/2969167481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fb80d1680 0x7f7fb80ccad0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00ebe0 tx=0x7f7fac00c520 comp rx=0 tx=0).stop 2024-03-08T06:02:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.069+0000 7f7fbd5a0640 1 -- 172.21.15.76:0/2969167481 shutdown_connections 2024-03-08T06:02:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.069+0000 7f7fbd5a0640 1 --2- 172.21.15.76:0/2969167481 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7fa406b500 0x7f7fa406d9c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.069+0000 7f7fbd5a0640 1 --2- 172.21.15.76:0/2969167481 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7fb815de30 0x7f7fb80b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.069+0000 7f7fbd5a0640 1 --2- 172.21.15.76:0/2969167481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fb80d1680 0x7f7fb80ccad0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.069+0000 7f7fbd5a0640 1 --2- 172.21.15.76:0/2969167481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fb80cef90 0x7f7fb81504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.069+0000 7f7fbd5a0640 1 -- 172.21.15.76:0/2969167481 >> 172.21.15.76:0/2969167481 conn(0x7f7fb8067070 msgr2=0x7f7fb815ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.070+0000 7f7fbd5a0640 1 -- 172.21.15.76:0/2969167481 shutdown_connections 2024-03-08T06:02:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.070+0000 7f7fbd5a0640 1 -- 172.21.15.76:0/2969167481 wait complete. 2024-03-08T06:02:41.081 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T06:02:41.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.182+0000 7fb2cb33a640 1 Processor -- start 2024-03-08T06:02:41.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.182+0000 7fb2cb33a640 1 -- start start 2024-03-08T06:02:41.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.183+0000 7fb2cb33a640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb2c4154df0 0x7fb2c40b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.183+0000 7fb2cb33a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb2c40b3640 0x7fb2c40cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.183+0000 7fb2cb33a640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c415f450 con 0x7fb2c40cc610 2024-03-08T06:02:41.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.183+0000 7fb2cb33a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c40af1e0 con 0x7fb2c40b3640 2024-03-08T06:02:41.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.183+0000 7fb2cb33a640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c40b3c80 con 0x7fb2c4154df0 2024-03-08T06:02:41.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.183+0000 7fb2c98d7640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb2c40cc610 0x7fb2c4071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40452/0 (socket says 172.21.15.76:40452) 2024-03-08T06:02:41.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.183+0000 7fb2c98d7640 1 -- 172.21.15.76:0/78203967 learned_addr learned my addr 172.21.15.76:0/78203967 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:41.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2bbfff640 1 -- 172.21.15.76:0/78203967 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171392888 0 0) 0x7fb2c415f450 con 0x7fb2c40cc610 2024-03-08T06:02:41.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2bbfff640 1 -- 172.21.15.76:0/78203967 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2b0003520 con 0x7fb2c40cc610 2024-03-08T06:02:41.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2c88d5640 1 --2- 172.21.15.76:0/78203967 >> v2:172.21.15.81:3300/0 conn(0x7fb2c40b3640 0x7fb2c40cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2c90d6640 1 --2- 172.21.15.76:0/78203967 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb2c4154df0 0x7fb2c40b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2bbfff640 1 -- 172.21.15.76:0/78203967 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3377698663 0 0) 0x7fb2b0003520 con 0x7fb2c40cc610 2024-03-08T06:02:41.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2bbfff640 1 -- 172.21.15.76:0/78203967 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb2c4154df0 msgr2=0x7fb2c40b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:41.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2bbfff640 1 --2- 172.21.15.76:0/78203967 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb2c4154df0 0x7fb2c40b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2bbfff640 1 -- 172.21.15.76:0/78203967 >> v2:172.21.15.81:3300/0 conn(0x7fb2c40b3640 msgr2=0x7fb2c40cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:41.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2bbfff640 1 --2- 172.21.15.76:0/78203967 >> v2:172.21.15.81:3300/0 conn(0x7fb2c40b3640 0x7fb2c40cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2bbfff640 1 -- 172.21.15.76:0/78203967 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c40cca30 con 0x7fb2c40cc610 2024-03-08T06:02:41.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2bbfff640 1 -- 172.21.15.76:0/78203967 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fb2bc001670 con 0x7fb2c40cc610 2024-03-08T06:02:41.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2bbfff640 1 -- 172.21.15.76:0/78203967 >> v1:172.21.15.76:6789/0 conn(0x7fb2c40cc610 legacy=0x7fb2c4071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2bbfff640 1 --2- 172.21.15.76:0/78203967 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb2c4154df0 0x7fb2b0003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2bbfff640 1 --2- 172.21.15.76:0/78203967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2b0004050 0x7fb2b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2bbfff640 1 --2- 172.21.15.76:0/78203967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2b0004d40 0x7fb2b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2bbfff640 1 -- 172.21.15.76:0/78203967 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c40af1e0 con 0x7fb2b0004d40 2024-03-08T06:02:41.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2bbfff640 1 -- 172.21.15.76:0/78203967 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c40b3c80 con 0x7fb2b0004050 2024-03-08T06:02:41.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2bbfff640 1 -- 172.21.15.76:0/78203967 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b000bc00 con 0x7fb2c4154df0 2024-03-08T06:02:41.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2c98d7640 1 --2- 172.21.15.76:0/78203967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2b0004d40 0x7fb2b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2c90d6640 1 --2- 172.21.15.76:0/78203967 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb2c4154df0 0x7fb2b0003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2c88d5640 1 --2- 172.21.15.76:0/78203967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2b0004050 0x7fb2b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2c98d7640 1 -- 172.21.15.76:0/78203967 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb2c4154df0 msgr2=0x7fb2b0003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:41.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2c98d7640 1 --2- 172.21.15.76:0/78203967 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb2c4154df0 0x7fb2b0003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2c98d7640 1 -- 172.21.15.76:0/78203967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2b0004050 msgr2=0x7fb2b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:41.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2c98d7640 1 --2- 172.21.15.76:0/78203967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2b0004050 0x7fb2b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.184+0000 7fb2c98d7640 1 -- 172.21.15.76:0/78203967 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b0003520 con 0x7fb2b0004d40 2024-03-08T06:02:41.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.185+0000 7fb2c98d7640 1 --2- 172.21.15.76:0/78203967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2b0004d40 0x7fb2b00051c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc007d10 tx=0x7fb2b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d2c97dae0bc883 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:41.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.185+0000 7fb2bbfff640 1 -- 172.21.15.76:0/78203967 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc007d50 con 0x7fb2b0004d40 2024-03-08T06:02:41.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.185+0000 7fb2bbfff640 1 -- 172.21.15.76:0/78203967 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc032070 con 0x7fb2b0004d40 2024-03-08T06:02:41.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.185+0000 7fb2bbfff640 1 -- 172.21.15.76:0/78203967 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0058d0 con 0x7fb2b0004d40 2024-03-08T06:02:41.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.186+0000 7fb2cb33a640 1 -- 172.21.15.76:0/78203967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2b0004d40 msgr2=0x7fb2b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.186+0000 7fb2cb33a640 1 --2- 172.21.15.76:0/78203967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2b0004d40 0x7fb2b00051c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc007d10 tx=0x7fb2b0005830 comp rx=0 tx=0).stop 2024-03-08T06:02:41.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.186+0000 7fb2c98d7640 1 -- 172.21.15.76:0/78203967 reap_dead start 2024-03-08T06:02:41.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.186+0000 7fb2cb33a640 1 -- 172.21.15.76:0/78203967 shutdown_connections 2024-03-08T06:02:41.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.186+0000 7fb2cb33a640 1 -- 172.21.15.76:0/78203967 >> 172.21.15.76:0/78203967 conn(0x7fb2c406ac80 msgr2=0x7fb2c406b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:41.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.186+0000 7fb2cb33a640 1 -- 172.21.15.76:0/78203967 shutdown_connections 2024-03-08T06:02:41.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.186+0000 7fb2cb33a640 1 -- 172.21.15.76:0/78203967 wait complete. 2024-03-08T06:02:41.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.187+0000 7fb2cb33a640 1 Processor -- start 2024-03-08T06:02:41.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.187+0000 7fb2cb33a640 1 -- start start 2024-03-08T06:02:41.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2cb33a640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb2b0004d40 0x7fb2c41522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2cb33a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2c4143590 0x7fb2c406e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2cb33a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2c4140b40 0x7fb2c4140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2cb33a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c415ec50 con 0x7fb2c4140b40 2024-03-08T06:02:41.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2cb33a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c415e970 con 0x7fb2c4143590 2024-03-08T06:02:41.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2cb33a640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c415ef30 con 0x7fb2b0004d40 2024-03-08T06:02:41.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2c98d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2c4140b40 0x7fb2c4140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2c98d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2c4140b40 0x7fb2c4140fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37036/0 (socket says 172.21.15.76:37036) 2024-03-08T06:02:41.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2c98d7640 1 -- 172.21.15.76:0/2980123141 learned_addr learned my addr 172.21.15.76:0/2980123141 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:41.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2c90d6640 1 --2- 172.21.15.76:0/2980123141 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb2b0004d40 0x7fb2c41522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2c88d5640 1 --2- 172.21.15.76:0/2980123141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2c4143590 0x7fb2c406e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2c98d7640 1 -- 172.21.15.76:0/2980123141 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb2b0004d40 msgr2=0x7fb2c41522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2c98d7640 1 --2- 172.21.15.76:0/2980123141 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb2b0004d40 0x7fb2c41522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2c98d7640 1 -- 172.21.15.76:0/2980123141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2c4143590 msgr2=0x7fb2c406e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2c98d7640 1 --2- 172.21.15.76:0/2980123141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2c4143590 0x7fb2c406e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2c98d7640 1 -- 172.21.15.76:0/2980123141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c40c49b0 con 0x7fb2c4140b40 2024-03-08T06:02:41.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2c98d7640 1 --2- 172.21.15.76:0/2980123141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2c4140b40 0x7fb2c4140fa0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc02e7b0 tx=0x7fb2bc001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:41.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2b9ffb640 1 -- 172.21.15.76:0/2980123141 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc001c90 con 0x7fb2c4140b40 2024-03-08T06:02:41.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2b9ffb640 1 -- 172.21.15.76:0/2980123141 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc004bf0 con 0x7fb2c4140b40 2024-03-08T06:02:41.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2cb33a640 1 -- 172.21.15.76:0/2980123141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c4151800 con 0x7fb2c4140b40 2024-03-08T06:02:41.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.188+0000 7fb2b9ffb640 1 -- 172.21.15.76:0/2980123141 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc007070 con 0x7fb2c4140b40 2024-03-08T06:02:41.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.189+0000 7fb2cb33a640 1 -- 172.21.15.76:0/2980123141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c4151dc0 con 0x7fb2c4140b40 2024-03-08T06:02:41.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.190+0000 7fb2cb33a640 1 -- 172.21.15.76:0/2980123141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c404f030 con 0x7fb2c4140b40 2024-03-08T06:02:41.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.190+0000 7fb2b9ffb640 1 -- 172.21.15.76:0/2980123141 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2bc007210 con 0x7fb2c4140b40 2024-03-08T06:02:41.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.191+0000 7fb2b9ffb640 1 --2- 172.21.15.76:0/2980123141 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb2a806b4d0 0x7fb2a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.191+0000 7fb2c90d6640 1 --2- 172.21.15.76:0/2980123141 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb2a806b4d0 0x7fb2a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.191+0000 7fb2b9ffb640 1 -- 172.21.15.76:0/2980123141 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fb2bc0ad520 con 0x7fb2c4140b40 2024-03-08T06:02:41.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.191+0000 7fb2c90d6640 1 --2- 172.21.15.76:0/2980123141 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb2a806b4d0 0x7fb2a806d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0003b30 tx=0x7fb2c0009760 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:41.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.195+0000 7fb2b9ffb640 1 -- 172.21.15.76:0/2980123141 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2bc07bba0 con 0x7fb2c4140b40 2024-03-08T06:02:41.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.377+0000 7fb2cb33a640 1 -- 172.21.15.76:0/2980123141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb2c406b270 con 0x7fb2c4140b40 2024-03-08T06:02:41.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.377+0000 7fb2b9ffb640 1 -- 172.21.15.76:0/2980123141 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb2bc07fc40 con 0x7fb2c4140b40 2024-03-08T06:02:41.377 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-03-08T06:02:41.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.379+0000 7fb2cb33a640 1 -- 172.21.15.76:0/2980123141 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb2a806b4d0 msgr2=0x7fb2a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.379+0000 7fb2cb33a640 1 --2- 172.21.15.76:0/2980123141 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb2a806b4d0 0x7fb2a806d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0003b30 tx=0x7fb2c0009760 comp rx=0 tx=0).stop 2024-03-08T06:02:41.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.379+0000 7fb2cb33a640 1 -- 172.21.15.76:0/2980123141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2c4140b40 msgr2=0x7fb2c4140fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.379+0000 7fb2cb33a640 1 --2- 172.21.15.76:0/2980123141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2c4140b40 0x7fb2c4140fa0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc02e7b0 tx=0x7fb2bc001ab0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.379+0000 7fb2cb33a640 1 -- 172.21.15.76:0/2980123141 shutdown_connections 2024-03-08T06:02:41.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.379+0000 7fb2cb33a640 1 --2- 172.21.15.76:0/2980123141 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb2a806b4d0 0x7fb2a806d990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.379+0000 7fb2cb33a640 1 --2- 172.21.15.76:0/2980123141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2c4140b40 0x7fb2c4140fa0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.379+0000 7fb2cb33a640 1 --2- 172.21.15.76:0/2980123141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2c4143590 0x7fb2c406e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.379+0000 7fb2cb33a640 1 --2- 172.21.15.76:0/2980123141 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb2b0004d40 0x7fb2c41522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.379+0000 7fb2cb33a640 1 -- 172.21.15.76:0/2980123141 >> 172.21.15.76:0/2980123141 conn(0x7fb2c406ac80 msgr2=0x7fb2c4153f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:41.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.379+0000 7fb2cb33a640 1 -- 172.21.15.76:0/2980123141 shutdown_connections 2024-03-08T06:02:41.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.379+0000 7fb2cb33a640 1 -- 172.21.15.76:0/2980123141 wait complete. 2024-03-08T06:02:41.390 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-08T06:02:41.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:41.390 DEBUG:teuthology.orchestra.run.smithi081:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-08T06:02:41.426 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-03-08T06:02:41.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:41.426 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-08T06:02:41.484 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:02:41.484 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:02:41.499 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056373,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17773 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-08T06:02:41.500 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:02:41.500 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:02:41.543 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14569 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-08T06:02:41.543 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:02:41.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.643+0000 7fc36b56c640 1 Processor -- start 2024-03-08T06:02:41.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.643+0000 7fc36b56c640 1 -- start start 2024-03-08T06:02:41.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.644+0000 7fc36b56c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc36406ac80 0x7fc3640cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.644+0000 7fc36b56c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc3640af9c0 0x7fc36406b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.644+0000 7fc36b56c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc36406eee0 0x7fc3640caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.644+0000 7fc36b56c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc36406b5a0 con 0x7fc36406eee0 2024-03-08T06:02:41.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.644+0000 7fc36b56c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3640d0150 con 0x7fc3640af9c0 2024-03-08T06:02:41.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.644+0000 7fc36b56c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640d0430 con 0x7fc36406ac80 2024-03-08T06:02:41.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.644+0000 7fc369b09640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc36406eee0 0x7fc3640caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.644+0000 7fc369b09640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc36406eee0 0x7fc3640caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37038/0 (socket says 172.21.15.76:37038) 2024-03-08T06:02:41.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.644+0000 7fc369b09640 1 -- 172.21.15.76:0/2890294670 learned_addr learned my addr 172.21.15.76:0/2890294670 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:41.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.644+0000 7fc368b07640 1 --2- 172.21.15.76:0/2890294670 >> v2:172.21.15.81:3300/0 conn(0x7fc3640af9c0 0x7fc36406b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.644+0000 7fc369308640 1 --2- 172.21.15.76:0/2890294670 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc36406ac80 0x7fc3640cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.644+0000 7fc369b09640 1 -- 172.21.15.76:0/2890294670 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc36406ac80 msgr2=0x7fc3640cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:41.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.644+0000 7fc369b09640 1 --2- 172.21.15.76:0/2890294670 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc36406ac80 0x7fc3640cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.644+0000 7fc369b09640 1 -- 172.21.15.76:0/2890294670 >> v2:172.21.15.81:3300/0 conn(0x7fc3640af9c0 msgr2=0x7fc36406b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:41.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.644+0000 7fc369b09640 1 --2- 172.21.15.76:0/2890294670 >> v2:172.21.15.81:3300/0 conn(0x7fc3640af9c0 0x7fc36406b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.644+0000 7fc369b09640 1 -- 172.21.15.76:0/2890294670 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36406e6e0 con 0x7fc36406eee0 2024-03-08T06:02:41.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.645+0000 7fc369b09640 1 --2- 172.21.15.76:0/2890294670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36406eee0 0x7fc3640caea0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc364070f10 tx=0x7fc35c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2a2693abf0b68d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:41.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.645+0000 7fc35bfff640 1 -- 172.21.15.76:0/2890294670 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c02ca70 con 0x7fc36406eee0 2024-03-08T06:02:41.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.645+0000 7fc35bfff640 1 -- 172.21.15.76:0/2890294670 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35c02cc10 con 0x7fc36406eee0 2024-03-08T06:02:41.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.645+0000 7fc35bfff640 1 -- 172.21.15.76:0/2890294670 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0343f0 con 0x7fc36406eee0 2024-03-08T06:02:41.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.646+0000 7fc36b56c640 1 -- 172.21.15.76:0/2890294670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36406eee0 msgr2=0x7fc3640caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.646+0000 7fc36b56c640 1 --2- 172.21.15.76:0/2890294670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36406eee0 0x7fc3640caea0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc364070f10 tx=0x7fc35c02bed0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.646+0000 7fc36b56c640 1 -- 172.21.15.76:0/2890294670 shutdown_connections 2024-03-08T06:02:41.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.646+0000 7fc36b56c640 1 --2- 172.21.15.76:0/2890294670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36406eee0 0x7fc3640caea0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.646+0000 7fc36b56c640 1 --2- 172.21.15.76:0/2890294670 >> v2:172.21.15.81:3300/0 conn(0x7fc3640af9c0 0x7fc36406b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.646+0000 7fc36b56c640 1 --2- 172.21.15.76:0/2890294670 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc36406ac80 0x7fc3640cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.646+0000 7fc36b56c640 1 -- 172.21.15.76:0/2890294670 >> 172.21.15.76:0/2890294670 conn(0x7fc364099a60 msgr2=0x7fc3640d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:41.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.646+0000 7fc36b56c640 1 -- 172.21.15.76:0/2890294670 shutdown_connections 2024-03-08T06:02:41.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.646+0000 7fc36b56c640 1 -- 172.21.15.76:0/2890294670 wait complete. 2024-03-08T06:02:41.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.647+0000 7fc36b56c640 1 Processor -- start 2024-03-08T06:02:41.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.647+0000 7fc36b56c640 1 -- start start 2024-03-08T06:02:41.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc36b56c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc36406ac80 0x7fc3640c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc36b56c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36406eee0 0x7fc3640c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc36b56c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc3640af9c0 0x7fc3641f1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc36b56c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36415e280 con 0x7fc36406eee0 2024-03-08T06:02:41.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc36b56c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640afe20 con 0x7fc36406ac80 2024-03-08T06:02:41.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc36b56c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36406f370 con 0x7fc3640af9c0 2024-03-08T06:02:41.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc368b07640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36406eee0 0x7fc3640c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc368b07640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36406eee0 0x7fc3640c7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37046/0 (socket says 172.21.15.76:37046) 2024-03-08T06:02:41.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc368b07640 1 -- 172.21.15.76:0/3828056424 learned_addr learned my addr 172.21.15.76:0/3828056424 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:41.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc369308640 1 --2- 172.21.15.76:0/3828056424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc36406ac80 0x7fc3640c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc369b09640 1 --2- 172.21.15.76:0/3828056424 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc3640af9c0 0x7fc3641f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc368b07640 1 -- 172.21.15.76:0/3828056424 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc3640af9c0 msgr2=0x7fc3641f1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc368b07640 1 --2- 172.21.15.76:0/3828056424 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc3640af9c0 0x7fc3641f1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc368b07640 1 -- 172.21.15.76:0/3828056424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc36406ac80 msgr2=0x7fc3640c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc368b07640 1 --2- 172.21.15.76:0/3828056424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc36406ac80 0x7fc3640c9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc368b07640 1 -- 172.21.15.76:0/3828056424 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3640c7fc0 con 0x7fc36406eee0 2024-03-08T06:02:41.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc368b07640 1 --2- 172.21.15.76:0/3828056424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36406eee0 0x7fc3640c7920 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc35400d5f0 tx=0x7fc35400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:41.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc359ffb640 1 -- 172.21.15.76:0/3828056424 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3540125f0 con 0x7fc36406eee0 2024-03-08T06:02:41.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc359ffb640 1 -- 172.21.15.76:0/3828056424 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc354019070 con 0x7fc36406eee0 2024-03-08T06:02:41.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc36b56c640 1 -- 172.21.15.76:0/3828056424 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3640c82a0 con 0x7fc36406eee0 2024-03-08T06:02:41.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.648+0000 7fc359ffb640 1 -- 172.21.15.76:0/3828056424 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3540129d0 con 0x7fc36406eee0 2024-03-08T06:02:41.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.649+0000 7fc36b56c640 1 -- 172.21.15.76:0/3828056424 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3640c8860 con 0x7fc36406eee0 2024-03-08T06:02:41.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.649+0000 7fc36b56c640 1 -- 172.21.15.76:0/3828056424 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36404f030 con 0x7fc36406eee0 2024-03-08T06:02:41.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.650+0000 7fc359ffb640 1 -- 172.21.15.76:0/3828056424 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc354012b70 con 0x7fc36406eee0 2024-03-08T06:02:41.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.650+0000 7fc359ffb640 1 --2- 172.21.15.76:0/3828056424 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc34806b500 0x7fc34806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.650+0000 7fc359ffb640 1 -- 172.21.15.76:0/3828056424 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fc35408a780 con 0x7fc36406eee0 2024-03-08T06:02:41.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.650+0000 7fc369308640 1 --2- 172.21.15.76:0/3828056424 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc34806b500 0x7fc34806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.651+0000 7fc369308640 1 --2- 172.21.15.76:0/3828056424 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc34806b500 0x7fc34806d9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc360006fd0 tx=0x7fc360008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:41.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.655+0000 7fc359ffb640 1 -- 172.21.15.76:0/3828056424 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc354059450 con 0x7fc36406eee0 2024-03-08T06:02:41.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.864+0000 7fc36b56c640 1 -- 172.21.15.76:0/3828056424 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3640cd040 con 0x7fc36406eee0 2024-03-08T06:02:41.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.864+0000 7fc359ffb640 1 -- 172.21.15.76:0/3828056424 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4932 (secure 0 0 0) 0x7fc354058d50 con 0x7fc36406eee0 2024-03-08T06:02:41.865 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:02:41.865 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/606148582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":606148582},{"type":"v1","addr":"172.21.15.81:6835","nonce":606148582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/278988477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":278988477},{"type":"v1","addr":"172.21.15.76:6836","nonce":278988477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2526423312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":2526423312},{"type":"v1","addr":"172.21.15.76:6837","nonce":2526423312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:01:47.232184+0000","modified":"2024-03-08T06:01:49.227002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5358":{"gid":5358,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6834/1503895376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1503895376},{"type":"v1","addr":"172.21.15.81:6834","nonce":1503895376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T06:02:41.865 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 15 2024-03-08T06:02:41.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.866+0000 7fc36b56c640 1 -- 172.21.15.76:0/3828056424 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc34806b500 msgr2=0x7fc34806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.866+0000 7fc36b56c640 1 --2- 172.21.15.76:0/3828056424 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc34806b500 0x7fc34806d9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc360006fd0 tx=0x7fc360008040 comp rx=0 tx=0).stop 2024-03-08T06:02:41.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.866+0000 7fc36b56c640 1 -- 172.21.15.76:0/3828056424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36406eee0 msgr2=0x7fc3640c7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.866+0000 7fc36b56c640 1 --2- 172.21.15.76:0/3828056424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36406eee0 0x7fc3640c7920 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc35400d5f0 tx=0x7fc35400dab0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.866+0000 7fc36b56c640 1 -- 172.21.15.76:0/3828056424 shutdown_connections 2024-03-08T06:02:41.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.866+0000 7fc36b56c640 1 --2- 172.21.15.76:0/3828056424 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc34806b500 0x7fc34806d9c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.866+0000 7fc36b56c640 1 --2- 172.21.15.76:0/3828056424 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc3640af9c0 0x7fc3641f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.866+0000 7fc36b56c640 1 --2- 172.21.15.76:0/3828056424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36406eee0 0x7fc3640c7920 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.866+0000 7fc36b56c640 1 --2- 172.21.15.76:0/3828056424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc36406ac80 0x7fc3640c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.866+0000 7fc36b56c640 1 -- 172.21.15.76:0/3828056424 >> 172.21.15.76:0/3828056424 conn(0x7fc364099a60 msgr2=0x7fc3640ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:41.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.866+0000 7fc36b56c640 1 -- 172.21.15.76:0/3828056424 shutdown_connections 2024-03-08T06:02:41.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.866+0000 7fc36b56c640 1 -- 172.21.15.76:0/3828056424 wait complete. 2024-03-08T06:02:41.878 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:02:41.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.978+0000 7fe75bfff640 1 Processor -- start 2024-03-08T06:02:41.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.978+0000 7fe75bfff640 1 -- start start 2024-03-08T06:02:41.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe75bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe75c15dda0 0x7fe75c0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe75bfff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe75c0cea00 0x7fe75c0cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe75bfff640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe75c06d810 con 0x7fe75c0cfd40 2024-03-08T06:02:41.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe75bfff640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75c15e180 con 0x7fe75c15dda0 2024-03-08T06:02:41.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe75bfff640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c0d0190 con 0x7fe75c0cea00 2024-03-08T06:02:41.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe75b7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe75c0cfd40 0x7fe75c070d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40458/0 (socket says 172.21.15.76:40458) 2024-03-08T06:02:41.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe75b7fe640 1 -- 172.21.15.76:0/49636370 learned_addr learned my addr 172.21.15.76:0/49636370 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:41.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe75a7fc640 1 --2- 172.21.15.76:0/49636370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe75c0cea00 0x7fe75c0cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe75affd640 1 --2- 172.21.15.76:0/49636370 >> v2:172.21.15.81:3300/0 conn(0x7fe75c15dda0 0x7fe75c0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe759ffb640 1 -- 172.21.15.76:0/49636370 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063718194 0 0) 0x7fe75c06d810 con 0x7fe75c0cfd40 2024-03-08T06:02:41.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe759ffb640 1 -- 172.21.15.76:0/49636370 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe744003520 con 0x7fe75c0cfd40 2024-03-08T06:02:41.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe759ffb640 1 -- 172.21.15.76:0/49636370 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2640306658 0 0) 0x7fe744003520 con 0x7fe75c0cfd40 2024-03-08T06:02:41.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe759ffb640 1 -- 172.21.15.76:0/49636370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe75c0cea00 msgr2=0x7fe75c0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:41.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe759ffb640 1 --2- 172.21.15.76:0/49636370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe75c0cea00 0x7fe75c0cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe759ffb640 1 -- 172.21.15.76:0/49636370 >> v2:172.21.15.81:3300/0 conn(0x7fe75c15dda0 msgr2=0x7fe75c0cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:41.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe759ffb640 1 --2- 172.21.15.76:0/49636370 >> v2:172.21.15.81:3300/0 conn(0x7fe75c15dda0 0x7fe75c0cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe759ffb640 1 -- 172.21.15.76:0/49636370 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c161a70 con 0x7fe75c0cfd40 2024-03-08T06:02:41.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.979+0000 7fe759ffb640 1 -- 172.21.15.76:0/49636370 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fe7540019f0 con 0x7fe75c0cfd40 2024-03-08T06:02:41.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.980+0000 7fe759ffb640 1 -- 172.21.15.76:0/49636370 >> v1:172.21.15.76:6789/0 conn(0x7fe75c0cfd40 legacy=0x7fe75c070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.980+0000 7fe759ffb640 1 --2- 172.21.15.76:0/49636370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe75c0cea00 0x7fe7440039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.980+0000 7fe759ffb640 1 --2- 172.21.15.76:0/49636370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe744003f30 0x7fe744004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.980+0000 7fe759ffb640 1 --2- 172.21.15.76:0/49636370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe744004d40 0x7fe7440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.980+0000 7fe759ffb640 1 -- 172.21.15.76:0/49636370 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c15e180 con 0x7fe744004d40 2024-03-08T06:02:41.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.980+0000 7fe759ffb640 1 -- 172.21.15.76:0/49636370 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c0d0190 con 0x7fe744003f30 2024-03-08T06:02:41.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.980+0000 7fe759ffb640 1 -- 172.21.15.76:0/49636370 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74400b9c0 con 0x7fe75c0cea00 2024-03-08T06:02:41.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.980+0000 7fe75b7fe640 1 --2- 172.21.15.76:0/49636370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe744004d40 0x7fe7440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.980+0000 7fe75a7fc640 1 --2- 172.21.15.76:0/49636370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe744003f30 0x7fe744004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.980+0000 7fe75affd640 1 --2- 172.21.15.76:0/49636370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe75c0cea00 0x7fe7440039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.980+0000 7fe75b7fe640 1 -- 172.21.15.76:0/49636370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe75c0cea00 msgr2=0x7fe7440039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.980+0000 7fe75b7fe640 1 --2- 172.21.15.76:0/49636370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe75c0cea00 0x7fe7440039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.980+0000 7fe75b7fe640 1 -- 172.21.15.76:0/49636370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe744003f30 msgr2=0x7fe744004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.980+0000 7fe75b7fe640 1 --2- 172.21.15.76:0/49636370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe744003f30 0x7fe744004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.980+0000 7fe75b7fe640 1 -- 172.21.15.76:0/49636370 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe744003520 con 0x7fe744004d40 2024-03-08T06:02:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.981+0000 7fe75b7fe640 1 --2- 172.21.15.76:0/49636370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe744004d40 0x7fe7440051c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe7540013d0 tx=0x7fe744005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2f99b88b6ee9ba server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.981+0000 7fe759ffb640 1 -- 172.21.15.76:0/49636370 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75400a4e0 con 0x7fe744004d40 2024-03-08T06:02:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.981+0000 7fe759ffb640 1 -- 172.21.15.76:0/49636370 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe754034650 con 0x7fe744004d40 2024-03-08T06:02:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.981+0000 7fe759ffb640 1 -- 172.21.15.76:0/49636370 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754034a40 con 0x7fe744004d40 2024-03-08T06:02:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.981+0000 7fe75bfff640 1 -- 172.21.15.76:0/49636370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe744004d40 msgr2=0x7fe7440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.981+0000 7fe75bfff640 1 --2- 172.21.15.76:0/49636370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe744004d40 0x7fe7440051c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe7540013d0 tx=0x7fe744005830 comp rx=0 tx=0).stop 2024-03-08T06:02:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.981+0000 7fe75b7fe640 1 -- 172.21.15.76:0/49636370 reap_dead start 2024-03-08T06:02:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.981+0000 7fe75bfff640 1 -- 172.21.15.76:0/49636370 shutdown_connections 2024-03-08T06:02:41.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.981+0000 7fe75bfff640 1 -- 172.21.15.76:0/49636370 >> 172.21.15.76:0/49636370 conn(0x7fe75c06f5a0 msgr2=0x7fe75c06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:41.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.981+0000 7fe75bfff640 1 -- 172.21.15.76:0/49636370 shutdown_connections 2024-03-08T06:02:41.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.981+0000 7fe75bfff640 1 -- 172.21.15.76:0/49636370 wait complete. 2024-03-08T06:02:41.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.983+0000 7fe75bfff640 1 Processor -- start 2024-03-08T06:02:41.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.983+0000 7fe75bfff640 1 -- start start 2024-03-08T06:02:41.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.983+0000 7fe75bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe744004d40 0x7fe75c156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.983+0000 7fe75bfff640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe75c0cb510 0x7fe75c160f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.983+0000 7fe75bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75c0b5520 0x7fe75c0b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.983+0000 7fe75bfff640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c162030 con 0x7fe75c0b5520 2024-03-08T06:02:41.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe75bfff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c161820 con 0x7fe744004d40 2024-03-08T06:02:41.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe75bfff640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c162310 con 0x7fe75c0cb510 2024-03-08T06:02:41.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe75b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75c0b5520 0x7fe75c0b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe75b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75c0b5520 0x7fe75c0b59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37050/0 (socket says 172.21.15.76:37050) 2024-03-08T06:02:41.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe75b7fe640 1 -- 172.21.15.76:0/3796681028 learned_addr learned my addr 172.21.15.76:0/3796681028 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:41.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe75affd640 1 --2- 172.21.15.76:0/3796681028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe744004d40 0x7fe75c156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe75a7fc640 1 --2- 172.21.15.76:0/3796681028 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe75c0cb510 0x7fe75c160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe75b7fe640 1 -- 172.21.15.76:0/3796681028 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe75c0cb510 msgr2=0x7fe75c160f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe75b7fe640 1 --2- 172.21.15.76:0/3796681028 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe75c0cb510 0x7fe75c160f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe75b7fe640 1 -- 172.21.15.76:0/3796681028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe744004d40 msgr2=0x7fe75c156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:41.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe75b7fe640 1 --2- 172.21.15.76:0/3796681028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe744004d40 0x7fe75c156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:41.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe75b7fe640 1 -- 172.21.15.76:0/3796681028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c0ca190 con 0x7fe75c0b5520 2024-03-08T06:02:41.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe75affd640 1 --2- 172.21.15.76:0/3796681028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe744004d40 0x7fe75c156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:41.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe75b7fe640 1 --2- 172.21.15.76:0/3796681028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75c0b5520 0x7fe75c0b59a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe754001cf0 tx=0x7fe754001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:41.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe7437fe640 1 -- 172.21.15.76:0/3796681028 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7540019c0 con 0x7fe75c0b5520 2024-03-08T06:02:41.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe7437fe640 1 -- 172.21.15.76:0/3796681028 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75403a040 con 0x7fe75c0b5520 2024-03-08T06:02:41.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe75bfff640 1 -- 172.21.15.76:0/3796681028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75c0ca470 con 0x7fe75c0b5520 2024-03-08T06:02:41.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe7437fe640 1 -- 172.21.15.76:0/3796681028 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754034a00 con 0x7fe75c0b5520 2024-03-08T06:02:41.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.984+0000 7fe75bfff640 1 -- 172.21.15.76:0/3796681028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75c0caa30 con 0x7fe75c0b5520 2024-03-08T06:02:41.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.985+0000 7fe75bfff640 1 -- 172.21.15.76:0/3796681028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75c06a1d0 con 0x7fe75c0b5520 2024-03-08T06:02:41.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.986+0000 7fe7437fe640 1 -- 172.21.15.76:0/3796681028 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe754047020 con 0x7fe75c0b5520 2024-03-08T06:02:41.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.986+0000 7fe7437fe640 1 --2- 172.21.15.76:0/3796681028 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe74806b500 0x7fe74806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:41.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.986+0000 7fe7437fe640 1 -- 172.21.15.76:0/3796681028 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fe7540ac030 con 0x7fe75c0b5520 2024-03-08T06:02:41.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.986+0000 7fe75affd640 1 --2- 172.21.15.76:0/3796681028 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe74806b500 0x7fe74806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:41.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.987+0000 7fe75affd640 1 --2- 172.21.15.76:0/3796681028 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe74806b500 0x7fe74806d9c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0d07b0 tx=0x7fe74c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:41.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:41.990+0000 7fe7437fe640 1 -- 172.21.15.76:0/3796681028 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe75400a580 con 0x7fe75c0b5520 2024-03-08T06:02:42.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.163+0000 7fe75bfff640 1 -- 172.21.15.76:0/3796681028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe75c0cd510 con 0x7fe75c0b5520 2024-03-08T06:02:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.164+0000 7fe7437fe640 1 -- 172.21.15.76:0/3796681028 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16850 (secure 0 0 0) 0x7fe75407ad00 con 0x7fe75c0b5520 2024-03-08T06:02:42.165 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:02:42.165 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":42,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:02:18.021974+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:01:45.628991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-08T06:01:46.195100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:02:42.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.166+0000 7fe75bfff640 1 -- 172.21.15.76:0/3796681028 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe74806b500 msgr2=0x7fe74806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.166+0000 7fe75bfff640 1 --2- 172.21.15.76:0/3796681028 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe74806b500 0x7fe74806d9c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0d07b0 tx=0x7fe74c008040 comp rx=0 tx=0).stop 2024-03-08T06:02:42.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.166+0000 7fe75bfff640 1 -- 172.21.15.76:0/3796681028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75c0b5520 msgr2=0x7fe75c0b59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.166+0000 7fe75bfff640 1 --2- 172.21.15.76:0/3796681028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75c0b5520 0x7fe75c0b59a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe754001cf0 tx=0x7fe754001360 comp rx=0 tx=0).stop 2024-03-08T06:02:42.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.166+0000 7fe75bfff640 1 -- 172.21.15.76:0/3796681028 shutdown_connections 2024-03-08T06:02:42.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.166+0000 7fe75bfff640 1 --2- 172.21.15.76:0/3796681028 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe74806b500 0x7fe74806d9c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.166+0000 7fe75bfff640 1 --2- 172.21.15.76:0/3796681028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75c0b5520 0x7fe75c0b59a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.166+0000 7fe75bfff640 1 --2- 172.21.15.76:0/3796681028 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe75c0cb510 0x7fe75c160f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.166+0000 7fe75bfff640 1 --2- 172.21.15.76:0/3796681028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe744004d40 0x7fe75c156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.166+0000 7fe75bfff640 1 -- 172.21.15.76:0/3796681028 >> 172.21.15.76:0/3796681028 conn(0x7fe75c06f5a0 msgr2=0x7fe75c0b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:42.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.166+0000 7fe75bfff640 1 -- 172.21.15.76:0/3796681028 shutdown_connections 2024-03-08T06:02:42.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.166+0000 7fe75bfff640 1 -- 172.21.15.76:0/3796681028 wait complete. 2024-03-08T06:02:42.178 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:00:23.961657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:01:45.628991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:01:46.195100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:02:42.178 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:02:42.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.281+0000 7f1e7cc84640 1 Processor -- start 2024-03-08T06:02:42.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.281+0000 7f1e7cc84640 1 -- start start 2024-03-08T06:02:42.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.281+0000 7f1e7cc84640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1e780cfbb0 0x7f1e7806b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:42.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.281+0000 7f1e7cc84640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e780cb360 con 0x7f1e7815e1b0 2024-03-08T06:02:42.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.281+0000 7f1e7cc84640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7815e9d0 con 0x7f1e780caee0 2024-03-08T06:02:42.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.281+0000 7f1e7cc84640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7815ecb0 con 0x7f1e780cfbb0 2024-03-08T06:02:42.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.281+0000 7f1e7659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1e780cfbb0 0x7f1e7806b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:42.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.282+0000 7f1e7659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1e780cfbb0 0x7f1e7806b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37060/0 (socket says 172.21.15.76:37060) 2024-03-08T06:02:42.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.282+0000 7f1e76d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1e7815e1b0 0x7f1e780cd4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40460/0 (socket says 172.21.15.76:40460) 2024-03-08T06:02:42.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.282+0000 7f1e7659c640 1 -- 172.21.15.76:0/4076313575 learned_addr learned my addr 172.21.15.76:0/4076313575 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:42.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.282+0000 7f1e7659c640 1 -- 172.21.15.76:0/4076313575 >> v1:172.21.15.81:6789/0 conn(0x7f1e780caee0 legacy=0x7f1e780cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:42.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.282+0000 7f1e7659c640 1 -- 172.21.15.76:0/4076313575 >> v1:172.21.15.76:6789/0 conn(0x7f1e7815e1b0 legacy=0x7f1e780cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.282+0000 7f1e75d9b640 1 -- 172.21.15.76:0/4076313575 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008631399 0 0) 0x7f1e780cb360 con 0x7f1e7815e1b0 2024-03-08T06:02:42.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.282+0000 7f1e7659c640 1 -- 172.21.15.76:0/4076313575 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7815ef90 con 0x7f1e780cfbb0 2024-03-08T06:02:42.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.283+0000 7f1e7659c640 1 --2- 172.21.15.76:0/4076313575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e780cfbb0 0x7f1e7806b690 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009870 tx=0x7f1e6802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88ac2daf9d5c280 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:42.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.283+0000 7f1e75d9b640 1 -- 172.21.15.76:0/4076313575 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68034070 con 0x7f1e780cfbb0 2024-03-08T06:02:42.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.283+0000 7f1e75d9b640 1 -- 172.21.15.76:0/4076313575 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e68030860 con 0x7f1e780cfbb0 2024-03-08T06:02:42.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.283+0000 7f1e75d9b640 1 -- 172.21.15.76:0/4076313575 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68030c30 con 0x7f1e780cfbb0 2024-03-08T06:02:42.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.283+0000 7f1e7cc84640 1 -- 172.21.15.76:0/4076313575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e780cfbb0 msgr2=0x7f1e7806b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.283+0000 7f1e7cc84640 1 --2- 172.21.15.76:0/4076313575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e780cfbb0 0x7f1e7806b690 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009870 tx=0x7f1e6802b9a0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.283+0000 7f1e7cc84640 1 -- 172.21.15.76:0/4076313575 shutdown_connections 2024-03-08T06:02:42.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.283+0000 7f1e7cc84640 1 --2- 172.21.15.76:0/4076313575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e780cfbb0 0x7f1e7806b690 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.283+0000 7f1e7cc84640 1 -- 172.21.15.76:0/4076313575 >> 172.21.15.76:0/4076313575 conn(0x7f1e780d0890 msgr2=0x7f1e780d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:42.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.283+0000 7f1e7cc84640 1 -- 172.21.15.76:0/4076313575 shutdown_connections 2024-03-08T06:02:42.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.283+0000 7f1e7cc84640 1 -- 172.21.15.76:0/4076313575 wait complete. 2024-03-08T06:02:42.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.285+0000 7f1e7cc84640 1 Processor -- start 2024-03-08T06:02:42.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.285+0000 7f1e7cc84640 1 -- start start 2024-03-08T06:02:42.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.285+0000 7f1e7cc84640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1e780caee0 0x7f1e780c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:42.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.285+0000 7f1e7cc84640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e780cfbb0 0x7f1e781f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:42.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.285+0000 7f1e7cc84640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e7815e1b0 0x7f1e7815c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:42.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.285+0000 7f1e7cc84640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780ccb70 con 0x7f1e780cfbb0 2024-03-08T06:02:42.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.285+0000 7f1e7cc84640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7815e660 con 0x7f1e7815e1b0 2024-03-08T06:02:42.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.285+0000 7f1e7cc84640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780cccf0 con 0x7f1e780caee0 2024-03-08T06:02:42.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.285+0000 7f1e6ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e780cfbb0 0x7f1e781f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:42.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.285+0000 7f1e6ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e780cfbb0 0x7f1e781f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37072/0 (socket says 172.21.15.76:37072) 2024-03-08T06:02:42.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.285+0000 7f1e6ffff640 1 -- 172.21.15.76:0/2493271822 learned_addr learned my addr 172.21.15.76:0/2493271822 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:42.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.286+0000 7f1e7659c640 1 --2- 172.21.15.76:0/2493271822 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1e780caee0 0x7f1e780c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:42.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.286+0000 7f1e76d9d640 1 --2- 172.21.15.76:0/2493271822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e7815e1b0 0x7f1e7815c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:42.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.286+0000 7f1e6ffff640 1 -- 172.21.15.76:0/2493271822 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1e780caee0 msgr2=0x7f1e780c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.286+0000 7f1e6ffff640 1 --2- 172.21.15.76:0/2493271822 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1e780caee0 0x7f1e780c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.286+0000 7f1e6ffff640 1 -- 172.21.15.76:0/2493271822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e7815e1b0 msgr2=0x7f1e7815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.286+0000 7f1e6ffff640 1 --2- 172.21.15.76:0/2493271822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e7815e1b0 0x7f1e7815c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.286+0000 7f1e6ffff640 1 -- 172.21.15.76:0/2493271822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e780c6050 con 0x7f1e780cfbb0 2024-03-08T06:02:42.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.286+0000 7f1e6ffff640 1 --2- 172.21.15.76:0/2493271822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e780cfbb0 0x7f1e781f4bb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000c430 tx=0x7f1e6000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:42.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.286+0000 7f1e6effd640 1 -- 172.21.15.76:0/2493271822 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60011790 con 0x7f1e780cfbb0 2024-03-08T06:02:42.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.286+0000 7f1e6effd640 1 -- 172.21.15.76:0/2493271822 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e60017070 con 0x7f1e780cfbb0 2024-03-08T06:02:42.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.286+0000 7f1e6effd640 1 -- 172.21.15.76:0/2493271822 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60011b70 con 0x7f1e780cfbb0 2024-03-08T06:02:42.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.286+0000 7f1e7cc84640 1 -- 172.21.15.76:0/2493271822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e780c6330 con 0x7f1e780cfbb0 2024-03-08T06:02:42.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.286+0000 7f1e7cc84640 1 -- 172.21.15.76:0/2493271822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e780c68f0 con 0x7f1e780cfbb0 2024-03-08T06:02:42.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.287+0000 7f1e7cc84640 1 -- 172.21.15.76:0/2493271822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7804f030 con 0x7f1e780cfbb0 2024-03-08T06:02:42.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.288+0000 7f1e6effd640 1 -- 172.21.15.76:0/2493271822 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e60011d10 con 0x7f1e780cfbb0 2024-03-08T06:02:42.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.288+0000 7f1e6effd640 1 --2- 172.21.15.76:0/2493271822 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1e5806b4d0 0x7f1e5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:42.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.288+0000 7f1e6effd640 1 -- 172.21.15.76:0/2493271822 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f1e6008a050 con 0x7f1e780cfbb0 2024-03-08T06:02:42.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.288+0000 7f1e7659c640 1 --2- 172.21.15.76:0/2493271822 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1e5806b4d0 0x7f1e5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:42.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.288+0000 7f1e7659c640 1 --2- 172.21.15.76:0/2493271822 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1e5806b4d0 0x7f1e5806d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1e68036000 tx=0x7f1e68008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:42.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.292+0000 7f1e6effd640 1 -- 172.21.15.76:0/2493271822 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1e600586d0 con 0x7f1e780cfbb0 2024-03-08T06:02:42.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.461+0000 7f1e7cc84640 1 -- 172.21.15.76:0/2493271822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e780cb360 con 0x7f1e780cfbb0 2024-03-08T06:02:42.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.462+0000 7f1e6effd640 1 -- 172.21.15.76:0/2493271822 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e6005c770 con 0x7f1e780cfbb0 2024-03-08T06:02:42.462 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-08T06:02:42.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.464+0000 7f1e7cc84640 1 -- 172.21.15.76:0/2493271822 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1e5806b4d0 msgr2=0x7f1e5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.464+0000 7f1e7cc84640 1 --2- 172.21.15.76:0/2493271822 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1e5806b4d0 0x7f1e5806d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1e68036000 tx=0x7f1e68008030 comp rx=0 tx=0).stop 2024-03-08T06:02:42.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.464+0000 7f1e7cc84640 1 -- 172.21.15.76:0/2493271822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e780cfbb0 msgr2=0x7f1e781f4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.464+0000 7f1e7cc84640 1 --2- 172.21.15.76:0/2493271822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e780cfbb0 0x7f1e781f4bb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000c430 tx=0x7f1e6000c8f0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.464+0000 7f1e7cc84640 1 -- 172.21.15.76:0/2493271822 shutdown_connections 2024-03-08T06:02:42.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.464+0000 7f1e7cc84640 1 --2- 172.21.15.76:0/2493271822 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f1e5806b4d0 0x7f1e5806d990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.464+0000 7f1e7cc84640 1 --2- 172.21.15.76:0/2493271822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e7815e1b0 0x7f1e7815c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.464+0000 7f1e7cc84640 1 --2- 172.21.15.76:0/2493271822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e780cfbb0 0x7f1e781f4bb0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.464+0000 7f1e7cc84640 1 --2- 172.21.15.76:0/2493271822 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1e780caee0 0x7f1e780c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.464+0000 7f1e7cc84640 1 -- 172.21.15.76:0/2493271822 >> 172.21.15.76:0/2493271822 conn(0x7f1e780d0890 msgr2=0x7f1e780cd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:42.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.464+0000 7f1e7cc84640 1 -- 172.21.15.76:0/2493271822 shutdown_connections 2024-03-08T06:02:42.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.464+0000 7f1e7cc84640 1 -- 172.21.15.76:0/2493271822 wait complete. 2024-03-08T06:02:42.475 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:02:42.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.577+0000 7f7b78c78640 1 Processor -- start 2024-03-08T06:02:42.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.578+0000 7f7b78c78640 1 -- start start 2024-03-08T06:02:42.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.578+0000 7f7b78c78640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7b740b3c50 0x7f7b740b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:42.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.578+0000 7f7b78c78640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b7406b0f0 0x7f7b740adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:42.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.578+0000 7f7b78c78640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b740b4570 con 0x7f7b7406ac80 2024-03-08T06:02:42.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.578+0000 7f7b78c78640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b7406b720 con 0x7f7b740b3c50 2024-03-08T06:02:42.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.578+0000 7f7b78c78640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7406ba00 con 0x7f7b7406b0f0 2024-03-08T06:02:42.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.579+0000 7f7b72d9d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b7406b0f0 0x7f7b740adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:42.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.579+0000 7f7b7259c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7b740b3c50 0x7f7b740b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:42.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.579+0000 7f7b6bfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f7b7406ac80 0x7f7b7406f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.76:60830/0 (socket says 172.21.15.76:60830) 2024-03-08T06:02:42.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.579+0000 7f7b6bfff640 1 -- 172.21.15.76:0/2321346605 learned_addr learned my addr 172.21.15.76:0/2321346605 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:42.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.579+0000 7f7b7259c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7b740b3c50 0x7f7b740b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.76:55158/0 (socket says 172.21.15.76:55158) 2024-03-08T06:02:42.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.579+0000 7f7b72d9d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b7406b0f0 0x7f7b740adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:38498/0 (socket says 172.21.15.76:38498) 2024-03-08T06:02:42.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.579+0000 7f7b72d9d640 1 -- 172.21.15.76:0/2321346605 >> v2:172.21.15.81:3300/0 conn(0x7f7b740b3c50 msgr2=0x7f7b740b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:42.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.579+0000 7f7b72d9d640 1 --2- 172.21.15.76:0/2321346605 >> v2:172.21.15.81:3300/0 conn(0x7f7b740b3c50 0x7f7b740b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.579+0000 7f7b72d9d640 1 -- 172.21.15.76:0/2321346605 >> v1:172.21.15.81:6789/0 conn(0x7f7b7406ac80 legacy=0x7f7b7406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.579+0000 7f7b72d9d640 1 -- 172.21.15.76:0/2321346605 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7406bce0 con 0x7f7b7406b0f0 2024-03-08T06:02:42.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.580+0000 7f7b72d9d640 1 --2- 172.21.15.76:0/2321346605 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b7406b0f0 0x7f7b740adea0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009dc0 tx=0x7f7b6c02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=814c991c928027e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:42.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.580+0000 7f7b71d9b640 1 -- 172.21.15.76:0/2321346605 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c039070 con 0x7f7b7406b0f0 2024-03-08T06:02:42.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.580+0000 7f7b71d9b640 1 -- 172.21.15.76:0/2321346605 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6c02cc40 con 0x7f7b7406b0f0 2024-03-08T06:02:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.580+0000 7f7b71d9b640 1 -- 172.21.15.76:0/2321346605 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c034440 con 0x7f7b7406b0f0 2024-03-08T06:02:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.580+0000 7f7b78c78640 1 -- 172.21.15.76:0/2321346605 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b7406b0f0 msgr2=0x7f7b740adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.580+0000 7f7b78c78640 1 --2- 172.21.15.76:0/2321346605 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b7406b0f0 0x7f7b740adea0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009dc0 tx=0x7f7b6c02be90 comp rx=0 tx=0).stop 2024-03-08T06:02:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.580+0000 7f7b78c78640 1 -- 172.21.15.76:0/2321346605 shutdown_connections 2024-03-08T06:02:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.580+0000 7f7b78c78640 1 --2- 172.21.15.76:0/2321346605 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b7406b0f0 0x7f7b740adea0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.580+0000 7f7b78c78640 1 --2- 172.21.15.76:0/2321346605 >> v2:172.21.15.81:3300/0 conn(0x7f7b740b3c50 0x7f7b740b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.580+0000 7f7b78c78640 1 -- 172.21.15.76:0/2321346605 >> 172.21.15.76:0/2321346605 conn(0x7f7b7406dd40 msgr2=0x7f7b7406e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.581+0000 7f7b78c78640 1 -- 172.21.15.76:0/2321346605 shutdown_connections 2024-03-08T06:02:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.581+0000 7f7b78c78640 1 -- 172.21.15.76:0/2321346605 wait complete. 2024-03-08T06:02:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.582+0000 7f7b78c78640 1 Processor -- start 2024-03-08T06:02:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.582+0000 7f7b78c78640 1 -- start start 2024-03-08T06:02:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.582+0000 7f7b78c78640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b7406ac80 0x7f7b741f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.582+0000 7f7b78c78640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b7406b0f0 0x7f7b74151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b78c78640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b740b3c50 0x7f7b74151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b78c78640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b74070500 con 0x7f7b740b3c50 2024-03-08T06:02:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b78c78640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7406c2a0 con 0x7f7b7406ac80 2024-03-08T06:02:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b78c78640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741612c0 con 0x7f7b7406b0f0 2024-03-08T06:02:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b72d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b740b3c50 0x7f7b74151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b72d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b740b3c50 0x7f7b74151660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37078/0 (socket says 172.21.15.76:37078) 2024-03-08T06:02:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b72d9d640 1 -- 172.21.15.76:0/3906051607 learned_addr learned my addr 172.21.15.76:0/3906051607 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b6bfff640 1 --2- 172.21.15.76:0/3906051607 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b7406b0f0 0x7f7b74151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b7259c640 1 --2- 172.21.15.76:0/3906051607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b7406ac80 0x7f7b741f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b72d9d640 1 -- 172.21.15.76:0/3906051607 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b7406b0f0 msgr2=0x7f7b74151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b72d9d640 1 --2- 172.21.15.76:0/3906051607 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b7406b0f0 0x7f7b74151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b72d9d640 1 -- 172.21.15.76:0/3906051607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b7406ac80 msgr2=0x7f7b741f2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b72d9d640 1 --2- 172.21.15.76:0/3906051607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b7406ac80 0x7f7b741f2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b72d9d640 1 -- 172.21.15.76:0/3906051607 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b740cc300 con 0x7f7b740b3c50 2024-03-08T06:02:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b72d9d640 1 --2- 172.21.15.76:0/3906051607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b740b3c50 0x7f7b74151660 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009dc0 tx=0x7f7b6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b6affd640 1 -- 172.21.15.76:0/3906051607 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c039050 con 0x7f7b740b3c50 2024-03-08T06:02:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b6affd640 1 -- 172.21.15.76:0/3906051607 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6c045070 con 0x7f7b740b3c50 2024-03-08T06:02:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b78c78640 1 -- 172.21.15.76:0/3906051607 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b740cd330 con 0x7f7b740b3c50 2024-03-08T06:02:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b6affd640 1 -- 172.21.15.76:0/3906051607 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c009450 con 0x7f7b740b3c50 2024-03-08T06:02:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.583+0000 7f7b78c78640 1 -- 172.21.15.76:0/3906051607 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b740cc850 con 0x7f7b740b3c50 2024-03-08T06:02:42.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.585+0000 7f7b78c78640 1 -- 172.21.15.76:0/3906051607 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b7404f030 con 0x7f7b740b3c50 2024-03-08T06:02:42.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.585+0000 7f7b6affd640 1 -- 172.21.15.76:0/3906051607 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b6c040670 con 0x7f7b740b3c50 2024-03-08T06:02:42.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.585+0000 7f7b6affd640 1 --2- 172.21.15.76:0/3906051607 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7b5406b4d0 0x7f7b5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:42.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.585+0000 7f7b6affd640 1 -- 172.21.15.76:0/3906051607 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f7b6c0accb0 con 0x7f7b740b3c50 2024-03-08T06:02:42.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.585+0000 7f7b7259c640 1 --2- 172.21.15.76:0/3906051607 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7b5406b4d0 0x7f7b5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:42.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.586+0000 7f7b7259c640 1 --2- 172.21.15.76:0/3906051607 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7b5406b4d0 0x7f7b5406d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009aa0 tx=0x7f7b64009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:42.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.590+0000 7f7b6affd640 1 -- 172.21.15.76:0/3906051607 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b6c07b3c0 con 0x7f7b740b3c50 2024-03-08T06:02:42.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.759+0000 7f7b78c78640 1 -- 172.21.15.76:0/3906051607 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7b741f2fe0 con 0x7f7b740b3c50 2024-03-08T06:02:42.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.759+0000 7f7b6affd640 1 -- 172.21.15.76:0/3906051607 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7b6c07f460 con 0x7f7b740b3c50 2024-03-08T06:02:42.760 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-08T06:02:42.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.761+0000 7f7b78c78640 1 -- 172.21.15.76:0/3906051607 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7b5406b4d0 msgr2=0x7f7b5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.761+0000 7f7b78c78640 1 --2- 172.21.15.76:0/3906051607 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7b5406b4d0 0x7f7b5406d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009aa0 tx=0x7f7b64009190 comp rx=0 tx=0).stop 2024-03-08T06:02:42.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.761+0000 7f7b78c78640 1 -- 172.21.15.76:0/3906051607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b740b3c50 msgr2=0x7f7b74151660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.761+0000 7f7b78c78640 1 --2- 172.21.15.76:0/3906051607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b740b3c50 0x7f7b74151660 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009dc0 tx=0x7f7b6c004a40 comp rx=0 tx=0).stop 2024-03-08T06:02:42.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.761+0000 7f7b78c78640 1 -- 172.21.15.76:0/3906051607 shutdown_connections 2024-03-08T06:02:42.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.761+0000 7f7b78c78640 1 --2- 172.21.15.76:0/3906051607 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7b5406b4d0 0x7f7b5406d990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.761+0000 7f7b78c78640 1 --2- 172.21.15.76:0/3906051607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b740b3c50 0x7f7b74151660 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.761+0000 7f7b78c78640 1 --2- 172.21.15.76:0/3906051607 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7b7406b0f0 0x7f7b74151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.761+0000 7f7b78c78640 1 --2- 172.21.15.76:0/3906051607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b7406ac80 0x7f7b741f2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.761+0000 7f7b78c78640 1 -- 172.21.15.76:0/3906051607 >> 172.21.15.76:0/3906051607 conn(0x7f7b7406dd40 msgr2=0x7f7b7406f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:42.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.761+0000 7f7b78c78640 1 -- 172.21.15.76:0/3906051607 shutdown_connections 2024-03-08T06:02:42.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.761+0000 7f7b78c78640 1 -- 172.21.15.76:0/3906051607 wait complete. 2024-03-08T06:02:42.773 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:02:42.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.876+0000 7fe03ec83640 1 Processor -- start 2024-03-08T06:02:42.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.876+0000 7fe03ec83640 1 -- start start 2024-03-08T06:02:42.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.877+0000 7fe03ec83640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe03806d8a0 0x7fe038162020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:42.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.877+0000 7fe03ec83640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe038161840 con 0x7fe0380ce5e0 2024-03-08T06:02:42.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.877+0000 7fe03ec83640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03815eb30 con 0x7fe038162560 2024-03-08T06:02:42.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.877+0000 7fe03ec83640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe038161b90 con 0x7fe03806d8a0 2024-03-08T06:02:42.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.877+0000 7fe03ca1f640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe0380ce5e0 0x7fe0380ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40468/0 (socket says 172.21.15.76:40468) 2024-03-08T06:02:42.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.877+0000 7fe03ca1f640 1 -- 172.21.15.76:0/4268290201 learned_addr learned my addr 172.21.15.76:0/4268290201 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:42.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.877+0000 7fe037fff640 1 --2- 172.21.15.76:0/4268290201 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe03806d8a0 0x7fe038162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:42.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.877+0000 7fe0377fe640 1 -- 172.21.15.76:0/4268290201 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4163030805 0 0) 0x7fe038161840 con 0x7fe0380ce5e0 2024-03-08T06:02:42.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.877+0000 7fe0377fe640 1 -- 172.21.15.76:0/4268290201 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe028003520 con 0x7fe0380ce5e0 2024-03-08T06:02:42.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.877+0000 7fe037fff640 1 -- 172.21.15.76:0/4268290201 >> v1:172.21.15.81:6789/0 conn(0x7fe038162560 legacy=0x7fe03815dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.877+0000 7fe0377fe640 1 -- 172.21.15.76:0/4268290201 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596511723 0 0) 0x7fe03815eb30 con 0x7fe038162560 2024-03-08T06:02:42.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.877+0000 7fe037fff640 1 -- 172.21.15.76:0/4268290201 >> v1:172.21.15.76:6789/0 conn(0x7fe0380ce5e0 legacy=0x7fe0380ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.877+0000 7fe037fff640 1 -- 172.21.15.76:0/4268290201 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03815ee10 con 0x7fe03806d8a0 2024-03-08T06:02:42.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.878+0000 7fe037fff640 1 --2- 172.21.15.76:0/4268290201 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe03806d8a0 0x7fe038162020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe0240097d0 tx=0x7fe02402b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dfe9126afdeb3c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:42.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.878+0000 7fe0377fe640 1 -- 172.21.15.76:0/4268290201 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024037070 con 0x7fe03806d8a0 2024-03-08T06:02:42.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.878+0000 7fe0377fe640 1 -- 172.21.15.76:0/4268290201 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe024030510 con 0x7fe03806d8a0 2024-03-08T06:02:42.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.878+0000 7fe0377fe640 1 -- 172.21.15.76:0/4268290201 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c001670 con 0x7fe03806d8a0 2024-03-08T06:02:42.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.879+0000 7fe03ec83640 1 -- 172.21.15.76:0/4268290201 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe03806d8a0 msgr2=0x7fe038162020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.879+0000 7fe03ec83640 1 --2- 172.21.15.76:0/4268290201 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe03806d8a0 0x7fe038162020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe0240097d0 tx=0x7fe02402b7b0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.879+0000 7fe03ec83640 1 -- 172.21.15.76:0/4268290201 shutdown_connections 2024-03-08T06:02:42.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.879+0000 7fe03ec83640 1 --2- 172.21.15.76:0/4268290201 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe03806d8a0 0x7fe038162020 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.879+0000 7fe03ec83640 1 -- 172.21.15.76:0/4268290201 >> 172.21.15.76:0/4268290201 conn(0x7fe03806ae60 msgr2=0x7fe03806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:42.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.879+0000 7fe03ec83640 1 -- 172.21.15.76:0/4268290201 shutdown_connections 2024-03-08T06:02:42.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.879+0000 7fe03ec83640 1 -- 172.21.15.76:0/4268290201 wait complete. 2024-03-08T06:02:42.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.880+0000 7fe03ec83640 1 Processor -- start 2024-03-08T06:02:42.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.880+0000 7fe03ec83640 1 -- start start 2024-03-08T06:02:42.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03ec83640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe03806d8a0 0x7fe038158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:42.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03ec83640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0380ce5e0 0x7fe038158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:42.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03ec83640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe038162560 0x7fe0381f7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:42.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03ec83640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03815e840 con 0x7fe038162560 2024-03-08T06:02:42.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03ec83640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03815e6c0 con 0x7fe03806d8a0 2024-03-08T06:02:42.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03ec83640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03806bfa0 con 0x7fe0380ce5e0 2024-03-08T06:02:42.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03d220640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe038162560 0x7fe0381f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:42.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03d220640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe038162560 0x7fe0381f7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37090/0 (socket says 172.21.15.76:37090) 2024-03-08T06:02:42.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03d220640 1 -- 172.21.15.76:0/2656114280 learned_addr learned my addr 172.21.15.76:0/2656114280 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:42.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03ca1f640 1 --2- 172.21.15.76:0/2656114280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe03806d8a0 0x7fe038158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:42.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe037fff640 1 --2- 172.21.15.76:0/2656114280 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0380ce5e0 0x7fe038158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:42.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03d220640 1 -- 172.21.15.76:0/2656114280 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0380ce5e0 msgr2=0x7fe038158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03d220640 1 --2- 172.21.15.76:0/2656114280 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0380ce5e0 0x7fe038158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03d220640 1 -- 172.21.15.76:0/2656114280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe03806d8a0 msgr2=0x7fe038158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:42.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03d220640 1 --2- 172.21.15.76:0/2656114280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe03806d8a0 0x7fe038158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:42.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03d220640 1 -- 172.21.15.76:0/2656114280 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0380b72e0 con 0x7fe038162560 2024-03-08T06:02:42.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03ca1f640 1 --2- 172.21.15.76:0/2656114280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe03806d8a0 0x7fe038158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:42.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.881+0000 7fe03d220640 1 --2- 172.21.15.76:0/2656114280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe038162560 0x7fe0381f7520 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe03000c7b0 tx=0x7fe03000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:42.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.882+0000 7fe0357fa640 1 -- 172.21.15.76:0/2656114280 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030003ba0 con 0x7fe038162560 2024-03-08T06:02:42.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.882+0000 7fe0357fa640 1 -- 172.21.15.76:0/2656114280 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe030003d40 con 0x7fe038162560 2024-03-08T06:02:42.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.882+0000 7fe0357fa640 1 -- 172.21.15.76:0/2656114280 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030005270 con 0x7fe038162560 2024-03-08T06:02:42.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.882+0000 7fe03ec83640 1 -- 172.21.15.76:0/2656114280 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0380cd130 con 0x7fe038162560 2024-03-08T06:02:42.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.882+0000 7fe03ec83640 1 -- 172.21.15.76:0/2656114280 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0380cd6f0 con 0x7fe038162560 2024-03-08T06:02:42.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.883+0000 7fe03ec83640 1 -- 172.21.15.76:0/2656114280 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03804f030 con 0x7fe038162560 2024-03-08T06:02:42.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.883+0000 7fe0357fa640 1 -- 172.21.15.76:0/2656114280 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe030005410 con 0x7fe038162560 2024-03-08T06:02:42.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.883+0000 7fe0357fa640 1 --2- 172.21.15.76:0/2656114280 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe01c06b4d0 0x7fe01c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:42.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.883+0000 7fe0357fa640 1 -- 172.21.15.76:0/2656114280 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fe03008a2b0 con 0x7fe038162560 2024-03-08T06:02:42.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.883+0000 7fe03ca1f640 1 --2- 172.21.15.76:0/2656114280 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe01c06b4d0 0x7fe01c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:42.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.884+0000 7fe03ca1f640 1 --2- 172.21.15.76:0/2656114280 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe01c06b4d0 0x7fe01c06d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe02c001700 tx=0x7fe02c001a70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:42.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:42.888+0000 7fe0357fa640 1 -- 172.21.15.76:0/2656114280 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe030058f90 con 0x7fe038162560 2024-03-08T06:02:43.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.056+0000 7fe03ec83640 1 -- 172.21.15.76:0/2656114280 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe03815c9f0 con 0x7fe038162560 2024-03-08T06:02:43.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.057+0000 7fe0357fa640 1 -- 172.21.15.76:0/2656114280 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe030058890 con 0x7fe038162560 2024-03-08T06:02:43.057 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-08T06:02:43.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.058+0000 7fe03ec83640 1 -- 172.21.15.76:0/2656114280 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe01c06b4d0 msgr2=0x7fe01c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:43.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.058+0000 7fe03ec83640 1 --2- 172.21.15.76:0/2656114280 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe01c06b4d0 0x7fe01c06d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe02c001700 tx=0x7fe02c001a70 comp rx=0 tx=0).stop 2024-03-08T06:02:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.058+0000 7fe03ec83640 1 -- 172.21.15.76:0/2656114280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe038162560 msgr2=0x7fe0381f7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.058+0000 7fe03ec83640 1 --2- 172.21.15.76:0/2656114280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe038162560 0x7fe0381f7520 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe03000c7b0 tx=0x7fe03000cbb0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.059+0000 7fe03ec83640 1 -- 172.21.15.76:0/2656114280 shutdown_connections 2024-03-08T06:02:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.059+0000 7fe03ec83640 1 --2- 172.21.15.76:0/2656114280 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe01c06b4d0 0x7fe01c06d990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.059+0000 7fe03ec83640 1 --2- 172.21.15.76:0/2656114280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe038162560 0x7fe0381f7520 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.059+0000 7fe03ec83640 1 --2- 172.21.15.76:0/2656114280 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0380ce5e0 0x7fe038158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.059+0000 7fe03ec83640 1 --2- 172.21.15.76:0/2656114280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe03806d8a0 0x7fe038158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.059+0000 7fe03ec83640 1 -- 172.21.15.76:0/2656114280 >> 172.21.15.76:0/2656114280 conn(0x7fe03806ae60 msgr2=0x7fe0380d1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.059+0000 7fe03ec83640 1 -- 172.21.15.76:0/2656114280 shutdown_connections 2024-03-08T06:02:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.059+0000 7fe03ec83640 1 -- 172.21.15.76:0/2656114280 wait complete. 2024-03-08T06:02:43.070 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:02:43.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.170+0000 7fe9bf807640 1 Processor -- start 2024-03-08T06:02:43.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.170+0000 7fe9bf807640 1 -- start start 2024-03-08T06:02:43.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.171+0000 7fe9bf807640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe9b80cfa90 0x7fe9b815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.171+0000 7fe9bf807640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe9b8071c70 0x7fe9b8072090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.171+0000 7fe9bf807640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b80725d0 con 0x7fe9b806c2e0 2024-03-08T06:02:43.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.171+0000 7fe9bf807640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b80cfe70 con 0x7fe9b8071c70 2024-03-08T06:02:43.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.171+0000 7fe9bf807640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b8158670 con 0x7fe9b80cfa90 2024-03-08T06:02:43.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.171+0000 7fe9bdda4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe9b8071c70 0x7fe9b8072090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.171+0000 7fe9bdda4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe9b8071c70 0x7fe9b8072090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37106/0 (socket says 172.21.15.76:37106) 2024-03-08T06:02:43.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.171+0000 7fe9bdda4640 1 -- 172.21.15.76:0/1572907041 learned_addr learned my addr 172.21.15.76:0/1572907041 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:43.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.171+0000 7fe9bdda4640 1 -- 172.21.15.76:0/1572907041 >> v2:172.21.15.81:3300/0 conn(0x7fe9b80cfa90 msgr2=0x7fe9b815ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:43.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.171+0000 7fe9bd5a3640 1 --2- 172.21.15.76:0/1572907041 >> v2:172.21.15.81:3300/0 conn(0x7fe9b80cfa90 0x7fe9b815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.171+0000 7fe9bdda4640 1 --2- 172.21.15.76:0/1572907041 >> v2:172.21.15.81:3300/0 conn(0x7fe9b80cfa90 0x7fe9b815ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.171+0000 7fe9bdda4640 1 -- 172.21.15.76:0/1572907041 >> v1:172.21.15.81:6789/0 conn(0x7fe9b806c2e0 legacy=0x7fe9b806ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:43.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.171+0000 7fe9bdda4640 1 -- 172.21.15.76:0/1572907041 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b806b390 con 0x7fe9b8071c70 2024-03-08T06:02:43.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.171+0000 7fe9bd5a3640 1 --2- 172.21.15.76:0/1572907041 >> v2:172.21.15.81:3300/0 conn(0x7fe9b80cfa90 0x7fe9b815ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:43.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.172+0000 7fe9bdda4640 1 --2- 172.21.15.76:0/1572907041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9b8071c70 0x7fe9b8072090 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe9b809dec0 tx=0x7fe9b002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52467cd4da426f90 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:43.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.172+0000 7fe9affff640 1 -- 172.21.15.76:0/1572907041 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b002ca80 con 0x7fe9b8071c70 2024-03-08T06:02:43.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.172+0000 7fe9affff640 1 -- 172.21.15.76:0/1572907041 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b002cc20 con 0x7fe9b8071c70 2024-03-08T06:02:43.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.172+0000 7fe9affff640 1 -- 172.21.15.76:0/1572907041 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00343f0 con 0x7fe9b8071c70 2024-03-08T06:02:43.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.173+0000 7fe9bf807640 1 -- 172.21.15.76:0/1572907041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9b8071c70 msgr2=0x7fe9b8072090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:43.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.173+0000 7fe9bf807640 1 --2- 172.21.15.76:0/1572907041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9b8071c70 0x7fe9b8072090 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe9b809dec0 tx=0x7fe9b002bee0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.173+0000 7fe9bf807640 1 -- 172.21.15.76:0/1572907041 shutdown_connections 2024-03-08T06:02:43.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.173+0000 7fe9bf807640 1 --2- 172.21.15.76:0/1572907041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9b8071c70 0x7fe9b8072090 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.173+0000 7fe9bf807640 1 --2- 172.21.15.76:0/1572907041 >> v2:172.21.15.81:3300/0 conn(0x7fe9b80cfa90 0x7fe9b815ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.173+0000 7fe9bf807640 1 -- 172.21.15.76:0/1572907041 >> 172.21.15.76:0/1572907041 conn(0x7fe9b80af650 msgr2=0x7fe9b80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:43.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.173+0000 7fe9bf807640 1 -- 172.21.15.76:0/1572907041 shutdown_connections 2024-03-08T06:02:43.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.173+0000 7fe9bf807640 1 -- 172.21.15.76:0/1572907041 wait complete. 2024-03-08T06:02:43.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.174+0000 7fe9bf807640 1 Processor -- start 2024-03-08T06:02:43.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.174+0000 7fe9bf807640 1 -- start start 2024-03-08T06:02:43.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bf807640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b806c2e0 0x7fe9b81467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bf807640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe9b8071c70 0x7fe9b8144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bf807640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9b80cfa90 0x7fe9b80b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bf807640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b806dd40 con 0x7fe9b80cfa90 2024-03-08T06:02:43.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bf807640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b80ad090 con 0x7fe9b806c2e0 2024-03-08T06:02:43.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bf807640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b806e020 con 0x7fe9b8071c70 2024-03-08T06:02:43.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bdda4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9b80cfa90 0x7fe9b80b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bdda4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9b80cfa90 0x7fe9b80b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37110/0 (socket says 172.21.15.76:37110) 2024-03-08T06:02:43.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bdda4640 1 -- 172.21.15.76:0/2643177372 learned_addr learned my addr 172.21.15.76:0/2643177372 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:43.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bcda2640 1 --2- 172.21.15.76:0/2643177372 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe9b8071c70 0x7fe9b8144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bd5a3640 1 --2- 172.21.15.76:0/2643177372 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b806c2e0 0x7fe9b81467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bdda4640 1 -- 172.21.15.76:0/2643177372 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe9b8071c70 msgr2=0x7fe9b8144ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:43.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bdda4640 1 --2- 172.21.15.76:0/2643177372 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe9b8071c70 0x7fe9b8144ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bdda4640 1 -- 172.21.15.76:0/2643177372 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b806c2e0 msgr2=0x7fe9b81467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:43.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bdda4640 1 --2- 172.21.15.76:0/2643177372 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b806c2e0 0x7fe9b81467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bdda4640 1 -- 172.21.15.76:0/2643177372 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b8146d00 con 0x7fe9b80cfa90 2024-03-08T06:02:43.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bd5a3640 1 --2- 172.21.15.76:0/2643177372 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b806c2e0 0x7fe9b81467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:43.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bdda4640 1 --2- 172.21.15.76:0/2643177372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9b80cfa90 0x7fe9b80b3fc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0036040 tx=0x7fe9b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:43.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9adffb640 1 -- 172.21.15.76:0/2643177372 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0043020 con 0x7fe9b80cfa90 2024-03-08T06:02:43.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9adffb640 1 -- 172.21.15.76:0/2643177372 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b003e450 con 0x7fe9b80cfa90 2024-03-08T06:02:43.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9bf807640 1 -- 172.21.15.76:0/2643177372 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b8145720 con 0x7fe9b80cfa90 2024-03-08T06:02:43.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.175+0000 7fe9adffb640 1 -- 172.21.15.76:0/2643177372 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b003e860 con 0x7fe9b80cfa90 2024-03-08T06:02:43.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.176+0000 7fe9bf807640 1 -- 172.21.15.76:0/2643177372 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b8145ce0 con 0x7fe9b80cfa90 2024-03-08T06:02:43.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.176+0000 7fe9bf807640 1 -- 172.21.15.76:0/2643177372 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b804f030 con 0x7fe9b80cfa90 2024-03-08T06:02:43.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.177+0000 7fe9adffb640 1 -- 172.21.15.76:0/2643177372 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9b0046020 con 0x7fe9b80cfa90 2024-03-08T06:02:43.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.177+0000 7fe9adffb640 1 --2- 172.21.15.76:0/2643177372 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe99c06b500 0x7fe99c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.177+0000 7fe9adffb640 1 -- 172.21.15.76:0/2643177372 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fe9b00ac9d0 con 0x7fe9b80cfa90 2024-03-08T06:02:43.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.177+0000 7fe9bd5a3640 1 --2- 172.21.15.76:0/2643177372 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe99c06b500 0x7fe99c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.178+0000 7fe9bd5a3640 1 --2- 172.21.15.76:0/2643177372 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe99c06b500 0x7fe99c06d9c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400a990 tx=0x7fe9b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:43.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.182+0000 7fe9adffb640 1 -- 172.21.15.76:0/2643177372 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9b004d070 con 0x7fe9b80cfa90 2024-03-08T06:02:43.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.387+0000 7fe9bf807640 1 -- 172.21.15.76:0/2643177372 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9b804f030 con 0x7fe9b80cfa90 2024-03-08T06:02:43.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.387+0000 7fe9adffb640 1 -- 172.21.15.76:0/2643177372 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4932 (secure 0 0 0) 0x7fe9b007b610 con 0x7fe9b80cfa90 2024-03-08T06:02:43.388 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:02:43.388 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/606148582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":606148582},{"type":"v1","addr":"172.21.15.81:6835","nonce":606148582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/278988477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":278988477},{"type":"v1","addr":"172.21.15.76:6836","nonce":278988477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2526423312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":2526423312},{"type":"v1","addr":"172.21.15.76:6837","nonce":2526423312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:01:47.232184+0000","modified":"2024-03-08T06:01:49.227002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5358":{"gid":5358,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6834/1503895376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1503895376},{"type":"v1","addr":"172.21.15.81:6834","nonce":1503895376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T06:02:43.389 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 15 2024-03-08T06:02:43.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.389+0000 7fe9bf807640 1 -- 172.21.15.76:0/2643177372 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe99c06b500 msgr2=0x7fe99c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:43.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.389+0000 7fe9bf807640 1 --2- 172.21.15.76:0/2643177372 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe99c06b500 0x7fe99c06d9c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400a990 tx=0x7fe9b4008040 comp rx=0 tx=0).stop 2024-03-08T06:02:43.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.389+0000 7fe9bf807640 1 -- 172.21.15.76:0/2643177372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9b80cfa90 msgr2=0x7fe9b80b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:43.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.389+0000 7fe9bf807640 1 --2- 172.21.15.76:0/2643177372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9b80cfa90 0x7fe9b80b3fc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0036040 tx=0x7fe9b0004a40 comp rx=0 tx=0).stop 2024-03-08T06:02:43.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.390+0000 7fe9bf807640 1 -- 172.21.15.76:0/2643177372 shutdown_connections 2024-03-08T06:02:43.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.390+0000 7fe9bf807640 1 --2- 172.21.15.76:0/2643177372 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe99c06b500 0x7fe99c06d9c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.390+0000 7fe9bf807640 1 --2- 172.21.15.76:0/2643177372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9b80cfa90 0x7fe9b80b3fc0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.390+0000 7fe9bf807640 1 --2- 172.21.15.76:0/2643177372 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe9b8071c70 0x7fe9b8144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.390+0000 7fe9bf807640 1 --2- 172.21.15.76:0/2643177372 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b806c2e0 0x7fe9b81467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.390+0000 7fe9bf807640 1 -- 172.21.15.76:0/2643177372 >> 172.21.15.76:0/2643177372 conn(0x7fe9b80af650 msgr2=0x7fe9b8160f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:43.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.390+0000 7fe9bf807640 1 -- 172.21.15.76:0/2643177372 shutdown_connections 2024-03-08T06:02:43.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.390+0000 7fe9bf807640 1 -- 172.21.15.76:0/2643177372 wait complete. 2024-03-08T06:02:43.401 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:02:43.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.500+0000 7fd034dd7640 1 Processor -- start 2024-03-08T06:02:43.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.500+0000 7fd034dd7640 1 -- start start 2024-03-08T06:02:43.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.501+0000 7fd034dd7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd03006c3d0 0x7fd03015ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.501+0000 7fd034dd7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd0300ad850 0x7fd0300adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.501+0000 7fd034dd7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0300ad090 con 0x7fd03006bce0 2024-03-08T06:02:43.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.501+0000 7fd034dd7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0300ae1c0 con 0x7fd03006c3d0 2024-03-08T06:02:43.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.501+0000 7fd034dd7640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03006e300 con 0x7fd0300ad850 2024-03-08T06:02:43.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.501+0000 7fd027fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd03006c3d0 0x7fd03015ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.501+0000 7fd027fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd03006c3d0 0x7fd03015ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37122/0 (socket says 172.21.15.76:37122) 2024-03-08T06:02:43.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.501+0000 7fd02e59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd03006bce0 0x7fd03006ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40474/0 (socket says 172.21.15.76:40474) 2024-03-08T06:02:43.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.501+0000 7fd027fff640 1 -- 172.21.15.76:0/1169514456 learned_addr learned my addr 172.21.15.76:0/1169514456 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:43.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.502+0000 7fd02ed9d640 1 --2- 172.21.15.76:0/1169514456 >> v2:172.21.15.81:3300/0 conn(0x7fd0300ad850 0x7fd0300adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.502+0000 7fd027fff640 1 -- 172.21.15.76:0/1169514456 >> v2:172.21.15.81:3300/0 conn(0x7fd0300ad850 msgr2=0x7fd0300adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:43.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.502+0000 7fd027fff640 1 --2- 172.21.15.76:0/1169514456 >> v2:172.21.15.81:3300/0 conn(0x7fd0300ad850 0x7fd0300adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.502+0000 7fd027fff640 1 -- 172.21.15.76:0/1169514456 >> v1:172.21.15.76:6789/0 conn(0x7fd03006bce0 legacy=0x7fd03006ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:43.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.502+0000 7fd027fff640 1 -- 172.21.15.76:0/1169514456 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03006dd40 con 0x7fd03006c3d0 2024-03-08T06:02:43.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.502+0000 7fd02dd9b640 1 -- 172.21.15.76:0/1169514456 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549574462 0 0) 0x7fd0300ad090 con 0x7fd03006bce0 2024-03-08T06:02:43.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.502+0000 7fd027fff640 1 --2- 172.21.15.76:0/1169514456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd03006c3d0 0x7fd03015ca00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd03006c860 tx=0x7fd01802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b1267173e38a17 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:43.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.503+0000 7fd02dd9b640 1 -- 172.21.15.76:0/1169514456 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018034070 con 0x7fd03006c3d0 2024-03-08T06:02:43.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.503+0000 7fd02dd9b640 1 -- 172.21.15.76:0/1169514456 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd018030860 con 0x7fd03006c3d0 2024-03-08T06:02:43.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.503+0000 7fd02dd9b640 1 -- 172.21.15.76:0/1169514456 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018030c30 con 0x7fd03006c3d0 2024-03-08T06:02:43.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.503+0000 7fd034dd7640 1 -- 172.21.15.76:0/1169514456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd03006c3d0 msgr2=0x7fd03015ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:43.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.503+0000 7fd034dd7640 1 --2- 172.21.15.76:0/1169514456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd03006c3d0 0x7fd03015ca00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd03006c860 tx=0x7fd01802b9a0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.503+0000 7fd034dd7640 1 -- 172.21.15.76:0/1169514456 shutdown_connections 2024-03-08T06:02:43.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.503+0000 7fd034dd7640 1 --2- 172.21.15.76:0/1169514456 >> v2:172.21.15.81:3300/0 conn(0x7fd0300ad850 0x7fd0300adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.503+0000 7fd034dd7640 1 --2- 172.21.15.76:0/1169514456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd03006c3d0 0x7fd03015ca00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.503+0000 7fd034dd7640 1 -- 172.21.15.76:0/1169514456 >> 172.21.15.76:0/1169514456 conn(0x7fd0300af650 msgr2=0x7fd0300b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:43.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.503+0000 7fd034dd7640 1 -- 172.21.15.76:0/1169514456 shutdown_connections 2024-03-08T06:02:43.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.503+0000 7fd034dd7640 1 -- 172.21.15.76:0/1169514456 wait complete. 2024-03-08T06:02:43.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.504+0000 7fd034dd7640 1 Processor -- start 2024-03-08T06:02:43.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.505+0000 7fd034dd7640 1 -- start start 2024-03-08T06:02:43.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.505+0000 7fd034dd7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd03006bce0 0x7fd0300d0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.505+0000 7fd034dd7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd03006c3d0 0x7fd03009c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.505+0000 7fd034dd7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0300ad850 0x7fd03009ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.505+0000 7fd034dd7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03006c0c0 con 0x7fd0300ad850 2024-03-08T06:02:43.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.505+0000 7fd034dd7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03006b390 con 0x7fd03006bce0 2024-03-08T06:02:43.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.505+0000 7fd034dd7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd030071c70 con 0x7fd03006c3d0 2024-03-08T06:02:43.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.505+0000 7fd02ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0300ad850 0x7fd03009ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.505+0000 7fd02ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0300ad850 0x7fd03009ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37136/0 (socket says 172.21.15.76:37136) 2024-03-08T06:02:43.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.505+0000 7fd02ed9d640 1 -- 172.21.15.76:0/103920505 learned_addr learned my addr 172.21.15.76:0/103920505 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:43.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.506+0000 7fd02e59c640 1 --2- 172.21.15.76:0/103920505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd03006bce0 0x7fd0300d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.506+0000 7fd027fff640 1 --2- 172.21.15.76:0/103920505 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd03006c3d0 0x7fd03009c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.506+0000 7fd02ed9d640 1 -- 172.21.15.76:0/103920505 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd03006c3d0 msgr2=0x7fd03009c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:43.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.506+0000 7fd02ed9d640 1 --2- 172.21.15.76:0/103920505 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd03006c3d0 0x7fd03009c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.506+0000 7fd02ed9d640 1 -- 172.21.15.76:0/103920505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd03006bce0 msgr2=0x7fd0300d0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:43.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.506+0000 7fd02ed9d640 1 --2- 172.21.15.76:0/103920505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd03006bce0 0x7fd0300d0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.506+0000 7fd02ed9d640 1 -- 172.21.15.76:0/103920505 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd030148bd0 con 0x7fd0300ad850 2024-03-08T06:02:43.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.506+0000 7fd02e59c640 1 --2- 172.21.15.76:0/103920505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd03006bce0 0x7fd0300d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:43.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.506+0000 7fd02ed9d640 1 --2- 172.21.15.76:0/103920505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0300ad850 0x7fd03009ccc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd02800cd70 tx=0x7fd02800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:43.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.506+0000 7fd026ffd640 1 -- 172.21.15.76:0/103920505 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028003b90 con 0x7fd0300ad850 2024-03-08T06:02:43.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.506+0000 7fd026ffd640 1 -- 172.21.15.76:0/103920505 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd028003d30 con 0x7fd0300ad850 2024-03-08T06:02:43.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.506+0000 7fd034dd7640 1 -- 172.21.15.76:0/103920505 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd030148eb0 con 0x7fd0300ad850 2024-03-08T06:02:43.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.506+0000 7fd026ffd640 1 -- 172.21.15.76:0/103920505 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028004240 con 0x7fd0300ad850 2024-03-08T06:02:43.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.506+0000 7fd034dd7640 1 -- 172.21.15.76:0/103920505 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd030149470 con 0x7fd0300ad850 2024-03-08T06:02:43.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.507+0000 7fd034dd7640 1 -- 172.21.15.76:0/103920505 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03004f030 con 0x7fd0300ad850 2024-03-08T06:02:43.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.507+0000 7fd026ffd640 1 -- 172.21.15.76:0/103920505 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0280043e0 con 0x7fd0300ad850 2024-03-08T06:02:43.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.508+0000 7fd026ffd640 1 --2- 172.21.15.76:0/103920505 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd01006b500 0x7fd01006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.508+0000 7fd026ffd640 1 -- 172.21.15.76:0/103920505 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fd02808afe0 con 0x7fd0300ad850 2024-03-08T06:02:43.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.508+0000 7fd02e59c640 1 --2- 172.21.15.76:0/103920505 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd01006b500 0x7fd01006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.508+0000 7fd02e59c640 1 --2- 172.21.15.76:0/103920505 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd01006b500 0x7fd01006d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd020000f60 tx=0x7fd020009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:43.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.512+0000 7fd026ffd640 1 -- 172.21.15.76:0/103920505 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd028059ba0 con 0x7fd0300ad850 2024-03-08T06:02:43.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.681+0000 7fd034dd7640 1 -- 172.21.15.76:0/103920505 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd030069f60 con 0x7fd0300ad850 2024-03-08T06:02:43.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.682+0000 7fd026ffd640 1 -- 172.21.15.76:0/103920505 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16850 (secure 0 0 0) 0x7fd0280594a0 con 0x7fd0300ad850 2024-03-08T06:02:43.683 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:02:43.683 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":42,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:02:18.021974+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:01:45.628991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-08T06:01:46.195100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:02:43.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.684+0000 7fd034dd7640 1 -- 172.21.15.76:0/103920505 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd01006b500 msgr2=0x7fd01006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:43.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.684+0000 7fd034dd7640 1 --2- 172.21.15.76:0/103920505 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd01006b500 0x7fd01006d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd020000f60 tx=0x7fd020009040 comp rx=0 tx=0).stop 2024-03-08T06:02:43.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.684+0000 7fd034dd7640 1 -- 172.21.15.76:0/103920505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0300ad850 msgr2=0x7fd03009ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:43.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.684+0000 7fd034dd7640 1 --2- 172.21.15.76:0/103920505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0300ad850 0x7fd03009ccc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd02800cd70 tx=0x7fd02800a660 comp rx=0 tx=0).stop 2024-03-08T06:02:43.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.684+0000 7fd034dd7640 1 -- 172.21.15.76:0/103920505 shutdown_connections 2024-03-08T06:02:43.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.684+0000 7fd034dd7640 1 --2- 172.21.15.76:0/103920505 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd01006b500 0x7fd01006d9c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.684+0000 7fd034dd7640 1 --2- 172.21.15.76:0/103920505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0300ad850 0x7fd03009ccc0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.684+0000 7fd034dd7640 1 --2- 172.21.15.76:0/103920505 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd03006c3d0 0x7fd03009c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.684+0000 7fd034dd7640 1 --2- 172.21.15.76:0/103920505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd03006bce0 0x7fd0300d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.684+0000 7fd034dd7640 1 -- 172.21.15.76:0/103920505 >> 172.21.15.76:0/103920505 conn(0x7fd0300af650 msgr2=0x7fd030156ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:43.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.684+0000 7fd034dd7640 1 -- 172.21.15.76:0/103920505 shutdown_connections 2024-03-08T06:02:43.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.684+0000 7fd034dd7640 1 -- 172.21.15.76:0/103920505 wait complete. 2024-03-08T06:02:43.696 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-08T06:02:43.696 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:02:43.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.796+0000 7ffb1a6ae640 1 Processor -- start 2024-03-08T06:02:43.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.796+0000 7ffb1a6ae640 1 -- start start 2024-03-08T06:02:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.797+0000 7ffb1a6ae640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb14071390 0x7ffb140cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.797+0000 7ffb1a6ae640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffb140cd310 0x7ffb140cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.797+0000 7ffb1a6ae640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1406d880 con 0x7ffb1406d3f0 2024-03-08T06:02:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.797+0000 7ffb1a6ae640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb1406e010 con 0x7ffb140cd310 2024-03-08T06:02:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.797+0000 7ffb1a6ae640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1406e2f0 con 0x7ffb14071390 2024-03-08T06:02:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.797+0000 7ffb18c4b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffb140cd310 0x7ffb140cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.797+0000 7ffb18c4b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffb140cd310 0x7ffb140cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37138/0 (socket says 172.21.15.76:37138) 2024-03-08T06:02:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.797+0000 7ffb18c4b640 1 -- 172.21.15.76:0/2561466430 learned_addr learned my addr 172.21.15.76:0/2561466430 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.797+0000 7ffb13fff640 1 --2- 172.21.15.76:0/2561466430 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb14071390 0x7ffb140cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.797+0000 7ffb18c4b640 1 -- 172.21.15.76:0/2561466430 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb14071390 msgr2=0x7ffb140cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.797+0000 7ffb18c4b640 1 --2- 172.21.15.76:0/2561466430 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb14071390 0x7ffb140cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.797+0000 7ffb18c4b640 1 -- 172.21.15.76:0/2561466430 >> v1:172.21.15.81:6789/0 conn(0x7ffb1406d3f0 legacy=0x7ffb14071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:43.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.797+0000 7ffb18c4b640 1 -- 172.21.15.76:0/2561466430 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb140ce7b0 con 0x7ffb140cd310 2024-03-08T06:02:43.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.798+0000 7ffb18c4b640 1 --2- 172.21.15.76:0/2561466430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb140cd310 0x7ffb140cd7c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009e50 tx=0x7ffb0c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1dd38d0de09cdd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:43.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.798+0000 7ffb12ffd640 1 -- 172.21.15.76:0/2561466430 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c02ca70 con 0x7ffb140cd310 2024-03-08T06:02:43.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.798+0000 7ffb12ffd640 1 -- 172.21.15.76:0/2561466430 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c02cc10 con 0x7ffb140cd310 2024-03-08T06:02:43.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.798+0000 7ffb12ffd640 1 -- 172.21.15.76:0/2561466430 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0343f0 con 0x7ffb140cd310 2024-03-08T06:02:43.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.798+0000 7ffb1a6ae640 1 -- 172.21.15.76:0/2561466430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb140cd310 msgr2=0x7ffb140cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:43.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.798+0000 7ffb1a6ae640 1 --2- 172.21.15.76:0/2561466430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb140cd310 0x7ffb140cd7c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009e50 tx=0x7ffb0c02bed0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.798+0000 7ffb1a6ae640 1 -- 172.21.15.76:0/2561466430 shutdown_connections 2024-03-08T06:02:43.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.798+0000 7ffb1a6ae640 1 --2- 172.21.15.76:0/2561466430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb140cd310 0x7ffb140cd7c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.798+0000 7ffb1a6ae640 1 --2- 172.21.15.76:0/2561466430 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb14071390 0x7ffb140cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.798+0000 7ffb1a6ae640 1 -- 172.21.15.76:0/2561466430 >> 172.21.15.76:0/2561466430 conn(0x7ffb140cf740 msgr2=0x7ffb140cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:43.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.798+0000 7ffb1a6ae640 1 -- 172.21.15.76:0/2561466430 shutdown_connections 2024-03-08T06:02:43.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.798+0000 7ffb1a6ae640 1 -- 172.21.15.76:0/2561466430 wait complete. 2024-03-08T06:02:43.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.800+0000 7ffb1a6ae640 1 Processor -- start 2024-03-08T06:02:43.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.800+0000 7ffb1a6ae640 1 -- start start 2024-03-08T06:02:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.800+0000 7ffb1a6ae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb14071390 0x7ffb140c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.800+0000 7ffb1a6ae640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb140cd310 0x7ffb140c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.800+0000 7ffb1a6ae640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb1414a000 0x7ffb1414a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.800+0000 7ffb1a6ae640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1415c060 con 0x7ffb1414a000 2024-03-08T06:02:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.800+0000 7ffb1a6ae640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1415bee0 con 0x7ffb14071390 2024-03-08T06:02:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.800+0000 7ffb1a6ae640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140cdd00 con 0x7ffb140cd310 2024-03-08T06:02:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb18c4b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb1414a000 0x7ffb1414a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb18c4b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb1414a000 0x7ffb1414a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37148/0 (socket says 172.21.15.76:37148) 2024-03-08T06:02:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb18c4b640 1 -- 172.21.15.76:0/3432279297 learned_addr learned my addr 172.21.15.76:0/3432279297 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb137fe640 1 --2- 172.21.15.76:0/3432279297 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb140cd310 0x7ffb140c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb13fff640 1 --2- 172.21.15.76:0/3432279297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb14071390 0x7ffb140c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb18c4b640 1 -- 172.21.15.76:0/3432279297 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb140cd310 msgr2=0x7ffb140c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb18c4b640 1 --2- 172.21.15.76:0/3432279297 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb140cd310 0x7ffb140c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb18c4b640 1 -- 172.21.15.76:0/3432279297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb14071390 msgr2=0x7ffb140c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:43.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb18c4b640 1 --2- 172.21.15.76:0/3432279297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb14071390 0x7ffb140c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:43.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb18c4b640 1 -- 172.21.15.76:0/3432279297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb141e8160 con 0x7ffb1414a000 2024-03-08T06:02:43.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb137fe640 1 --2- 172.21.15.76:0/3432279297 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb140cd310 0x7ffb140c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:43.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb18c4b640 1 --2- 172.21.15.76:0/3432279297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb1414a000 0x7ffb1414a490 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009e50 tx=0x7ffb0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:43.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb10ff9640 1 -- 172.21.15.76:0/3432279297 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c043020 con 0x7ffb1414a000 2024-03-08T06:02:43.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb10ff9640 1 -- 172.21.15.76:0/3432279297 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c03e450 con 0x7ffb1414a000 2024-03-08T06:02:43.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb1a6ae640 1 -- 172.21.15.76:0/3432279297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb14148010 con 0x7ffb1414a000 2024-03-08T06:02:43.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb10ff9640 1 -- 172.21.15.76:0/3432279297 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c009510 con 0x7ffb1414a000 2024-03-08T06:02:43.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.801+0000 7ffb1a6ae640 1 -- 172.21.15.76:0/3432279297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb141485d0 con 0x7ffb1414a000 2024-03-08T06:02:43.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.802+0000 7ffb1a6ae640 1 -- 172.21.15.76:0/3432279297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb140ce110 con 0x7ffb1414a000 2024-03-08T06:02:43.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.803+0000 7ffb10ff9640 1 -- 172.21.15.76:0/3432279297 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb0c046020 con 0x7ffb1414a000 2024-03-08T06:02:43.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.803+0000 7ffb10ff9640 1 --2- 172.21.15.76:0/3432279297 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffb0006b500 0x7ffb0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:43.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.803+0000 7ffb10ff9640 1 -- 172.21.15.76:0/3432279297 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7ffb0c0acb80 con 0x7ffb1414a000 2024-03-08T06:02:43.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.803+0000 7ffb13fff640 1 --2- 172.21.15.76:0/3432279297 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffb0006b500 0x7ffb0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:43.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.804+0000 7ffb13fff640 1 --2- 172.21.15.76:0/3432279297 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffb0006b500 0x7ffb0006d9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffb04006fd0 tx=0x7ffb04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:43.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:43.807+0000 7ffb10ff9640 1 -- 172.21.15.76:0/3432279297 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb140ce110 con 0x7ffb1414a000 2024-03-08T06:02:44.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.019+0000 7ffb1a6ae640 1 -- 172.21.15.76:0/3432279297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb1404f030 con 0x7ffb1414a000 2024-03-08T06:02:44.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.020+0000 7ffb10ff9640 1 -- 172.21.15.76:0/3432279297 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4932 (secure 0 0 0) 0x7ffb0c04d030 con 0x7ffb1414a000 2024-03-08T06:02:44.020 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:02:44.020 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/606148582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":606148582},{"type":"v1","addr":"172.21.15.81:6835","nonce":606148582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/278988477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":278988477},{"type":"v1","addr":"172.21.15.76:6836","nonce":278988477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2526423312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":2526423312},{"type":"v1","addr":"172.21.15.76:6837","nonce":2526423312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:01:47.232184+0000","modified":"2024-03-08T06:01:49.227002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5358":{"gid":5358,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6834/1503895376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1503895376},{"type":"v1","addr":"172.21.15.81:6834","nonce":1503895376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T06:02:44.020 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 15 2024-03-08T06:02:44.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.022+0000 7ffb1a6ae640 1 -- 172.21.15.76:0/3432279297 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffb0006b500 msgr2=0x7ffb0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.022+0000 7ffb1a6ae640 1 --2- 172.21.15.76:0/3432279297 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffb0006b500 0x7ffb0006d9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffb04006fd0 tx=0x7ffb04008040 comp rx=0 tx=0).stop 2024-03-08T06:02:44.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.022+0000 7ffb1a6ae640 1 -- 172.21.15.76:0/3432279297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb1414a000 msgr2=0x7ffb1414a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.022+0000 7ffb1a6ae640 1 --2- 172.21.15.76:0/3432279297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb1414a000 0x7ffb1414a490 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009e50 tx=0x7ffb0c004a40 comp rx=0 tx=0).stop 2024-03-08T06:02:44.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.022+0000 7ffb1a6ae640 1 -- 172.21.15.76:0/3432279297 shutdown_connections 2024-03-08T06:02:44.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.022+0000 7ffb1a6ae640 1 --2- 172.21.15.76:0/3432279297 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffb0006b500 0x7ffb0006d9c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.022+0000 7ffb1a6ae640 1 --2- 172.21.15.76:0/3432279297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb1414a000 0x7ffb1414a490 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.022+0000 7ffb1a6ae640 1 --2- 172.21.15.76:0/3432279297 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffb140cd310 0x7ffb140c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.022+0000 7ffb1a6ae640 1 --2- 172.21.15.76:0/3432279297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb14071390 0x7ffb140c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.022+0000 7ffb1a6ae640 1 -- 172.21.15.76:0/3432279297 >> 172.21.15.76:0/3432279297 conn(0x7ffb140cf740 msgr2=0x7ffb14071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:44.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.022+0000 7ffb1a6ae640 1 -- 172.21.15.76:0/3432279297 shutdown_connections 2024-03-08T06:02:44.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.022+0000 7ffb1a6ae640 1 -- 172.21.15.76:0/3432279297 wait complete. 2024-03-08T06:02:44.033 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:02:44.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.133+0000 7f635add4640 1 Processor -- start 2024-03-08T06:02:44.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.133+0000 7f635add4640 1 -- start start 2024-03-08T06:02:44.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.134+0000 7f635add4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f635406c300 0x7f635406dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.134+0000 7f635add4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63540722c0 0x7f635415d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.134+0000 7f635add4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635406b730 con 0x7f6354071ee0 2024-03-08T06:02:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.134+0000 7f635add4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f635406ba10 con 0x7f635406c300 2024-03-08T06:02:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.134+0000 7f635add4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635406bcf0 con 0x7f63540722c0 2024-03-08T06:02:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.134+0000 7f6353fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f635406c300 0x7f635406dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.134+0000 7f6353fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f635406c300 0x7f635406dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37160/0 (socket says 172.21.15.76:37160) 2024-03-08T06:02:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.134+0000 7f6353fff640 1 -- 172.21.15.76:0/2709277553 learned_addr learned my addr 172.21.15.76:0/2709277553 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.134+0000 7f6358b70640 1 --1- 172.21.15.76:0/2709277553 >> v1:172.21.15.76:6789/0 conn(0x7f6354071ee0 0x7f63540b2f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40486/0 (socket says 172.21.15.76:40486) 2024-03-08T06:02:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.134+0000 7f6359371640 1 --2- 172.21.15.76:0/2709277553 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63540722c0 0x7f635415d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.134+0000 7f6353fff640 1 -- 172.21.15.76:0/2709277553 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63540722c0 msgr2=0x7f635415d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.134+0000 7f6353fff640 1 --2- 172.21.15.76:0/2709277553 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63540722c0 0x7f635415d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.134+0000 7f6353fff640 1 -- 172.21.15.76:0/2709277553 >> v1:172.21.15.76:6789/0 conn(0x7f6354071ee0 legacy=0x7f63540b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.134+0000 7f6353fff640 1 -- 172.21.15.76:0/2709277553 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63540ad090 con 0x7f635406c300 2024-03-08T06:02:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.134+0000 7f63537fe640 1 -- 172.21.15.76:0/2709277553 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286079438 0 0) 0x7f635406b730 con 0x7f6354071ee0 2024-03-08T06:02:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.135+0000 7f6353fff640 1 --2- 172.21.15.76:0/2709277553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635406c300 0x7f635406dba0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6340009870 tx=0x7f634002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd544e535e76b16 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.135+0000 7f63537fe640 1 -- 172.21.15.76:0/2709277553 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6340034070 con 0x7f635406c300 2024-03-08T06:02:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.135+0000 7f63537fe640 1 -- 172.21.15.76:0/2709277553 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6340030840 con 0x7f635406c300 2024-03-08T06:02:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.135+0000 7f63537fe640 1 -- 172.21.15.76:0/2709277553 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6340030c10 con 0x7f635406c300 2024-03-08T06:02:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.136+0000 7f635add4640 1 -- 172.21.15.76:0/2709277553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635406c300 msgr2=0x7f635406dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.136+0000 7f635add4640 1 --2- 172.21.15.76:0/2709277553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635406c300 0x7f635406dba0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6340009870 tx=0x7f634002b980 comp rx=0 tx=0).stop 2024-03-08T06:02:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.136+0000 7f635add4640 1 -- 172.21.15.76:0/2709277553 shutdown_connections 2024-03-08T06:02:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.136+0000 7f635add4640 1 --2- 172.21.15.76:0/2709277553 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63540722c0 0x7f635415d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.136+0000 7f635add4640 1 --2- 172.21.15.76:0/2709277553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635406c300 0x7f635406dba0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.136+0000 7f635add4640 1 -- 172.21.15.76:0/2709277553 >> 172.21.15.76:0/2709277553 conn(0x7f63540af650 msgr2=0x7f63540b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.136+0000 7f635add4640 1 -- 172.21.15.76:0/2709277553 shutdown_connections 2024-03-08T06:02:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.136+0000 7f635add4640 1 -- 172.21.15.76:0/2709277553 wait complete. 2024-03-08T06:02:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.137+0000 7f635add4640 1 Processor -- start 2024-03-08T06:02:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.137+0000 7f635add4640 1 -- start start 2024-03-08T06:02:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f635add4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635406c300 0x7f63541481f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f635add4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6354071ee0 0x7f6354143ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f635add4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63540722c0 0x7f6354144b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f635add4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635415df10 con 0x7f635406c300 2024-03-08T06:02:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f635add4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635415dd90 con 0x7f6354071ee0 2024-03-08T06:02:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f635add4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635415e090 con 0x7f63540722c0 2024-03-08T06:02:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f6358b70640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635406c300 0x7f63541481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f6358b70640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635406c300 0x7f63541481f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37174/0 (socket says 172.21.15.76:37174) 2024-03-08T06:02:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f6358b70640 1 -- 172.21.15.76:0/1795018358 learned_addr learned my addr 172.21.15.76:0/1795018358 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f6359371640 1 --2- 172.21.15.76:0/1795018358 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63540722c0 0x7f6354144b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f6353fff640 1 --2- 172.21.15.76:0/1795018358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6354071ee0 0x7f6354143ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f6358b70640 1 -- 172.21.15.76:0/1795018358 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63540722c0 msgr2=0x7f6354144b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f6358b70640 1 --2- 172.21.15.76:0/1795018358 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63540722c0 0x7f6354144b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f6358b70640 1 -- 172.21.15.76:0/1795018358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6354071ee0 msgr2=0x7f6354143ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f6358b70640 1 --2- 172.21.15.76:0/1795018358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6354071ee0 0x7f6354143ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f6358b70640 1 -- 172.21.15.76:0/1795018358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6354147ff0 con 0x7f635406c300 2024-03-08T06:02:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f6359371640 1 --2- 172.21.15.76:0/1795018358 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63540722c0 0x7f6354144b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f6358b70640 1 --2- 172.21.15.76:0/1795018358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635406c300 0x7f63541481f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f634800c500 tx=0x7f634800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f63517fa640 1 -- 172.21.15.76:0/1795018358 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6348018070 con 0x7f635406c300 2024-03-08T06:02:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.138+0000 7f63517fa640 1 -- 172.21.15.76:0/1795018358 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6348007580 con 0x7f635406c300 2024-03-08T06:02:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.139+0000 7f635add4640 1 -- 172.21.15.76:0/1795018358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6354154350 con 0x7f635406c300 2024-03-08T06:02:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.139+0000 7f63517fa640 1 -- 172.21.15.76:0/1795018358 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6348004a20 con 0x7f635406c300 2024-03-08T06:02:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.139+0000 7f635add4640 1 -- 172.21.15.76:0/1795018358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6354154070 con 0x7f635406c300 2024-03-08T06:02:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.139+0000 7f635add4640 1 -- 172.21.15.76:0/1795018358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635404f030 con 0x7f635406c300 2024-03-08T06:02:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.140+0000 7f63517fa640 1 -- 172.21.15.76:0/1795018358 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f634801b070 con 0x7f635406c300 2024-03-08T06:02:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.140+0000 7f63517fa640 1 --2- 172.21.15.76:0/1795018358 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f633806b4d0 0x7f633806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.140+0000 7f63517fa640 1 -- 172.21.15.76:0/1795018358 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f6348089630 con 0x7f635406c300 2024-03-08T06:02:44.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.140+0000 7f6353fff640 1 --2- 172.21.15.76:0/1795018358 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f633806b4d0 0x7f633806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.141+0000 7f6353fff640 1 --2- 172.21.15.76:0/1795018358 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f633806b4d0 0x7f633806d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6340036000 tx=0x7f6340008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:44.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.145+0000 7f63517fa640 1 -- 172.21.15.76:0/1795018358 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6354154350 con 0x7f635406c300 2024-03-08T06:02:44.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.350+0000 7f635add4640 1 -- 172.21.15.76:0/1795018358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63540b12d0 con 0x7f635406c300 2024-03-08T06:02:44.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.350+0000 7f63517fa640 1 -- 172.21.15.76:0/1795018358 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f6354147ff0 con 0x7f635406c300 2024-03-08T06:02:44.351 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:02:44.351 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/606148582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":606148582},{"type":"v1","addr":"172.21.15.81:6835","nonce":606148582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/278988477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":278988477},{"type":"v1","addr":"172.21.15.76:6836","nonce":278988477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2526423312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":2526423312},{"type":"v1","addr":"172.21.15.76:6837","nonce":2526423312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:01:47.232184+0000","modified":"2024-03-08T06:01:49.227002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5358":{"gid":5358,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6834/1503895376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1503895376},{"type":"v1","addr":"172.21.15.81:6834","nonce":1503895376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T06:02:44.351 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 15 2024-03-08T06:02:44.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.352+0000 7f635add4640 1 -- 172.21.15.76:0/1795018358 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f633806b4d0 msgr2=0x7f633806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.352+0000 7f635add4640 1 --2- 172.21.15.76:0/1795018358 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f633806b4d0 0x7f633806d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6340036000 tx=0x7f6340008030 comp rx=0 tx=0).stop 2024-03-08T06:02:44.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.352+0000 7f635add4640 1 -- 172.21.15.76:0/1795018358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635406c300 msgr2=0x7f63541481f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.352+0000 7f635add4640 1 --2- 172.21.15.76:0/1795018358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635406c300 0x7f63541481f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f634800c500 tx=0x7f634800c9c0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.352+0000 7f635add4640 1 -- 172.21.15.76:0/1795018358 shutdown_connections 2024-03-08T06:02:44.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.352+0000 7f635add4640 1 --2- 172.21.15.76:0/1795018358 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f633806b4d0 0x7f633806d990 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.352+0000 7f635add4640 1 --2- 172.21.15.76:0/1795018358 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63540722c0 0x7f6354144b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.352+0000 7f635add4640 1 --2- 172.21.15.76:0/1795018358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6354071ee0 0x7f6354143ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.352+0000 7f635add4640 1 --2- 172.21.15.76:0/1795018358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635406c300 0x7f63541481f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.352+0000 7f635add4640 1 -- 172.21.15.76:0/1795018358 >> 172.21.15.76:0/1795018358 conn(0x7f63540af650 msgr2=0x7f635415a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:44.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.352+0000 7f635add4640 1 -- 172.21.15.76:0/1795018358 shutdown_connections 2024-03-08T06:02:44.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.352+0000 7f635add4640 1 -- 172.21.15.76:0/1795018358 wait complete. 2024-03-08T06:02:44.364 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:02:44.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.462+0000 7f6f8cec3640 1 Processor -- start 2024-03-08T06:02:44.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.462+0000 7f6f8cec3640 1 -- start start 2024-03-08T06:02:44.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.463+0000 7f6f8cec3640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6f880ceb00 0x7f6f881623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.463+0000 7f6f8cec3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f880cffc0 con 0x7f6f8815dff0 2024-03-08T06:02:44.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.463+0000 7f6f8cec3640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f880d02a0 con 0x7f6f8815dc10 2024-03-08T06:02:44.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.463+0000 7f6f8cec3640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f8806c780 con 0x7f6f880ceb00 2024-03-08T06:02:44.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.463+0000 7f6f86d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6f8815dff0 0x7f6f8806cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40494/0 (socket says 172.21.15.76:40494) 2024-03-08T06:02:44.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f86d9d640 1 -- 172.21.15.76:0/673254425 learned_addr learned my addr 172.21.15.76:0/673254425 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:44.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f85d9b640 1 --2- 172.21.15.76:0/673254425 >> v2:172.21.15.81:3300/0 conn(0x7f6f880ceb00 0x7f6f881623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f8559a640 1 -- 172.21.15.76:0/673254425 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153340376 0 0) 0x7f6f880cffc0 con 0x7f6f8815dff0 2024-03-08T06:02:44.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f8559a640 1 -- 172.21.15.76:0/673254425 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f74003520 con 0x7f6f8815dff0 2024-03-08T06:02:44.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f8559a640 1 -- 172.21.15.76:0/673254425 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 510310665 0 0) 0x7f6f74003520 con 0x7f6f8815dff0 2024-03-08T06:02:44.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f8559a640 1 -- 172.21.15.76:0/673254425 >> v2:172.21.15.81:3300/0 conn(0x7f6f880ceb00 msgr2=0x7f6f881623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:44.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f8559a640 1 --2- 172.21.15.76:0/673254425 >> v2:172.21.15.81:3300/0 conn(0x7f6f880ceb00 0x7f6f881623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f8559a640 1 -- 172.21.15.76:0/673254425 >> v1:172.21.15.81:6789/0 conn(0x7f6f8815dc10 legacy=0x7f6f880ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f8559a640 1 -- 172.21.15.76:0/673254425 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8806c410 con 0x7f6f8815dff0 2024-03-08T06:02:44.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f8559a640 1 -- 172.21.15.76:0/673254425 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7f6f780019f0 con 0x7f6f8815dff0 2024-03-08T06:02:44.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f8559a640 1 -- 172.21.15.76:0/673254425 >> v1:172.21.15.76:6789/0 conn(0x7f6f8815dff0 legacy=0x7f6f8806cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f8559a640 1 --2- 172.21.15.76:0/673254425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f74003a50 0x7f6f74003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f8559a640 1 --2- 172.21.15.76:0/673254425 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6f74004800 0x7f6f74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f8559a640 1 --2- 172.21.15.76:0/673254425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f740052e0 0x7f6f7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f8559a640 1 -- 172.21.15.76:0/673254425 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8806c780 con 0x7f6f740052e0 2024-03-08T06:02:44.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f8559a640 1 -- 172.21.15.76:0/673254425 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7400c1b0 con 0x7f6f74003a50 2024-03-08T06:02:44.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f8559a640 1 -- 172.21.15.76:0/673254425 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7400c3b0 con 0x7f6f74004800 2024-03-08T06:02:44.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f85d9b640 1 --2- 172.21.15.76:0/673254425 >> v2:172.21.15.81:3300/0 conn(0x7f6f880ceb00 0x7f6f881623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:02:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f86d9d640 1 --2- 172.21.15.76:0/673254425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f740052e0 0x7f6f7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f8659c640 1 --2- 172.21.15.76:0/673254425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f74003a50 0x7f6f74003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f85d9b640 1 --2- 172.21.15.76:0/673254425 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6f74004800 0x7f6f74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f86d9d640 1 -- 172.21.15.76:0/673254425 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6f74004800 msgr2=0x7f6f74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f86d9d640 1 --2- 172.21.15.76:0/673254425 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6f74004800 0x7f6f74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f86d9d640 1 -- 172.21.15.76:0/673254425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f74003a50 msgr2=0x7f6f74003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.464+0000 7f6f86d9d640 1 --2- 172.21.15.76:0/673254425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f74003a50 0x7f6f74003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.465+0000 7f6f86d9d640 1 -- 172.21.15.76:0/673254425 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f74003520 con 0x7f6f740052e0 2024-03-08T06:02:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.465+0000 7f6f86d9d640 1 -- 172.21.15.76:0/673254425 reap_dead start 2024-03-08T06:02:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.465+0000 7f6f86d9d640 1 --2- 172.21.15.76:0/673254425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f740052e0 0x7f6f7400b9c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6f7800a830 tx=0x7f6f74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a130752c6420094 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.465+0000 7f6f8559a640 1 -- 172.21.15.76:0/673254425 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7800a480 con 0x7f6f740052e0 2024-03-08T06:02:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.465+0000 7f6f8559a640 1 -- 172.21.15.76:0/673254425 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f78034630 con 0x7f6f740052e0 2024-03-08T06:02:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.465+0000 7f6f8559a640 1 -- 172.21.15.76:0/673254425 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f78034a00 con 0x7f6f740052e0 2024-03-08T06:02:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.466+0000 7f6f8cec3640 1 -- 172.21.15.76:0/673254425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f740052e0 msgr2=0x7f6f7400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.466+0000 7f6f8cec3640 1 --2- 172.21.15.76:0/673254425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f740052e0 0x7f6f7400b9c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6f7800a830 tx=0x7f6f74005830 comp rx=0 tx=0).stop 2024-03-08T06:02:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.466+0000 7f6f8cec3640 1 -- 172.21.15.76:0/673254425 shutdown_connections 2024-03-08T06:02:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.466+0000 7f6f8cec3640 1 --2- 172.21.15.76:0/673254425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f740052e0 0x7f6f7400b9c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.466+0000 7f6f8cec3640 1 -- 172.21.15.76:0/673254425 >> 172.21.15.76:0/673254425 conn(0x7f6f8806f5e0 msgr2=0x7f6f8806f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.466+0000 7f6f8cec3640 1 -- 172.21.15.76:0/673254425 shutdown_connections 2024-03-08T06:02:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.466+0000 7f6f8cec3640 1 -- 172.21.15.76:0/673254425 wait complete. 2024-03-08T06:02:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.467+0000 7f6f8cec3640 1 Processor -- start 2024-03-08T06:02:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.467+0000 7f6f8cec3640 1 -- start start 2024-03-08T06:02:44.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.468+0000 7f6f8cec3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f740052e0 0x7f6f8815a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.468+0000 7f6f8cec3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f880b0f00 0x7f6f880b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.468+0000 7f6f8cec3640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6f880cb1a0 0x7f6f880cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.468+0000 7f6f8cec3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f881623f0 con 0x7f6f740052e0 2024-03-08T06:02:44.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.468+0000 7f6f8cec3640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f880d0580 con 0x7f6f880b0f00 2024-03-08T06:02:44.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.468+0000 7f6f8cec3640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f88162570 con 0x7f6f880cb1a0 2024-03-08T06:02:44.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.468+0000 7f6f8659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f740052e0 0x7f6f8815a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.468+0000 7f6f8659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f740052e0 0x7f6f8815a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37186/0 (socket says 172.21.15.76:37186) 2024-03-08T06:02:44.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.468+0000 7f6f8659c640 1 -- 172.21.15.76:0/2454321105 learned_addr learned my addr 172.21.15.76:0/2454321105 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.468+0000 7f6f8659c640 1 -- 172.21.15.76:0/2454321105 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6f880cb1a0 msgr2=0x7f6f880cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.468+0000 7f6f8659c640 1 --2- 172.21.15.76:0/2454321105 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6f880cb1a0 0x7f6f880cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.468+0000 7f6f8659c640 1 -- 172.21.15.76:0/2454321105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f880b0f00 msgr2=0x7f6f880b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.468+0000 7f6f8659c640 1 --2- 172.21.15.76:0/2454321105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f880b0f00 0x7f6f880b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.468+0000 7f6f8659c640 1 -- 172.21.15.76:0/2454321105 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f880ca920 con 0x7f6f740052e0 2024-03-08T06:02:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.469+0000 7f6f8659c640 1 --2- 172.21.15.76:0/2454321105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f740052e0 0x7f6f8815a770 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000f5f0 tx=0x7f6f8000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.469+0000 7f6f7effd640 1 -- 172.21.15.76:0/2454321105 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f800052e0 con 0x7f6f740052e0 2024-03-08T06:02:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.469+0000 7f6f7effd640 1 -- 172.21.15.76:0/2454321105 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f80018070 con 0x7f6f740052e0 2024-03-08T06:02:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.469+0000 7f6f8cec3640 1 -- 172.21.15.76:0/2454321105 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f880cac00 con 0x7f6f740052e0 2024-03-08T06:02:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.469+0000 7f6f7effd640 1 -- 172.21.15.76:0/2454321105 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80003810 con 0x7f6f740052e0 2024-03-08T06:02:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.469+0000 7f6f8cec3640 1 -- 172.21.15.76:0/2454321105 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f880b1380 con 0x7f6f740052e0 2024-03-08T06:02:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.470+0000 7f6f8cec3640 1 -- 172.21.15.76:0/2454321105 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f880b4600 con 0x7f6f740052e0 2024-03-08T06:02:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.470+0000 7f6f7effd640 1 -- 172.21.15.76:0/2454321105 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f800039b0 con 0x7f6f740052e0 2024-03-08T06:02:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.470+0000 7f6f7effd640 1 --2- 172.21.15.76:0/2454321105 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6f6806b500 0x7f6f6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.471+0000 7f6f7effd640 1 -- 172.21.15.76:0/2454321105 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f6f8008a030 con 0x7f6f740052e0 2024-03-08T06:02:44.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.471+0000 7f6f85d9b640 1 --2- 172.21.15.76:0/2454321105 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6f6806b500 0x7f6f6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.471+0000 7f6f85d9b640 1 --2- 172.21.15.76:0/2454321105 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6f6806b500 0x7f6f6806d9c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6f881610b0 tx=0x7f6f70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:44.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.475+0000 7f6f7effd640 1 -- 172.21.15.76:0/2454321105 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f80058d90 con 0x7f6f740052e0 2024-03-08T06:02:44.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.645+0000 7f6f8cec3640 1 -- 172.21.15.76:0/2454321105 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f8804f030 con 0x7f6f740052e0 2024-03-08T06:02:44.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.647+0000 7f6f7effd640 1 -- 172.21.15.76:0/2454321105 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16850 (secure 0 0 0) 0x7f6f80058690 con 0x7f6f740052e0 2024-03-08T06:02:44.648 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:02:44.648 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":42,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:02:18.021974+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:01:45.628991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-08T06:01:46.195100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:02:44.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.649+0000 7f6f8cec3640 1 -- 172.21.15.76:0/2454321105 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6f6806b500 msgr2=0x7f6f6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.649+0000 7f6f8cec3640 1 --2- 172.21.15.76:0/2454321105 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6f6806b500 0x7f6f6806d9c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6f881610b0 tx=0x7f6f70008040 comp rx=0 tx=0).stop 2024-03-08T06:02:44.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.649+0000 7f6f8cec3640 1 -- 172.21.15.76:0/2454321105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f740052e0 msgr2=0x7f6f8815a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.649+0000 7f6f8cec3640 1 --2- 172.21.15.76:0/2454321105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f740052e0 0x7f6f8815a770 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000f5f0 tx=0x7f6f8000fab0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.649+0000 7f6f8cec3640 1 -- 172.21.15.76:0/2454321105 shutdown_connections 2024-03-08T06:02:44.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.649+0000 7f6f8cec3640 1 --2- 172.21.15.76:0/2454321105 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f6f6806b500 0x7f6f6806d9c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.649+0000 7f6f8cec3640 1 --2- 172.21.15.76:0/2454321105 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6f880cb1a0 0x7f6f880cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.649+0000 7f6f8cec3640 1 --2- 172.21.15.76:0/2454321105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f880b0f00 0x7f6f880b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.649+0000 7f6f8cec3640 1 --2- 172.21.15.76:0/2454321105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f740052e0 0x7f6f8815a770 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.649+0000 7f6f8cec3640 1 -- 172.21.15.76:0/2454321105 >> 172.21.15.76:0/2454321105 conn(0x7f6f8806f5e0 msgr2=0x7f6f8815e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:44.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.649+0000 7f6f8cec3640 1 -- 172.21.15.76:0/2454321105 shutdown_connections 2024-03-08T06:02:44.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.649+0000 7f6f8cec3640 1 -- 172.21.15.76:0/2454321105 wait complete. 2024-03-08T06:02:44.661 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-08T06:02:44.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.762+0000 7f4ffa4f4640 1 Processor -- start 2024-03-08T06:02:44.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.762+0000 7f4ffa4f4640 1 -- start start 2024-03-08T06:02:44.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.763+0000 7f4ffa4f4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4ff40d2150 0x7f4ff40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.763+0000 7f4ffa4f4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff40cfa80 con 0x7f4ff40ce690 2024-03-08T06:02:44.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.763+0000 7f4ffa4f4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff40cefb0 con 0x7f4ff40d0800 2024-03-08T06:02:44.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.763+0000 7f4ffa4f4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff40b1000 con 0x7f4ff40d2150 2024-03-08T06:02:44.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.763+0000 7f4ff37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4ff40d2150 0x7f4ff40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.763+0000 7f4ff37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4ff40d2150 0x7f4ff40cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37202/0 (socket says 172.21.15.76:37202) 2024-03-08T06:02:44.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.763+0000 7f4ff37fe640 1 -- 172.21.15.76:0/3956642719 learned_addr learned my addr 172.21.15.76:0/3956642719 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:44.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.763+0000 7f4ff3fff640 1 --1- 172.21.15.76:0/3956642719 >> v1:172.21.15.76:6789/0 conn(0x7f4ff40ce690 0x7f4ff40b0840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40506/0 (socket says 172.21.15.76:40506) 2024-03-08T06:02:44.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.763+0000 7f4ff37fe640 1 -- 172.21.15.76:0/3956642719 >> v1:172.21.15.81:6789/0 conn(0x7f4ff40d0800 legacy=0x7f4ff40b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:44.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.763+0000 7f4ff2ffd640 1 -- 172.21.15.76:0/3956642719 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735455473 0 0) 0x7f4ff40cfa80 con 0x7f4ff40ce690 2024-03-08T06:02:44.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.763+0000 7f4ff37fe640 1 -- 172.21.15.76:0/3956642719 >> v1:172.21.15.76:6789/0 conn(0x7f4ff40ce690 legacy=0x7f4ff40b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.763+0000 7f4ff37fe640 1 -- 172.21.15.76:0/3956642719 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff40cfd20 con 0x7f4ff40d2150 2024-03-08T06:02:44.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.764+0000 7f4ff37fe640 1 --2- 172.21.15.76:0/3956642719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ff40d2150 0x7f4ff40cea70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009870 tx=0x7f4fe802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbfe5f1002692b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:44.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.764+0000 7f4ff2ffd640 1 -- 172.21.15.76:0/3956642719 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe802f840 con 0x7f4ff40d2150 2024-03-08T06:02:44.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.764+0000 7f4ff2ffd640 1 -- 172.21.15.76:0/3956642719 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe802f9e0 con 0x7f4ff40d2150 2024-03-08T06:02:44.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.764+0000 7f4ff2ffd640 1 -- 172.21.15.76:0/3956642719 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe802fdb0 con 0x7f4ff40d2150 2024-03-08T06:02:44.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.765+0000 7f4ffa4f4640 1 -- 172.21.15.76:0/3956642719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ff40d2150 msgr2=0x7f4ff40cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.765+0000 7f4ffa4f4640 1 --2- 172.21.15.76:0/3956642719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ff40d2150 0x7f4ff40cea70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009870 tx=0x7f4fe802ba20 comp rx=0 tx=0).stop 2024-03-08T06:02:44.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.765+0000 7f4ffa4f4640 1 -- 172.21.15.76:0/3956642719 shutdown_connections 2024-03-08T06:02:44.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.765+0000 7f4ffa4f4640 1 --2- 172.21.15.76:0/3956642719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ff40d2150 0x7f4ff40cea70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.765+0000 7f4ffa4f4640 1 -- 172.21.15.76:0/3956642719 >> 172.21.15.76:0/3956642719 conn(0x7f4ff406ae60 msgr2=0x7f4ff406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:44.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.765+0000 7f4ffa4f4640 1 -- 172.21.15.76:0/3956642719 shutdown_connections 2024-03-08T06:02:44.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.765+0000 7f4ffa4f4640 1 -- 172.21.15.76:0/3956642719 wait complete. 2024-03-08T06:02:44.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.766+0000 7f4ffa4f4640 1 Processor -- start 2024-03-08T06:02:44.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.766+0000 7f4ffa4f4640 1 -- start start 2024-03-08T06:02:44.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ffa4f4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ff40ce690 0x7f4ff415a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ffa4f4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ff40d0800 0x7f4ff40c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ffa4f4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ff40d2150 0x7f4ff40cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ffa4f4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff415be20 con 0x7f4ff40ce690 2024-03-08T06:02:44.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ffa4f4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff40d02e0 con 0x7f4ff40d2150 2024-03-08T06:02:44.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ffa4f4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff415bfa0 con 0x7f4ff40d0800 2024-03-08T06:02:44.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ff3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ff40ce690 0x7f4ff415a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ff3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ff40ce690 0x7f4ff415a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37216/0 (socket says 172.21.15.76:37216) 2024-03-08T06:02:44.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ff3fff640 1 -- 172.21.15.76:0/935007732 learned_addr learned my addr 172.21.15.76:0/935007732 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:44.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ff37fe640 1 --2- 172.21.15.76:0/935007732 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ff40d0800 0x7f4ff40c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ff8a91640 1 --2- 172.21.15.76:0/935007732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ff40d2150 0x7f4ff40cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ff3fff640 1 -- 172.21.15.76:0/935007732 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ff40d0800 msgr2=0x7f4ff40c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ff3fff640 1 --2- 172.21.15.76:0/935007732 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ff40d0800 0x7f4ff40c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ff3fff640 1 -- 172.21.15.76:0/935007732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ff40d2150 msgr2=0x7f4ff40cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ff3fff640 1 --2- 172.21.15.76:0/935007732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ff40d2150 0x7f4ff40cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ff3fff640 1 -- 172.21.15.76:0/935007732 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff40b0b10 con 0x7f4ff40ce690 2024-03-08T06:02:44.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ff3fff640 1 --2- 172.21.15.76:0/935007732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ff40ce690 0x7f4ff415a220 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400c4a0 tx=0x7f4fe400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:44.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ff0ff9640 1 -- 172.21.15.76:0/935007732 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4018070 con 0x7f4ff40ce690 2024-03-08T06:02:44.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ff0ff9640 1 -- 172.21.15.76:0/935007732 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe40074e0 con 0x7f4ff40ce690 2024-03-08T06:02:44.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ffa4f4640 1 -- 172.21.15.76:0/935007732 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff40c6b00 con 0x7f4ff40ce690 2024-03-08T06:02:44.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ff0ff9640 1 -- 172.21.15.76:0/935007732 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4004a20 con 0x7f4ff40ce690 2024-03-08T06:02:44.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.767+0000 7f4ffa4f4640 1 -- 172.21.15.76:0/935007732 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff40c73a0 con 0x7f4ff40ce690 2024-03-08T06:02:44.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.768+0000 7f4ffa4f4640 1 -- 172.21.15.76:0/935007732 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff40c6ef0 con 0x7f4ff40ce690 2024-03-08T06:02:44.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.769+0000 7f4ff0ff9640 1 -- 172.21.15.76:0/935007732 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4fe401b070 con 0x7f4ff40ce690 2024-03-08T06:02:44.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.769+0000 7f4ff0ff9640 1 --2- 172.21.15.76:0/935007732 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4fe006b4d0 0x7f4fe006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:44.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.769+0000 7f4ff37fe640 1 --2- 172.21.15.76:0/935007732 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4fe006b4d0 0x7f4fe006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:44.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.769+0000 7f4ff0ff9640 1 -- 172.21.15.76:0/935007732 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f4fe40895e0 con 0x7f4ff40ce690 2024-03-08T06:02:44.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.770+0000 7f4ff37fe640 1 --2- 172.21.15.76:0/935007732 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4fe006b4d0 0x7f4fe006d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8036000 tx=0x7f4fe8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:44.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.773+0000 7f4ff0ff9640 1 -- 172.21.15.76:0/935007732 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4fe4057c60 con 0x7f4ff40ce690 2024-03-08T06:02:44.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.982+0000 7f4ffa4f4640 1 -- 172.21.15.76:0/935007732 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4ff40cba50 con 0x7f4ff40ce690 2024-03-08T06:02:44.983 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-03-08T06:02:44.982+0000 7fcc89c70640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-08T06:02:44.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.984+0000 7f4ff0ff9640 1 -- 172.21.15.76:0/935007732 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4fe405bd00 con 0x7f4ff40ce690 2024-03-08T06:02:44.984 INFO:teuthology.orchestra.run.smithi076.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-08T06:02:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.987+0000 7f4ffa4f4640 1 -- 172.21.15.76:0/935007732 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4fe006b4d0 msgr2=0x7f4fe006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.987+0000 7f4ffa4f4640 1 --2- 172.21.15.76:0/935007732 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4fe006b4d0 0x7f4fe006d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8036000 tx=0x7f4fe8008030 comp rx=0 tx=0).stop 2024-03-08T06:02:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.987+0000 7f4ffa4f4640 1 -- 172.21.15.76:0/935007732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ff40ce690 msgr2=0x7f4ff415a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.987+0000 7f4ffa4f4640 1 --2- 172.21.15.76:0/935007732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ff40ce690 0x7f4ff415a220 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400c4a0 tx=0x7f4fe400c960 comp rx=0 tx=0).stop 2024-03-08T06:02:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.988+0000 7f4ffa4f4640 1 -- 172.21.15.76:0/935007732 shutdown_connections 2024-03-08T06:02:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.988+0000 7f4ffa4f4640 1 --2- 172.21.15.76:0/935007732 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4fe006b4d0 0x7f4fe006d990 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.988+0000 7f4ffa4f4640 1 --2- 172.21.15.76:0/935007732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ff40d2150 0x7f4ff40cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.988+0000 7f4ffa4f4640 1 --2- 172.21.15.76:0/935007732 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ff40d0800 0x7f4ff40c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.988+0000 7f4ffa4f4640 1 --2- 172.21.15.76:0/935007732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ff40ce690 0x7f4ff415a220 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.988+0000 7f4ffa4f4640 1 -- 172.21.15.76:0/935007732 >> 172.21.15.76:0/935007732 conn(0x7f4ff406ae60 msgr2=0x7f4ff40b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.988+0000 7f4ffa4f4640 1 -- 172.21.15.76:0/935007732 shutdown_connections 2024-03-08T06:02:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:44.988+0000 7f4ffa4f4640 1 -- 172.21.15.76:0/935007732 wait complete. 2024-03-08T06:02:44.999 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-08T06:02:45.027 INFO:tasks.ceph.mds.b.smithi081.stdout:starting mds.b at 2024-03-08T06:02:45.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.102+0000 7fd68affd640 1 Processor -- start 2024-03-08T06:02:45.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.102+0000 7fd68affd640 1 -- start start 2024-03-08T06:02:45.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd68affd640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd68415dbc0 0x7fd6840d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:45.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd68affd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd6840b9840 0x7fd6840b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd68affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd6840b5b20 0x7fd68406c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd68affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6840b5f90 con 0x7fd6840b5b20 2024-03-08T06:02:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd68affd640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6840d27d0 con 0x7fd6840b9840 2024-03-08T06:02:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd68affd640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840d2ab0 con 0x7fd68415dbc0 2024-03-08T06:02:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd68959a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd6840b5b20 0x7fd68406c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd68959a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd6840b5b20 0x7fd68406c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37220/0 (socket says 172.21.15.76:37220) 2024-03-08T06:02:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd68959a640 1 -- 172.21.15.76:0/1647224855 learned_addr learned my addr 172.21.15.76:0/1647224855 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd688d99640 1 --2- 172.21.15.76:0/1647224855 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd68415dbc0 0x7fd6840d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd683fff640 1 --2- 172.21.15.76:0/1647224855 >> v2:172.21.15.81:3300/0 conn(0x7fd6840b9840 0x7fd6840b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd68959a640 1 -- 172.21.15.76:0/1647224855 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd68415dbc0 msgr2=0x7fd6840d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd68959a640 1 --2- 172.21.15.76:0/1647224855 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd68415dbc0 0x7fd6840d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd68959a640 1 -- 172.21.15.76:0/1647224855 >> v2:172.21.15.81:3300/0 conn(0x7fd6840b9840 msgr2=0x7fd6840b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd68959a640 1 --2- 172.21.15.76:0/1647224855 >> v2:172.21.15.81:3300/0 conn(0x7fd6840b9840 0x7fd6840b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:45.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd68959a640 1 -- 172.21.15.76:0/1647224855 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6840d2d90 con 0x7fd6840b5b20 2024-03-08T06:02:45.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.103+0000 7fd683fff640 1 --2- 172.21.15.76:0/1647224855 >> v2:172.21.15.81:3300/0 conn(0x7fd6840b9840 0x7fd6840b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:45.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.104+0000 7fd68959a640 1 --2- 172.21.15.76:0/1647224855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6840b5b20 0x7fd68406c190 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd6840d29e0 tx=0x7fd67c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b371de70a3c0a346 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:45.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.104+0000 7fd6837fe640 1 -- 172.21.15.76:0/1647224855 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c02ca70 con 0x7fd6840b5b20 2024-03-08T06:02:45.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.104+0000 7fd6837fe640 1 -- 172.21.15.76:0/1647224855 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67c02cc10 con 0x7fd6840b5b20 2024-03-08T06:02:45.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.104+0000 7fd6837fe640 1 -- 172.21.15.76:0/1647224855 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c0343f0 con 0x7fd6840b5b20 2024-03-08T06:02:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.104+0000 7fd68affd640 1 -- 172.21.15.76:0/1647224855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6840b5b20 msgr2=0x7fd68406c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.104+0000 7fd68affd640 1 --2- 172.21.15.76:0/1647224855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6840b5b20 0x7fd68406c190 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd6840d29e0 tx=0x7fd67c02bed0 comp rx=0 tx=0).stop 2024-03-08T06:02:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.104+0000 7fd68affd640 1 -- 172.21.15.76:0/1647224855 shutdown_connections 2024-03-08T06:02:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.104+0000 7fd68affd640 1 --2- 172.21.15.76:0/1647224855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6840b5b20 0x7fd68406c190 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.104+0000 7fd68affd640 1 --2- 172.21.15.76:0/1647224855 >> v2:172.21.15.81:3300/0 conn(0x7fd6840b9840 0x7fd6840b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.104+0000 7fd68affd640 1 --2- 172.21.15.76:0/1647224855 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd68415dbc0 0x7fd6840d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.104+0000 7fd68affd640 1 -- 172.21.15.76:0/1647224855 >> 172.21.15.76:0/1647224855 conn(0x7fd6840670d0 msgr2=0x7fd6840d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.105+0000 7fd68affd640 1 -- 172.21.15.76:0/1647224855 shutdown_connections 2024-03-08T06:02:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.105+0000 7fd68affd640 1 -- 172.21.15.76:0/1647224855 wait complete. 2024-03-08T06:02:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.106+0000 7fd68affd640 1 Processor -- start 2024-03-08T06:02:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.106+0000 7fd68affd640 1 -- start start 2024-03-08T06:02:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68affd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6840b5b20 0x7fd6841f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68affd640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd6840b9840 0x7fd6841f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd68415dbc0 0x7fd6841f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68affd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840d2430 con 0x7fd68415dbc0 2024-03-08T06:02:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68affd640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840d3870 con 0x7fd6840b5b20 2024-03-08T06:02:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68affd640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840d25b0 con 0x7fd6840b9840 2024-03-08T06:02:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68959a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd68415dbc0 0x7fd6841f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68959a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd68415dbc0 0x7fd6841f73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37232/0 (socket says 172.21.15.76:37232) 2024-03-08T06:02:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68959a640 1 -- 172.21.15.76:0/3771646918 learned_addr learned my addr 172.21.15.76:0/3771646918 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd688d99640 1 --2- 172.21.15.76:0/3771646918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6840b5b20 0x7fd6841f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd683fff640 1 --2- 172.21.15.76:0/3771646918 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd6840b9840 0x7fd6841f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68959a640 1 -- 172.21.15.76:0/3771646918 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd6840b9840 msgr2=0x7fd6841f6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68959a640 1 --2- 172.21.15.76:0/3771646918 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd6840b9840 0x7fd6841f6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68959a640 1 -- 172.21.15.76:0/3771646918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6840b5b20 msgr2=0x7fd6841f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68959a640 1 --2- 172.21.15.76:0/3771646918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6840b5b20 0x7fd6841f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68959a640 1 -- 172.21.15.76:0/3771646918 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6840cd830 con 0x7fd68415dbc0 2024-03-08T06:02:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd688d99640 1 --2- 172.21.15.76:0/3771646918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6840b5b20 0x7fd6841f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68959a640 1 --2- 172.21.15.76:0/3771646918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd68415dbc0 0x7fd6841f73e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd67c036040 tx=0x7fd67c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd6817fa640 1 -- 172.21.15.76:0/3771646918 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c043020 con 0x7fd68415dbc0 2024-03-08T06:02:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd6817fa640 1 -- 172.21.15.76:0/3771646918 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67c03e450 con 0x7fd68415dbc0 2024-03-08T06:02:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68affd640 1 -- 172.21.15.76:0/3771646918 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6840cd270 con 0x7fd68415dbc0 2024-03-08T06:02:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd6817fa640 1 -- 172.21.15.76:0/3771646918 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c009510 con 0x7fd68415dbc0 2024-03-08T06:02:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.107+0000 7fd68affd640 1 -- 172.21.15.76:0/3771646918 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6841475f0 con 0x7fd68415dbc0 2024-03-08T06:02:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.108+0000 7fd68affd640 1 -- 172.21.15.76:0/3771646918 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd68404f030 con 0x7fd68415dbc0 2024-03-08T06:02:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.109+0000 7fd6817fa640 1 -- 172.21.15.76:0/3771646918 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd67c046020 con 0x7fd68415dbc0 2024-03-08T06:02:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.109+0000 7fd6817fa640 1 --2- 172.21.15.76:0/3771646918 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd66806b4d0 0x7fd66806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.109+0000 7fd6817fa640 1 -- 172.21.15.76:0/3771646918 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fd67c0acba0 con 0x7fd68415dbc0 2024-03-08T06:02:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.109+0000 7fd688d99640 1 --2- 172.21.15.76:0/3771646918 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd66806b4d0 0x7fd66806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.110+0000 7fd688d99640 1 --2- 172.21.15.76:0/3771646918 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd66806b4d0 0x7fd66806d990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd678006fd0 tx=0x7fd678008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:45.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.113+0000 7fd6817fa640 1 -- 172.21.15.76:0/3771646918 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd67c04d030 con 0x7fd68415dbc0 2024-03-08T06:02:45.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.321+0000 7fd68affd640 1 -- 172.21.15.76:0/3771646918 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd6840c7a70 con 0x7fd68415dbc0 2024-03-08T06:02:45.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.985+0000 7fd6817fa640 1 -- 172.21.15.76:0/3771646918 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd67c07b8b0 con 0x7fd68415dbc0 2024-03-08T06:02:45.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.988+0000 7fd68affd640 1 -- 172.21.15.76:0/3771646918 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd66806b4d0 msgr2=0x7fd66806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:45.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.988+0000 7fd68affd640 1 --2- 172.21.15.76:0/3771646918 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd66806b4d0 0x7fd66806d990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd678006fd0 tx=0x7fd678008040 comp rx=0 tx=0).stop 2024-03-08T06:02:45.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.988+0000 7fd68affd640 1 -- 172.21.15.76:0/3771646918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd68415dbc0 msgr2=0x7fd6841f73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:45.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.988+0000 7fd68affd640 1 --2- 172.21.15.76:0/3771646918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd68415dbc0 0x7fd6841f73e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd67c036040 tx=0x7fd67c004a40 comp rx=0 tx=0).stop 2024-03-08T06:02:45.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.988+0000 7fd68affd640 1 -- 172.21.15.76:0/3771646918 shutdown_connections 2024-03-08T06:02:45.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.988+0000 7fd68affd640 1 --2- 172.21.15.76:0/3771646918 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd66806b4d0 0x7fd66806d990 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:45.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.988+0000 7fd68affd640 1 --2- 172.21.15.76:0/3771646918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd68415dbc0 0x7fd6841f73e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:45.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.988+0000 7fd68affd640 1 --2- 172.21.15.76:0/3771646918 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd6840b9840 0x7fd6841f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:45.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.988+0000 7fd68affd640 1 --2- 172.21.15.76:0/3771646918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6840b5b20 0x7fd6841f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:45.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.988+0000 7fd68affd640 1 -- 172.21.15.76:0/3771646918 >> 172.21.15.76:0/3771646918 conn(0x7fd6840670d0 msgr2=0x7fd6840b6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:45.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.988+0000 7fd68affd640 1 -- 172.21.15.76:0/3771646918 shutdown_connections 2024-03-08T06:02:45.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:45.988+0000 7fd68affd640 1 -- 172.21.15.76:0/3771646918 wait complete. 2024-03-08T06:02:46.001 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-08T06:02:46.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.108+0000 7ff7a6064640 1 Processor -- start 2024-03-08T06:02:46.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.108+0000 7ff7a6064640 1 -- start start 2024-03-08T06:02:46.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.109+0000 7ff7a6064640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7a006eaa0 0x7ff7a006ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:46.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.109+0000 7ff7a6064640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a015e740 con 0x7ff7a0154d90 2024-03-08T06:02:46.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.109+0000 7ff7a6064640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a015ba90 con 0x7ff7a0155170 2024-03-08T06:02:46.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.109+0000 7ff7a6064640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a006f450 con 0x7ff7a006eaa0 2024-03-08T06:02:46.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.109+0000 7ff79f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff7a0154d90 0x7ff7a00b31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40508/0 (socket says 172.21.15.76:40508) 2024-03-08T06:02:46.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.109+0000 7ff79f7fe640 1 -- 172.21.15.76:0/2219625378 learned_addr learned my addr 172.21.15.76:0/2219625378 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:46.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.109+0000 7ff79e7fc640 1 -- 172.21.15.76:0/2219625378 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276194944 0 0) 0x7ff7a015e740 con 0x7ff7a0154d90 2024-03-08T06:02:46.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.109+0000 7ff79e7fc640 1 -- 172.21.15.76:0/2219625378 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff788003520 con 0x7ff7a0154d90 2024-03-08T06:02:46.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.109+0000 7ff79ffff640 1 --2- 172.21.15.76:0/2219625378 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7a006eaa0 0x7ff7a006ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:46.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.109+0000 7ff79e7fc640 1 -- 172.21.15.76:0/2219625378 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2467104083 0 0) 0x7ff788003520 con 0x7ff7a0154d90 2024-03-08T06:02:46.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.109+0000 7ff79e7fc640 1 -- 172.21.15.76:0/2219625378 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7a006eaa0 msgr2=0x7ff7a006ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:46.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.109+0000 7ff79e7fc640 1 --2- 172.21.15.76:0/2219625378 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7a006eaa0 0x7ff7a006ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.109+0000 7ff79e7fc640 1 -- 172.21.15.76:0/2219625378 >> v1:172.21.15.81:6789/0 conn(0x7ff7a0155170 legacy=0x7ff7a015af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.109+0000 7ff79e7fc640 1 -- 172.21.15.76:0/2219625378 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a015bd70 con 0x7ff7a0154d90 2024-03-08T06:02:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.109+0000 7ff79e7fc640 1 -- 172.21.15.76:0/2219625378 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7ff790001670 con 0x7ff7a0154d90 2024-03-08T06:02:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.110+0000 7ff79e7fc640 1 -- 172.21.15.76:0/2219625378 >> v1:172.21.15.76:6789/0 conn(0x7ff7a0154d90 legacy=0x7ff7a00b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.110+0000 7ff79e7fc640 1 --2- 172.21.15.76:0/2219625378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7880039e0 0x7ff788003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.110+0000 7ff79e7fc640 1 --2- 172.21.15.76:0/2219625378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff788004800 0x7ff788004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.110+0000 7ff79e7fc640 1 --2- 172.21.15.76:0/2219625378 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7a006eaa0 0x7ff7880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.110+0000 7ff79e7fc640 1 -- 172.21.15.76:0/2219625378 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a006f450 con 0x7ff7880039e0 2024-03-08T06:02:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.110+0000 7ff79e7fc640 1 -- 172.21.15.76:0/2219625378 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78800dc80 con 0x7ff788004800 2024-03-08T06:02:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.110+0000 7ff79e7fc640 1 -- 172.21.15.76:0/2219625378 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78800de80 con 0x7ff7a006eaa0 2024-03-08T06:02:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.110+0000 7ff79f7fe640 1 --2- 172.21.15.76:0/2219625378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7880039e0 0x7ff788003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.110+0000 7ff79effd640 1 --2- 172.21.15.76:0/2219625378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff788004800 0x7ff788004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.110+0000 7ff79ffff640 1 --2- 172.21.15.76:0/2219625378 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7a006eaa0 0x7ff7880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.110+0000 7ff79f7fe640 1 -- 172.21.15.76:0/2219625378 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7a006eaa0 msgr2=0x7ff7880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.110+0000 7ff79f7fe640 1 --2- 172.21.15.76:0/2219625378 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7a006eaa0 0x7ff7880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.110+0000 7ff79f7fe640 1 -- 172.21.15.76:0/2219625378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff788004800 msgr2=0x7ff788004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:46.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.110+0000 7ff79f7fe640 1 --2- 172.21.15.76:0/2219625378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff788004800 0x7ff788004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:46.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.110+0000 7ff79f7fe640 1 -- 172.21.15.76:0/2219625378 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff788003520 con 0x7ff7880039e0 2024-03-08T06:02:46.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.110+0000 7ff79ffff640 1 --2- 172.21.15.76:0/2219625378 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7a006eaa0 0x7ff7880051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:46.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.111+0000 7ff79f7fe640 1 --2- 172.21.15.76:0/2219625378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7880039e0 0x7ff788003e00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff79000c210 tx=0x7ff788005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7ded8ff6f8c21f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:46.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.111+0000 7ff79e7fc640 1 -- 172.21.15.76:0/2219625378 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790007cf0 con 0x7ff7880039e0 2024-03-08T06:02:46.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.111+0000 7ff79e7fc640 1 -- 172.21.15.76:0/2219625378 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff790030070 con 0x7ff7880039e0 2024-03-08T06:02:46.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.111+0000 7ff79e7fc640 1 -- 172.21.15.76:0/2219625378 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790005d30 con 0x7ff7880039e0 2024-03-08T06:02:46.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.111+0000 7ff7a6064640 1 -- 172.21.15.76:0/2219625378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7880039e0 msgr2=0x7ff788003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:46.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.111+0000 7ff7a6064640 1 --2- 172.21.15.76:0/2219625378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7880039e0 0x7ff788003e00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff79000c210 tx=0x7ff788005830 comp rx=0 tx=0).stop 2024-03-08T06:02:46.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.111+0000 7ff79ffff640 1 -- 172.21.15.76:0/2219625378 reap_dead start 2024-03-08T06:02:46.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.111+0000 7ff7a6064640 1 -- 172.21.15.76:0/2219625378 shutdown_connections 2024-03-08T06:02:46.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.111+0000 7ff7a6064640 1 -- 172.21.15.76:0/2219625378 >> 172.21.15.76:0/2219625378 conn(0x7ff7a006ac80 msgr2=0x7ff7a006b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:46.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.111+0000 7ff7a6064640 1 -- 172.21.15.76:0/2219625378 shutdown_connections 2024-03-08T06:02:46.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.111+0000 7ff7a6064640 1 -- 172.21.15.76:0/2219625378 wait complete. 2024-03-08T06:02:46.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.113+0000 7ff7a6064640 1 Processor -- start 2024-03-08T06:02:46.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.113+0000 7ff7a6064640 1 -- start start 2024-03-08T06:02:46.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.113+0000 7ff7a6064640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7880039e0 0x7ff7a00b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:46.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.113+0000 7ff7a6064640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7a0144a60 0x7ff7a00ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:46.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.113+0000 7ff7a6064640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7a00adf30 0x7ff7a01473b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:46.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.113+0000 7ff7a6064640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a00b29f0 con 0x7ff7880039e0 2024-03-08T06:02:46.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.113+0000 7ff7a6064640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a00b2d60 con 0x7ff7a00adf30 2024-03-08T06:02:46.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.113+0000 7ff7a6064640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a015b630 con 0x7ff7a0144a60 2024-03-08T06:02:46.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.113+0000 7ff79f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7880039e0 0x7ff7a00b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:46.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.114+0000 7ff79f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7880039e0 0x7ff7a00b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37242/0 (socket says 172.21.15.76:37242) 2024-03-08T06:02:46.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.114+0000 7ff79f7fe640 1 -- 172.21.15.76:0/3402266673 learned_addr learned my addr 172.21.15.76:0/3402266673 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:46.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.114+0000 7ff79effd640 1 --2- 172.21.15.76:0/3402266673 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7a0144a60 0x7ff7a00ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:46.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.114+0000 7ff79ffff640 1 --2- 172.21.15.76:0/3402266673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7a00adf30 0x7ff7a01473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:46.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.114+0000 7ff79f7fe640 1 -- 172.21.15.76:0/3402266673 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7a0144a60 msgr2=0x7ff7a00ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:46.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.114+0000 7ff79f7fe640 1 --2- 172.21.15.76:0/3402266673 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7a0144a60 0x7ff7a00ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:46.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.114+0000 7ff79f7fe640 1 -- 172.21.15.76:0/3402266673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7a00adf30 msgr2=0x7ff7a01473b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:46.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.114+0000 7ff79f7fe640 1 --2- 172.21.15.76:0/3402266673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7a00adf30 0x7ff7a01473b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:46.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.114+0000 7ff79f7fe640 1 -- 172.21.15.76:0/3402266673 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a0148120 con 0x7ff7880039e0 2024-03-08T06:02:46.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.114+0000 7ff79ffff640 1 --2- 172.21.15.76:0/3402266673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7a00adf30 0x7ff7a01473b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:46.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.114+0000 7ff79f7fe640 1 --2- 172.21.15.76:0/3402266673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7880039e0 0x7ff7a00b1e40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff79002c4e0 tx=0x7ff7a015bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:46.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.114+0000 7ff787fff640 1 -- 172.21.15.76:0/3402266673 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790045070 con 0x7ff7880039e0 2024-03-08T06:02:46.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.114+0000 7ff787fff640 1 -- 172.21.15.76:0/3402266673 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff790004bf0 con 0x7ff7880039e0 2024-03-08T06:02:46.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.114+0000 7ff787fff640 1 -- 172.21.15.76:0/3402266673 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790007cf0 con 0x7ff7880039e0 2024-03-08T06:02:46.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.114+0000 7ff7a6064640 1 -- 172.21.15.76:0/3402266673 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a00b1360 con 0x7ff7880039e0 2024-03-08T06:02:46.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.114+0000 7ff7a6064640 1 -- 172.21.15.76:0/3402266673 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a00b1920 con 0x7ff7880039e0 2024-03-08T06:02:46.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.115+0000 7ff7a6064640 1 -- 172.21.15.76:0/3402266673 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a004f030 con 0x7ff7880039e0 2024-03-08T06:02:46.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.115+0000 7ff787fff640 1 -- 172.21.15.76:0/3402266673 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff790007070 con 0x7ff7880039e0 2024-03-08T06:02:46.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.116+0000 7ff787fff640 1 --2- 172.21.15.76:0/3402266673 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff78c06b500 0x7ff78c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:46.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.116+0000 7ff787fff640 1 -- 172.21.15.76:0/3402266673 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6216+0+0 (secure 0 0 0) 0x7ff7900ab2d0 con 0x7ff7880039e0 2024-03-08T06:02:46.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.116+0000 7ff79effd640 1 --2- 172.21.15.76:0/3402266673 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff78c06b500 0x7ff78c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:46.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.116+0000 7ff79effd640 1 --2- 172.21.15.76:0/3402266673 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff78c06b500 0x7ff78c06d9c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00ac8f0 tx=0x7ff794009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:46.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.120+0000 7ff787fff640 1 -- 172.21.15.76:0/3402266673 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff7900799e0 con 0x7ff7880039e0 2024-03-08T06:02:46.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:46.291+0000 7ff7a6064640 1 -- 172.21.15.76:0/3402266673 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff7a00ae310 con 0x7ff7880039e0 2024-03-08T06:02:47.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.085+0000 7ff787fff640 1 -- 172.21.15.76:0/3402266673 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff79007da80 con 0x7ff7880039e0 2024-03-08T06:02:47.086 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' removed 2024-03-08T06:02:47.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.088+0000 7ff7a6064640 1 -- 172.21.15.76:0/3402266673 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff78c06b500 msgr2=0x7ff78c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:47.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.088+0000 7ff7a6064640 1 --2- 172.21.15.76:0/3402266673 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff78c06b500 0x7ff78c06d9c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00ac8f0 tx=0x7ff794009040 comp rx=0 tx=0).stop 2024-03-08T06:02:47.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.088+0000 7ff7a6064640 1 -- 172.21.15.76:0/3402266673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7880039e0 msgr2=0x7ff7a00b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:47.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.088+0000 7ff7a6064640 1 --2- 172.21.15.76:0/3402266673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7880039e0 0x7ff7a00b1e40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff79002c4e0 tx=0x7ff7a015bc60 comp rx=0 tx=0).stop 2024-03-08T06:02:47.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.088+0000 7ff7a6064640 1 -- 172.21.15.76:0/3402266673 shutdown_connections 2024-03-08T06:02:47.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.088+0000 7ff7a6064640 1 --2- 172.21.15.76:0/3402266673 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ff78c06b500 0x7ff78c06d9c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:47.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.088+0000 7ff7a6064640 1 --2- 172.21.15.76:0/3402266673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7a00adf30 0x7ff7a01473b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:47.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.088+0000 7ff7a6064640 1 --2- 172.21.15.76:0/3402266673 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7a0144a60 0x7ff7a00ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:47.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.088+0000 7ff7a6064640 1 --2- 172.21.15.76:0/3402266673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7880039e0 0x7ff7a00b1e40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:47.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.088+0000 7ff7a6064640 1 -- 172.21.15.76:0/3402266673 >> 172.21.15.76:0/3402266673 conn(0x7ff7a006ac80 msgr2=0x7ff7a015d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:47.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.088+0000 7ff7a6064640 1 -- 172.21.15.76:0/3402266673 shutdown_connections 2024-03-08T06:02:47.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.088+0000 7ff7a6064640 1 -- 172.21.15.76:0/3402266673 wait complete. 2024-03-08T06:02:47.104 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-08T06:02:47.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.213+0000 7fb19b247640 1 Processor -- start 2024-03-08T06:02:47.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.213+0000 7fb19b247640 1 -- start start 2024-03-08T06:02:47.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.214+0000 7fb19b247640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb19415ddd0 0x7fb1940ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:47.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.214+0000 7fb19b247640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1940ce540 0x7fb1940d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:47.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.214+0000 7fb19b247640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1940cf080 0x7fb1940d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:47.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.214+0000 7fb19b247640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1940d2a80 con 0x7fb1940ce540 2024-03-08T06:02:47.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.214+0000 7fb19b247640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb19415e1b0 con 0x7fb19415ddd0 2024-03-08T06:02:47.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.214+0000 7fb19b247640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1940d2cd0 con 0x7fb1940cf080 2024-03-08T06:02:47.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.214+0000 7fb193fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1940ce540 0x7fb1940d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:47.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.214+0000 7fb193fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1940ce540 0x7fb1940d0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37248/0 (socket says 172.21.15.76:37248) 2024-03-08T06:02:47.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.214+0000 7fb193fff640 1 -- 172.21.15.76:0/2037150922 learned_addr learned my addr 172.21.15.76:0/2037150922 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.215+0000 7fb1997e4640 1 --2- 172.21.15.76:0/2037150922 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1940cf080 0x7fb1940d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.215+0000 7fb193fff640 1 -- 172.21.15.76:0/2037150922 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1940cf080 msgr2=0x7fb1940d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.215+0000 7fb193fff640 1 --2- 172.21.15.76:0/2037150922 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1940cf080 0x7fb1940d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.215+0000 7fb193fff640 1 -- 172.21.15.76:0/2037150922 >> v2:172.21.15.81:3300/0 conn(0x7fb19415ddd0 msgr2=0x7fb1940ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.215+0000 7fb198fe3640 1 --2- 172.21.15.76:0/2037150922 >> v2:172.21.15.81:3300/0 conn(0x7fb19415ddd0 0x7fb1940ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.215+0000 7fb193fff640 1 --2- 172.21.15.76:0/2037150922 >> v2:172.21.15.81:3300/0 conn(0x7fb19415ddd0 0x7fb1940ceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.215+0000 7fb193fff640 1 -- 172.21.15.76:0/2037150922 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1940d2fb0 con 0x7fb1940ce540 2024-03-08T06:02:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.215+0000 7fb193fff640 1 --2- 172.21.15.76:0/2037150922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1940ce540 0x7fb1940d0500 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb1940b6530 tx=0x7fb18802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc514d320a206d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.216+0000 7fb1937fe640 1 -- 172.21.15.76:0/2037150922 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18802e070 con 0x7fb1940ce540 2024-03-08T06:02:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.216+0000 7fb1937fe640 1 -- 172.21.15.76:0/2037150922 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb188037070 con 0x7fb1940ce540 2024-03-08T06:02:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.216+0000 7fb1937fe640 1 -- 172.21.15.76:0/2037150922 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188031a00 con 0x7fb1940ce540 2024-03-08T06:02:47.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.216+0000 7fb19b247640 1 -- 172.21.15.76:0/2037150922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1940ce540 msgr2=0x7fb1940d0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:47.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.216+0000 7fb19b247640 1 --2- 172.21.15.76:0/2037150922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1940ce540 0x7fb1940d0500 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb1940b6530 tx=0x7fb18802b880 comp rx=0 tx=0).stop 2024-03-08T06:02:47.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.216+0000 7fb19b247640 1 -- 172.21.15.76:0/2037150922 shutdown_connections 2024-03-08T06:02:47.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.216+0000 7fb19b247640 1 --2- 172.21.15.76:0/2037150922 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1940cf080 0x7fb1940d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:47.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.216+0000 7fb19b247640 1 --2- 172.21.15.76:0/2037150922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1940ce540 0x7fb1940d0500 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:47.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.216+0000 7fb19b247640 1 --2- 172.21.15.76:0/2037150922 >> v2:172.21.15.81:3300/0 conn(0x7fb19415ddd0 0x7fb1940ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:47.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.216+0000 7fb19b247640 1 -- 172.21.15.76:0/2037150922 >> 172.21.15.76:0/2037150922 conn(0x7fb194066fc0 msgr2=0x7fb1940b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:47.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.216+0000 7fb19b247640 1 -- 172.21.15.76:0/2037150922 shutdown_connections 2024-03-08T06:02:47.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.216+0000 7fb19b247640 1 -- 172.21.15.76:0/2037150922 wait complete. 2024-03-08T06:02:47.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.217+0000 7fb19b247640 1 Processor -- start 2024-03-08T06:02:47.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.217+0000 7fb19b247640 1 -- start start 2024-03-08T06:02:47.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.218+0000 7fb19b247640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1940ce540 0x7fb19414c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:47.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.218+0000 7fb19b247640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1940cf080 0x7fb19415edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:47.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.218+0000 7fb19b247640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19415ddd0 0x7fb19415f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:47.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.218+0000 7fb19b247640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19406c390 con 0x7fb19415ddd0 2024-03-08T06:02:47.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.218+0000 7fb19b247640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1940d3570 con 0x7fb1940ce540 2024-03-08T06:02:47.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.218+0000 7fb19b247640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19406c510 con 0x7fb1940cf080 2024-03-08T06:02:47.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.218+0000 7fb1997e4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19415ddd0 0x7fb19415f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:47.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.218+0000 7fb1997e4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19415ddd0 0x7fb19415f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37250/0 (socket says 172.21.15.76:37250) 2024-03-08T06:02:47.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.218+0000 7fb1997e4640 1 -- 172.21.15.76:0/572322350 learned_addr learned my addr 172.21.15.76:0/572322350 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:47.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.219+0000 7fb193fff640 1 --2- 172.21.15.76:0/572322350 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1940cf080 0x7fb19415edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:47.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.219+0000 7fb198fe3640 1 --2- 172.21.15.76:0/572322350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1940ce540 0x7fb19414c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:47.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.219+0000 7fb1997e4640 1 -- 172.21.15.76:0/572322350 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1940cf080 msgr2=0x7fb19415edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:47.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.219+0000 7fb1997e4640 1 --2- 172.21.15.76:0/572322350 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1940cf080 0x7fb19415edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:47.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.219+0000 7fb1997e4640 1 -- 172.21.15.76:0/572322350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1940ce540 msgr2=0x7fb19414c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:47.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.219+0000 7fb1997e4640 1 --2- 172.21.15.76:0/572322350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1940ce540 0x7fb19414c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:47.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.219+0000 7fb1997e4640 1 -- 172.21.15.76:0/572322350 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1940cd6b0 con 0x7fb19415ddd0 2024-03-08T06:02:47.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.219+0000 7fb193fff640 1 --2- 172.21.15.76:0/572322350 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1940cf080 0x7fb19415edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:47.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.219+0000 7fb198fe3640 1 --2- 172.21.15.76:0/572322350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1940ce540 0x7fb19414c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:47.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.219+0000 7fb1997e4640 1 --2- 172.21.15.76:0/572322350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19415ddd0 0x7fb19415f310 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00ccf0 tx=0x7fb18c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:47.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.219+0000 7fb1917fa640 1 -- 172.21.15.76:0/572322350 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c003b90 con 0x7fb19415ddd0 2024-03-08T06:02:47.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.219+0000 7fb1917fa640 1 -- 172.21.15.76:0/572322350 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18c003d30 con 0x7fb19415ddd0 2024-03-08T06:02:47.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.219+0000 7fb1917fa640 1 -- 172.21.15.76:0/572322350 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c004240 con 0x7fb19415ddd0 2024-03-08T06:02:47.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.219+0000 7fb19b247640 1 -- 172.21.15.76:0/572322350 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19414c8b0 con 0x7fb19415ddd0 2024-03-08T06:02:47.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.219+0000 7fb19b247640 1 -- 172.21.15.76:0/572322350 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19414ce70 con 0x7fb19415ddd0 2024-03-08T06:02:47.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.220+0000 7fb19b247640 1 -- 172.21.15.76:0/572322350 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19404f030 con 0x7fb19415ddd0 2024-03-08T06:02:47.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.221+0000 7fb1917fa640 1 -- 172.21.15.76:0/572322350 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb18c0043e0 con 0x7fb19415ddd0 2024-03-08T06:02:47.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.221+0000 7fb1917fa640 1 --2- 172.21.15.76:0/572322350 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb18006b4d0 0x7fb18006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:47.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.221+0000 7fb1917fa640 1 -- 172.21.15.76:0/572322350 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5778+0+0 (secure 0 0 0) 0x7fb18c08ad50 con 0x7fb19415ddd0 2024-03-08T06:02:47.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.221+0000 7fb198fe3640 1 --2- 172.21.15.76:0/572322350 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb18006b4d0 0x7fb18006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:47.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.221+0000 7fb198fe3640 1 --2- 172.21.15.76:0/572322350 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb18006b4d0 0x7fb18006d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb18400b820 tx=0x7fb184008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:47.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.225+0000 7fb1917fa640 1 -- 172.21.15.76:0/572322350 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb18c08e030 con 0x7fb19415ddd0 2024-03-08T06:02:47.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:47.399+0000 7fb19b247640 1 -- 172.21.15.76:0/572322350 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb1941553e0 con 0x7fb19415ddd0 2024-03-08T06:02:48.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.092+0000 7fb1917fa640 1 -- 172.21.15.76:0/572322350 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb18c059b80 con 0x7fb19415ddd0 2024-03-08T06:02:48.093 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' removed 2024-03-08T06:02:48.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.096+0000 7fb19b247640 1 -- 172.21.15.76:0/572322350 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb18006b4d0 msgr2=0x7fb18006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:48.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.096+0000 7fb19b247640 1 --2- 172.21.15.76:0/572322350 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb18006b4d0 0x7fb18006d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb18400b820 tx=0x7fb184008040 comp rx=0 tx=0).stop 2024-03-08T06:02:48.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.096+0000 7fb19b247640 1 -- 172.21.15.76:0/572322350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19415ddd0 msgr2=0x7fb19415f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.096+0000 7fb19b247640 1 --2- 172.21.15.76:0/572322350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19415ddd0 0x7fb19415f310 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00ccf0 tx=0x7fb18c00a660 comp rx=0 tx=0).stop 2024-03-08T06:02:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.097+0000 7fb19b247640 1 -- 172.21.15.76:0/572322350 shutdown_connections 2024-03-08T06:02:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.097+0000 7fb19b247640 1 --2- 172.21.15.76:0/572322350 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb18006b4d0 0x7fb18006d990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.097+0000 7fb19b247640 1 --2- 172.21.15.76:0/572322350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19415ddd0 0x7fb19415f310 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.097+0000 7fb19b247640 1 --2- 172.21.15.76:0/572322350 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1940cf080 0x7fb19415edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.097+0000 7fb19b247640 1 --2- 172.21.15.76:0/572322350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1940ce540 0x7fb19414c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.097+0000 7fb19b247640 1 -- 172.21.15.76:0/572322350 >> 172.21.15.76:0/572322350 conn(0x7fb194066fc0 msgr2=0x7fb19406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.097+0000 7fb19b247640 1 -- 172.21.15.76:0/572322350 shutdown_connections 2024-03-08T06:02:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.097+0000 7fb19b247640 1 -- 172.21.15.76:0/572322350 wait complete. 2024-03-08T06:02:48.117 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-08T06:02:48.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.222+0000 7f4ad5023640 1 Processor -- start 2024-03-08T06:02:48.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.222+0000 7f4ad5023640 1 -- start start 2024-03-08T06:02:48.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.223+0000 7f4ad5023640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad00ce5e0 0x7f4ad006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:48.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.223+0000 7f4ad5023640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4ad00ce9c0 0x7f4ad0162020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:48.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.223+0000 7f4ad5023640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad0161840 con 0x7f4ad0162560 2024-03-08T06:02:48.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.223+0000 7f4ad5023640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad0161b90 con 0x7f4ad00ce9c0 2024-03-08T06:02:48.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.223+0000 7f4ad5023640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad015eb30 con 0x7f4ad00ce5e0 2024-03-08T06:02:48.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.223+0000 7f4ace59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4ad00ce9c0 0x7f4ad0162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:48.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.223+0000 7f4ace59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4ad00ce9c0 0x7f4ad0162020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37254/0 (socket says 172.21.15.76:37254) 2024-03-08T06:02:48.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.223+0000 7f4ace59c640 1 -- 172.21.15.76:0/4199390282 learned_addr learned my addr 172.21.15.76:0/4199390282 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:48.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.223+0000 7f4aced9d640 1 --2- 172.21.15.76:0/4199390282 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad00ce5e0 0x7f4ad006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:48.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.223+0000 7f4ace59c640 1 -- 172.21.15.76:0/4199390282 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad00ce5e0 msgr2=0x7f4ad006d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:48.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.223+0000 7f4ace59c640 1 --2- 172.21.15.76:0/4199390282 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad00ce5e0 0x7f4ad006d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.223+0000 7f4ace59c640 1 -- 172.21.15.76:0/4199390282 >> v1:172.21.15.81:6789/0 conn(0x7f4ad0162560 legacy=0x7f4ad015dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:48.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.223+0000 7f4ace59c640 1 -- 172.21.15.76:0/4199390282 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad015ee10 con 0x7f4ad00ce9c0 2024-03-08T06:02:48.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.224+0000 7f4ace59c640 1 --2- 172.21.15.76:0/4199390282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ad00ce9c0 0x7f4ad0162020 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009870 tx=0x7f4ac402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1875239f034ea97 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:48.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.224+0000 7f4acdd9b640 1 -- 172.21.15.76:0/4199390282 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4034070 con 0x7f4ad00ce9c0 2024-03-08T06:02:48.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.224+0000 7f4acdd9b640 1 -- 172.21.15.76:0/4199390282 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac4030840 con 0x7f4ad00ce9c0 2024-03-08T06:02:48.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.224+0000 7f4acdd9b640 1 -- 172.21.15.76:0/4199390282 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4030c10 con 0x7f4ad00ce9c0 2024-03-08T06:02:48.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.225+0000 7f4ad5023640 1 -- 172.21.15.76:0/4199390282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ad00ce9c0 msgr2=0x7f4ad0162020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:48.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.225+0000 7f4ad5023640 1 --2- 172.21.15.76:0/4199390282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ad00ce9c0 0x7f4ad0162020 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009870 tx=0x7f4ac402b980 comp rx=0 tx=0).stop 2024-03-08T06:02:48.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.225+0000 7f4ad5023640 1 -- 172.21.15.76:0/4199390282 shutdown_connections 2024-03-08T06:02:48.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.225+0000 7f4ad5023640 1 --2- 172.21.15.76:0/4199390282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ad00ce9c0 0x7f4ad0162020 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.225+0000 7f4ad5023640 1 --2- 172.21.15.76:0/4199390282 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad00ce5e0 0x7f4ad006d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.225+0000 7f4ad5023640 1 -- 172.21.15.76:0/4199390282 >> 172.21.15.76:0/4199390282 conn(0x7f4ad006ae60 msgr2=0x7f4ad006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:48.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.225+0000 7f4ad5023640 1 -- 172.21.15.76:0/4199390282 shutdown_connections 2024-03-08T06:02:48.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.225+0000 7f4ad5023640 1 -- 172.21.15.76:0/4199390282 wait complete. 2024-03-08T06:02:48.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.226+0000 7f4ad5023640 1 Processor -- start 2024-03-08T06:02:48.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.226+0000 7f4ad5023640 1 -- start start 2024-03-08T06:02:48.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.227+0000 7f4ad5023640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad00ce5e0 0x7f4ad014cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:48.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.227+0000 7f4ad5023640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ad00ce9c0 0x7f4ad00cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:48.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.227+0000 7f4ad5023640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ad0162560 0x7f4ad014f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:48.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.227+0000 7f4ad5023640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad015e6c0 con 0x7f4ad0162560 2024-03-08T06:02:48.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.227+0000 7f4ad5023640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad00cee20 con 0x7f4ad00ce9c0 2024-03-08T06:02:48.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.227+0000 7f4ad5023640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad015e840 con 0x7f4ad00ce5e0 2024-03-08T06:02:48.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.227+0000 7f4acf59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ad0162560 0x7f4ad014f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:48.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.227+0000 7f4acf59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ad0162560 0x7f4ad014f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37268/0 (socket says 172.21.15.76:37268) 2024-03-08T06:02:48.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.227+0000 7f4acf59e640 1 -- 172.21.15.76:0/2213798284 learned_addr learned my addr 172.21.15.76:0/2213798284 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:48.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.227+0000 7f4aced9d640 1 --2- 172.21.15.76:0/2213798284 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad00ce5e0 0x7f4ad014cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:48.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.227+0000 7f4ace59c640 1 --2- 172.21.15.76:0/2213798284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ad00ce9c0 0x7f4ad00cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:48.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.227+0000 7f4acf59e640 1 -- 172.21.15.76:0/2213798284 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad00ce5e0 msgr2=0x7f4ad014cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:48.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.227+0000 7f4acf59e640 1 --2- 172.21.15.76:0/2213798284 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad00ce5e0 0x7f4ad014cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.227+0000 7f4acf59e640 1 -- 172.21.15.76:0/2213798284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ad00ce9c0 msgr2=0x7f4ad00cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:48.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.227+0000 7f4acf59e640 1 --2- 172.21.15.76:0/2213798284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ad00ce9c0 0x7f4ad00cb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.227+0000 7f4acf59e640 1 -- 172.21.15.76:0/2213798284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad014d2e0 con 0x7f4ad0162560 2024-03-08T06:02:48.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.228+0000 7f4acf59e640 1 --2- 172.21.15.76:0/2213798284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ad0162560 0x7f4ad014f710 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800c700 tx=0x7f4ac800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:48.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.228+0000 7f4ab77fe640 1 -- 172.21.15.76:0/2213798284 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8016020 con 0x7f4ad0162560 2024-03-08T06:02:48.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.228+0000 7f4ab77fe640 1 -- 172.21.15.76:0/2213798284 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac800aa50 con 0x7f4ad0162560 2024-03-08T06:02:48.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.228+0000 7f4ad5023640 1 -- 172.21.15.76:0/2213798284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad015d440 con 0x7f4ad0162560 2024-03-08T06:02:48.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.228+0000 7f4ab77fe640 1 -- 172.21.15.76:0/2213798284 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac800ce10 con 0x7f4ad0162560 2024-03-08T06:02:48.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.228+0000 7f4ad5023640 1 -- 172.21.15.76:0/2213798284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad014ec30 con 0x7f4ad0162560 2024-03-08T06:02:48.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.229+0000 7f4ad5023640 1 -- 172.21.15.76:0/2213798284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad00b7790 con 0x7f4ad0162560 2024-03-08T06:02:48.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.229+0000 7f4ab77fe640 1 -- 172.21.15.76:0/2213798284 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ac8019020 con 0x7f4ad0162560 2024-03-08T06:02:48.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.229+0000 7f4ab77fe640 1 --2- 172.21.15.76:0/2213798284 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4abc06b4d0 0x7f4abc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:48.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.230+0000 7f4ab77fe640 1 -- 172.21.15.76:0/2213798284 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4ac808a3e0 con 0x7f4ad0162560 2024-03-08T06:02:48.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.230+0000 7f4aced9d640 1 --2- 172.21.15.76:0/2213798284 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4abc06b4d0 0x7f4abc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:48.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.230+0000 7f4aced9d640 1 --2- 172.21.15.76:0/2213798284 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4abc06b4d0 0x7f4abc06d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0006fd0 tx=0x7f4ac0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:48.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.234+0000 7f4ab77fe640 1 -- 172.21.15.76:0/2213798284 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ac8020020 con 0x7f4ad0162560 2024-03-08T06:02:48.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.403+0000 7f4ad5023640 1 -- 172.21.15.76:0/2213798284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4ad00cb8f0 con 0x7f4ad0162560 2024-03-08T06:02:48.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.406+0000 7f4ab77fe640 1 -- 172.21.15.76:0/2213798284 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4ad014d2e0 con 0x7f4ad0162560 2024-03-08T06:02:48.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.408+0000 7f4ad5023640 1 -- 172.21.15.76:0/2213798284 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4abc06b4d0 msgr2=0x7f4abc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:48.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.408+0000 7f4ad5023640 1 --2- 172.21.15.76:0/2213798284 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4abc06b4d0 0x7f4abc06d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0006fd0 tx=0x7f4ac0008040 comp rx=0 tx=0).stop 2024-03-08T06:02:48.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.408+0000 7f4ad5023640 1 -- 172.21.15.76:0/2213798284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ad0162560 msgr2=0x7f4ad014f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:48.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.408+0000 7f4ad5023640 1 --2- 172.21.15.76:0/2213798284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ad0162560 0x7f4ad014f710 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800c700 tx=0x7f4ac800cbc0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.408+0000 7f4ad5023640 1 -- 172.21.15.76:0/2213798284 shutdown_connections 2024-03-08T06:02:48.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.408+0000 7f4ad5023640 1 --2- 172.21.15.76:0/2213798284 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4abc06b4d0 0x7f4abc06d990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.408+0000 7f4ad5023640 1 --2- 172.21.15.76:0/2213798284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ad0162560 0x7f4ad014f710 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.408+0000 7f4ad5023640 1 --2- 172.21.15.76:0/2213798284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ad00ce9c0 0x7f4ad00cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.408+0000 7f4ad5023640 1 --2- 172.21.15.76:0/2213798284 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad00ce5e0 0x7f4ad014cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.408+0000 7f4ad5023640 1 -- 172.21.15.76:0/2213798284 >> 172.21.15.76:0/2213798284 conn(0x7f4ad006ae60 msgr2=0x7f4ad00d1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:48.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.408+0000 7f4ad5023640 1 -- 172.21.15.76:0/2213798284 shutdown_connections 2024-03-08T06:02:48.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.408+0000 7f4ad5023640 1 -- 172.21.15.76:0/2213798284 wait complete. 2024-03-08T06:02:48.420 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters' 2024-03-08T06:02:48.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.520+0000 7f5f140d4640 1 Processor -- start 2024-03-08T06:02:48.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.520+0000 7f5f140d4640 1 -- start start 2024-03-08T06:02:48.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f140d4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5f0c15dc60 0x7f5f0c0d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:48.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f140d4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5f0c15e040 0x7f5f0c099b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:48.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f140d4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f0c06bfa0 0x7f5f0c09a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:48.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f140d4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f0c0b4360 con 0x7f5f0c15dc60 2024-03-08T06:02:48.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f140d4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f0c0cfcc0 con 0x7f5f0c15e040 2024-03-08T06:02:48.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f140d4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c0cff60 con 0x7f5f0c06bfa0 2024-03-08T06:02:48.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f11e70640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5f0c15dc60 0x7f5f0c0d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:48.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f11e70640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5f0c15dc60 0x7f5f0c0d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48434/0 (socket says 172.21.15.76:48434) 2024-03-08T06:02:48.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f11e70640 1 -- 172.21.15.76:0/1608937600 learned_addr learned my addr 172.21.15.76:0/1608937600 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:48.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f12671640 1 --2- 172.21.15.76:0/1608937600 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f0c06bfa0 0x7f5f0c09a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:48.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f1166f640 1 --2- 172.21.15.76:0/1608937600 >> v2:172.21.15.81:3300/0 conn(0x7f5f0c15e040 0x7f5f0c099b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:48.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f11e70640 1 -- 172.21.15.76:0/1608937600 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f0c06bfa0 msgr2=0x7f5f0c09a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:48.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f11e70640 1 --2- 172.21.15.76:0/1608937600 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f0c06bfa0 0x7f5f0c09a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f11e70640 1 -- 172.21.15.76:0/1608937600 >> v2:172.21.15.81:3300/0 conn(0x7f5f0c15e040 msgr2=0x7f5f0c099b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:48.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f11e70640 1 --2- 172.21.15.76:0/1608937600 >> v2:172.21.15.81:3300/0 conn(0x7f5f0c15e040 0x7f5f0c099b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f11e70640 1 -- 172.21.15.76:0/1608937600 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c0d0240 con 0x7f5f0c15dc60 2024-03-08T06:02:48.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f12671640 1 --2- 172.21.15.76:0/1608937600 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f0c06bfa0 0x7f5f0c09a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:48.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.521+0000 7f5f1166f640 1 --2- 172.21.15.76:0/1608937600 >> v2:172.21.15.81:3300/0 conn(0x7f5f0c15e040 0x7f5f0c099b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:48.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.522+0000 7f5f11e70640 1 --2- 172.21.15.76:0/1608937600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0c15dc60 0x7f5f0c0d1000 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c072820 tx=0x7f5f0802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816c4e43d55d7a4f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:48.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.522+0000 7f5f10e6e640 1 -- 172.21.15.76:0/1608937600 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08034070 con 0x7f5f0c15dc60 2024-03-08T06:02:48.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.522+0000 7f5f10e6e640 1 -- 172.21.15.76:0/1608937600 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f08030860 con 0x7f5f0c15dc60 2024-03-08T06:02:48.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.522+0000 7f5f10e6e640 1 -- 172.21.15.76:0/1608937600 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08030c30 con 0x7f5f0c15dc60 2024-03-08T06:02:48.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.522+0000 7f5f140d4640 1 -- 172.21.15.76:0/1608937600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0c15dc60 msgr2=0x7f5f0c0d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:48.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.522+0000 7f5f140d4640 1 --2- 172.21.15.76:0/1608937600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0c15dc60 0x7f5f0c0d1000 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c072820 tx=0x7f5f0802b9a0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.523+0000 7f5f140d4640 1 -- 172.21.15.76:0/1608937600 shutdown_connections 2024-03-08T06:02:48.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.523+0000 7f5f140d4640 1 --2- 172.21.15.76:0/1608937600 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f0c06bfa0 0x7f5f0c09a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.523+0000 7f5f140d4640 1 --2- 172.21.15.76:0/1608937600 >> v2:172.21.15.81:3300/0 conn(0x7f5f0c15e040 0x7f5f0c099b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.523+0000 7f5f140d4640 1 --2- 172.21.15.76:0/1608937600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0c15dc60 0x7f5f0c0d1000 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.523+0000 7f5f140d4640 1 -- 172.21.15.76:0/1608937600 >> 172.21.15.76:0/1608937600 conn(0x7f5f0c0b3440 msgr2=0x7f5f0c0b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:48.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.523+0000 7f5f140d4640 1 -- 172.21.15.76:0/1608937600 shutdown_connections 2024-03-08T06:02:48.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.523+0000 7f5f140d4640 1 -- 172.21.15.76:0/1608937600 wait complete. 2024-03-08T06:02:48.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.524+0000 7f5f140d4640 1 Processor -- start 2024-03-08T06:02:48.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.524+0000 7f5f140d4640 1 -- start start 2024-03-08T06:02:48.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.524+0000 7f5f140d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0c06bfa0 0x7f5f0c0b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:48.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.524+0000 7f5f140d4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f0c15dc60 0x7f5f0c0cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:48.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.524+0000 7f5f140d4640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f0c15e040 0x7f5f0c158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:48.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.524+0000 7f5f140d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c0d0ae0 con 0x7f5f0c06bfa0 2024-03-08T06:02:48.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.524+0000 7f5f140d4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c0d0800 con 0x7f5f0c15dc60 2024-03-08T06:02:48.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.524+0000 7f5f140d4640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c0d1540 con 0x7f5f0c15e040 2024-03-08T06:02:48.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f11e70640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0c06bfa0 0x7f5f0c0b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:48.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f11e70640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0c06bfa0 0x7f5f0c0b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48436/0 (socket says 172.21.15.76:48436) 2024-03-08T06:02:48.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f11e70640 1 -- 172.21.15.76:0/4279705961 learned_addr learned my addr 172.21.15.76:0/4279705961 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:48.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f12671640 1 --2- 172.21.15.76:0/4279705961 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f0c15e040 0x7f5f0c158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:48.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f1166f640 1 --2- 172.21.15.76:0/4279705961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f0c15dc60 0x7f5f0c0cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:48.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f11e70640 1 -- 172.21.15.76:0/4279705961 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f0c15e040 msgr2=0x7f5f0c158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:48.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f11e70640 1 --2- 172.21.15.76:0/4279705961 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f0c15e040 0x7f5f0c158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f11e70640 1 -- 172.21.15.76:0/4279705961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f0c15dc60 msgr2=0x7f5f0c0cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:48.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f11e70640 1 --2- 172.21.15.76:0/4279705961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f0c15dc60 0x7f5f0c0cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:48.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f11e70640 1 -- 172.21.15.76:0/4279705961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c0b5bb0 con 0x7f5f0c06bfa0 2024-03-08T06:02:48.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f12671640 1 --2- 172.21.15.76:0/4279705961 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f0c15e040 0x7f5f0c158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:48.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f1166f640 1 --2- 172.21.15.76:0/4279705961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f0c15dc60 0x7f5f0c0cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:48.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f11e70640 1 --2- 172.21.15.76:0/4279705961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0c06bfa0 0x7f5f0c0b5340 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5f0802b9e0 tx=0x7f5f08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:48.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f027fc640 1 -- 172.21.15.76:0/4279705961 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08048070 con 0x7f5f0c06bfa0 2024-03-08T06:02:48.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f027fc640 1 -- 172.21.15.76:0/4279705961 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f08039d90 con 0x7f5f0c06bfa0 2024-03-08T06:02:48.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f027fc640 1 -- 172.21.15.76:0/4279705961 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08034040 con 0x7f5f0c06bfa0 2024-03-08T06:02:48.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f140d4640 1 -- 172.21.15.76:0/4279705961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0c0b5e90 con 0x7f5f0c06bfa0 2024-03-08T06:02:48.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.525+0000 7f5f140d4640 1 -- 172.21.15.76:0/4279705961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0c0b6450 con 0x7f5f0c06bfa0 2024-03-08T06:02:48.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.526+0000 7f5f140d4640 1 -- 172.21.15.76:0/4279705961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0c0b77e0 con 0x7f5f0c06bfa0 2024-03-08T06:02:48.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.527+0000 7f5f027fc640 1 -- 172.21.15.76:0/4279705961 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f0804b070 con 0x7f5f0c06bfa0 2024-03-08T06:02:48.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.527+0000 7f5f027fc640 1 --2- 172.21.15.76:0/4279705961 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5ef006b4d0 0x7f5ef006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:48.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.527+0000 7f5f027fc640 1 -- 172.21.15.76:0/4279705961 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5f080ac210 con 0x7f5f0c06bfa0 2024-03-08T06:02:48.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.527+0000 7f5f1166f640 1 --2- 172.21.15.76:0/4279705961 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5ef006b4d0 0x7f5ef006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:48.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.528+0000 7f5f1166f640 1 --2- 172.21.15.76:0/4279705961 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5ef006b4d0 0x7f5ef006d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0caba0 tx=0x7f5efc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:48.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.531+0000 7f5f027fc640 1 -- 172.21.15.76:0/4279705961 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5f0807b290 con 0x7f5f0c06bfa0 2024-03-08T06:02:48.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:48.765+0000 7f5f140d4640 1 -- 172.21.15.76:0/4279705961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters"]} v 0) v1 -- 0x7f5f0c09a670 con 0x7f5f0c06bfa0 2024-03-08T06:02:49.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:49.093+0000 7f5f027fc640 1 -- 172.21.15.76:0/4279705961 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters"]}]=0 v131) v1 ==== 164+0+0 (secure 0 0 0) 0x7f5f0807ab90 con 0x7f5f0c06bfa0 2024-03-08T06:02:49.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:49.095+0000 7f5f140d4640 1 -- 172.21.15.76:0/4279705961 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5ef006b4d0 msgr2=0x7f5ef006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:49.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:49.095+0000 7f5f140d4640 1 --2- 172.21.15.76:0/4279705961 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5ef006b4d0 0x7f5ef006d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0caba0 tx=0x7f5efc008040 comp rx=0 tx=0).stop 2024-03-08T06:02:49.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:49.095+0000 7f5f140d4640 1 -- 172.21.15.76:0/4279705961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0c06bfa0 msgr2=0x7f5f0c0b5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:49.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:49.095+0000 7f5f140d4640 1 --2- 172.21.15.76:0/4279705961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0c06bfa0 0x7f5f0c0b5340 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5f0802b9e0 tx=0x7f5f08004730 comp rx=0 tx=0).stop 2024-03-08T06:02:49.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:49.095+0000 7f5f140d4640 1 -- 172.21.15.76:0/4279705961 shutdown_connections 2024-03-08T06:02:49.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:49.095+0000 7f5f140d4640 1 --2- 172.21.15.76:0/4279705961 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5ef006b4d0 0x7f5ef006d990 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:49.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:49.095+0000 7f5f140d4640 1 --2- 172.21.15.76:0/4279705961 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f0c15e040 0x7f5f0c158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:49.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:49.095+0000 7f5f140d4640 1 --2- 172.21.15.76:0/4279705961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f0c15dc60 0x7f5f0c0cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:49.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:49.095+0000 7f5f140d4640 1 --2- 172.21.15.76:0/4279705961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0c06bfa0 0x7f5f0c0b5340 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:49.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:49.095+0000 7f5f140d4640 1 -- 172.21.15.76:0/4279705961 >> 172.21.15.76:0/4279705961 conn(0x7f5f0c0b3440 msgr2=0x7f5f0c0b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:49.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:49.095+0000 7f5f140d4640 1 -- 172.21.15.76:0/4279705961 shutdown_connections 2024-03-08T06:02:49.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:49.095+0000 7f5f140d4640 1 -- 172.21.15.76:0/4279705961 wait complete. 2024-03-08T06:02:49.108 INFO:tasks.cephfs_test_runner:That the per-client labelled perf counters depict the clients ... ERROR 2024-03-08T06:02:49.108 INFO:tasks.cephfs_test_runner: 2024-03-08T06:02:49.108 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-08T06:02:49.108 INFO:tasks.cephfs_test_runner:ERROR: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-08T06:02:49.108 INFO:tasks.cephfs_test_runner:That the per-client labelled perf counters depict the clients 2024-03-08T06:02:49.108 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_joscollin_ceph_de7a7e179fbee9e314fa702f8f6c09d121c5e1a7/qa/tasks/cephfs/test_admin.py", line 57, in test_per_client_labeled_perf_counters 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner: while proceed(): 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner: raise MaxWhileTries(error_msg) 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner:teuthology.exceptions.MaxWhileTries: 'wait for counters - client.5541' reached maximum tries (31) after waiting for 30 seconds 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner: 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner:Ran 1 test in 86.730s 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner: 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner: 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner:ERROR: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner:That the per-client labelled perf counters depict the clients 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-08T06:02:49.109 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-08T06:02:49.110 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_joscollin_ceph_de7a7e179fbee9e314fa702f8f6c09d121c5e1a7/qa/tasks/cephfs/test_admin.py", line 57, in test_per_client_labeled_perf_counters 2024-03-08T06:02:49.110 INFO:tasks.cephfs_test_runner: while proceed(): 2024-03-08T06:02:49.110 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ 2024-03-08T06:02:49.110 INFO:tasks.cephfs_test_runner: raise MaxWhileTries(error_msg) 2024-03-08T06:02:49.110 INFO:tasks.cephfs_test_runner:teuthology.exceptions.MaxWhileTries: 'wait for counters - client.5541' reached maximum tries (31) after waiting for 30 seconds 2024-03-08T06:02:49.110 INFO:tasks.cephfs_test_runner: 2024-03-08T06:02:49.110 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_joscollin_ceph_de7a7e179fbee9e314fa702f8f6c09d121c5e1a7/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-08T06:02:49.331 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=08b5231ef43244be9c46188226d0f8e9 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_joscollin_ceph_de7a7e179fbee9e314fa702f8f6c09d121c5e1a7/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-08T06:02:49.332 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-08T06:02:49.341 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-08T06:02:49.353 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-08T06:02:49.353 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:02:49.353 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:02:49.373 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056373,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17773 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-08T06:02:49.373 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-08T06:02:49.373 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:02:49.429 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056373,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17773 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-08T06:02:49.429 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:02:49.429 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:02:49.449 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14569 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-08T06:02:49.449 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-08T06:02:49.449 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:02:49.507 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14569 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-08T06:02:49.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:49.507 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-03-08T06:02:49.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-08T06:02:49.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-08T06:02:49.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:49.523 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-08T06:02:49.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:49.590 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T06:02:49.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:49.658 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2024-03-08T06:02:49.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:49.760 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-03-08T06:02:49.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:49.774 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2024-03-08T06:02:49.845 INFO:teuthology.orchestra.run.smithi076.stderr:Cannot find device "ceph-brx" 2024-03-08T06:02:49.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:02:49.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:49.847 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-03-08T06:02:49.865 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-08T06:02:49.865 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-08T06:02:49.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:49.865 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-08T06:02:49.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:49.938 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-08T06:02:50.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:50.008 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link delete ceph-brx 2024-03-08T06:02:50.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:50.100 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-03-08T06:02:50.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:02:50.115 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link delete ceph-brx 2024-03-08T06:02:50.184 INFO:teuthology.orchestra.run.smithi081.stderr:Cannot find device "ceph-brx" 2024-03-08T06:02:50.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:02:50.186 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-08T06:02:50.196 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-08T06:02:50.196 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:02:50.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.297+0000 7fd58630d640 1 Processor -- start 2024-03-08T06:02:50.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.297+0000 7fd58630d640 1 -- start start 2024-03-08T06:02:50.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.298+0000 7fd58630d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd58015a9f0 0x7fd580071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.298+0000 7fd58630d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd5800c98e0 0x7fd5800c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.298+0000 7fd58630d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd5800ca2d0 0x7fd5800cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.298+0000 7fd58630d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5800ce790 con 0x7fd5800c98e0 2024-03-08T06:02:50.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.298+0000 7fd58630d640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5800722e0 con 0x7fd58015a9f0 2024-03-08T06:02:50.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.298+0000 7fd58630d640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5800ae8d0 con 0x7fd5800ca2d0 2024-03-08T06:02:50.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.298+0000 7fd57f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd5800c98e0 0x7fd5800c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.298+0000 7fd57f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd5800c98e0 0x7fd5800c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48438/0 (socket says 172.21.15.76:48438) 2024-03-08T06:02:50.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.298+0000 7fd57f7fe640 1 -- 172.21.15.76:0/100266977 learned_addr learned my addr 172.21.15.76:0/100266977 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:50.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.298+0000 7fd57ffff640 1 --2- 172.21.15.76:0/100266977 >> v2:172.21.15.81:3300/0 conn(0x7fd58015a9f0 0x7fd580071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.298+0000 7fd5848aa640 1 --2- 172.21.15.76:0/100266977 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd5800ca2d0 0x7fd5800cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.298+0000 7fd57f7fe640 1 -- 172.21.15.76:0/100266977 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd5800ca2d0 msgr2=0x7fd5800cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:50.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.298+0000 7fd57f7fe640 1 --2- 172.21.15.76:0/100266977 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd5800ca2d0 0x7fd5800cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.298+0000 7fd57f7fe640 1 -- 172.21.15.76:0/100266977 >> v2:172.21.15.81:3300/0 conn(0x7fd58015a9f0 msgr2=0x7fd580071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:50.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.298+0000 7fd57f7fe640 1 --2- 172.21.15.76:0/100266977 >> v2:172.21.15.81:3300/0 conn(0x7fd58015a9f0 0x7fd580071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.298+0000 7fd57f7fe640 1 -- 172.21.15.76:0/100266977 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5800aebb0 con 0x7fd5800c98e0 2024-03-08T06:02:50.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.299+0000 7fd57f7fe640 1 --2- 172.21.15.76:0/100266977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5800c98e0 0x7fd5800c9d90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd580072c20 tx=0x7fd56802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c0ef3ad83321bb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:50.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.299+0000 7fd57effd640 1 -- 172.21.15.76:0/100266977 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568035020 con 0x7fd5800c98e0 2024-03-08T06:02:50.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.299+0000 7fd57effd640 1 -- 172.21.15.76:0/100266977 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd568031870 con 0x7fd5800c98e0 2024-03-08T06:02:50.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.299+0000 7fd57effd640 1 -- 172.21.15.76:0/100266977 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568031c40 con 0x7fd5800c98e0 2024-03-08T06:02:50.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.300+0000 7fd58630d640 1 -- 172.21.15.76:0/100266977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5800c98e0 msgr2=0x7fd5800c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:50.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.300+0000 7fd58630d640 1 --2- 172.21.15.76:0/100266977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5800c98e0 0x7fd5800c9d90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd580072c20 tx=0x7fd56802b910 comp rx=0 tx=0).stop 2024-03-08T06:02:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.300+0000 7fd58630d640 1 -- 172.21.15.76:0/100266977 shutdown_connections 2024-03-08T06:02:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.300+0000 7fd58630d640 1 --2- 172.21.15.76:0/100266977 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd5800ca2d0 0x7fd5800cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.300+0000 7fd58630d640 1 --2- 172.21.15.76:0/100266977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5800c98e0 0x7fd5800c9d90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.300+0000 7fd58630d640 1 --2- 172.21.15.76:0/100266977 >> v2:172.21.15.81:3300/0 conn(0x7fd58015a9f0 0x7fd580071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.300+0000 7fd58630d640 1 -- 172.21.15.76:0/100266977 >> 172.21.15.76:0/100266977 conn(0x7fd5800cf740 msgr2=0x7fd5800cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.300+0000 7fd58630d640 1 -- 172.21.15.76:0/100266977 shutdown_connections 2024-03-08T06:02:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.300+0000 7fd58630d640 1 -- 172.21.15.76:0/100266977 wait complete. 2024-03-08T06:02:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.301+0000 7fd58630d640 1 Processor -- start 2024-03-08T06:02:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.301+0000 7fd58630d640 1 -- start start 2024-03-08T06:02:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd58630d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5800c98e0 0x7fd58015b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd58630d640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd5800ca2d0 0x7fd5800b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd58630d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd58015a9f0 0x7fd5800b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd58630d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58015af50 con 0x7fd5800c98e0 2024-03-08T06:02:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd58630d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58015add0 con 0x7fd58015a9f0 2024-03-08T06:02:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd58630d640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5800cf160 con 0x7fd5800ca2d0 2024-03-08T06:02:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd57ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5800c98e0 0x7fd58015b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd57ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5800c98e0 0x7fd58015b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48450/0 (socket says 172.21.15.76:48450) 2024-03-08T06:02:50.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd57ffff640 1 -- 172.21.15.76:0/3567311976 learned_addr learned my addr 172.21.15.76:0/3567311976 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:50.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd5848aa640 1 --2- 172.21.15.76:0/3567311976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd58015a9f0 0x7fd5800b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd57f7fe640 1 --2- 172.21.15.76:0/3567311976 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd5800ca2d0 0x7fd5800b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd57ffff640 1 -- 172.21.15.76:0/3567311976 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd5800ca2d0 msgr2=0x7fd5800b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:50.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd57ffff640 1 --2- 172.21.15.76:0/3567311976 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd5800ca2d0 0x7fd5800b2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd57ffff640 1 -- 172.21.15.76:0/3567311976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd58015a9f0 msgr2=0x7fd5800b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:50.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd57ffff640 1 --2- 172.21.15.76:0/3567311976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd58015a9f0 0x7fd5800b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd57ffff640 1 -- 172.21.15.76:0/3567311976 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58015b760 con 0x7fd5800c98e0 2024-03-08T06:02:50.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd5848aa640 1 --2- 172.21.15.76:0/3567311976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd58015a9f0 0x7fd5800b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:50.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.302+0000 7fd57ffff640 1 --2- 172.21.15.76:0/3567311976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5800c98e0 0x7fd58015b0d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd57000d620 tx=0x7fd57000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:50.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.303+0000 7fd57cff9640 1 -- 172.21.15.76:0/3567311976 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000b620 con 0x7fd5800c98e0 2024-03-08T06:02:50.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.303+0000 7fd57cff9640 1 -- 172.21.15.76:0/3567311976 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd570019070 con 0x7fd5800c98e0 2024-03-08T06:02:50.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.303+0000 7fd58630d640 1 -- 172.21.15.76:0/3567311976 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58015ba40 con 0x7fd5800c98e0 2024-03-08T06:02:50.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.303+0000 7fd57cff9640 1 -- 172.21.15.76:0/3567311976 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000ba00 con 0x7fd5800c98e0 2024-03-08T06:02:50.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.303+0000 7fd58630d640 1 -- 172.21.15.76:0/3567311976 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58015c000 con 0x7fd5800c98e0 2024-03-08T06:02:50.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.304+0000 7fd58630d640 1 -- 172.21.15.76:0/3567311976 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58004f030 con 0x7fd5800c98e0 2024-03-08T06:02:50.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.304+0000 7fd57cff9640 1 -- 172.21.15.76:0/3567311976 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd57000bba0 con 0x7fd5800c98e0 2024-03-08T06:02:50.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.304+0000 7fd57cff9640 1 --2- 172.21.15.76:0/3567311976 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd56006b4d0 0x7fd56006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.305+0000 7fd57cff9640 1 -- 172.21.15.76:0/3567311976 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd57008a470 con 0x7fd5800c98e0 2024-03-08T06:02:50.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.305+0000 7fd57f7fe640 1 --2- 172.21.15.76:0/3567311976 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd56006b4d0 0x7fd56006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.305+0000 7fd57f7fe640 1 --2- 172.21.15.76:0/3567311976 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd56006b4d0 0x7fd56006d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd568036000 tx=0x7fd568008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:50.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.309+0000 7fd57cff9640 1 -- 172.21.15.76:0/3567311976 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5700594f0 con 0x7fd5800c98e0 2024-03-08T06:02:50.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.475+0000 7fd58630d640 1 -- 172.21.15.76:0/3567311976 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd5800b09d0 con 0x7fd56006b4d0 2024-03-08T06:02:50.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.476+0000 7fd57cff9640 1 -- 172.21.15.76:0/3567311976 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13517 (secure 0 0 0) 0x7fd5800b09d0 con 0x7fd56006b4d0 2024-03-08T06:02:50.477 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:02:50.477 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-08T06:02:50.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.478+0000 7fd58630d640 1 -- 172.21.15.76:0/3567311976 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd56006b4d0 msgr2=0x7fd56006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:50.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.478+0000 7fd58630d640 1 --2- 172.21.15.76:0/3567311976 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd56006b4d0 0x7fd56006d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd568036000 tx=0x7fd568008030 comp rx=0 tx=0).stop 2024-03-08T06:02:50.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.478+0000 7fd58630d640 1 -- 172.21.15.76:0/3567311976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5800c98e0 msgr2=0x7fd58015b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:50.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.478+0000 7fd58630d640 1 --2- 172.21.15.76:0/3567311976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5800c98e0 0x7fd58015b0d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd57000d620 tx=0x7fd57000dae0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.478+0000 7fd58630d640 1 -- 172.21.15.76:0/3567311976 shutdown_connections 2024-03-08T06:02:50.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.478+0000 7fd58630d640 1 --2- 172.21.15.76:0/3567311976 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd56006b4d0 0x7fd56006d990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.478+0000 7fd58630d640 1 --2- 172.21.15.76:0/3567311976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd58015a9f0 0x7fd5800b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.478+0000 7fd58630d640 1 --2- 172.21.15.76:0/3567311976 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd5800ca2d0 0x7fd5800b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.478+0000 7fd58630d640 1 --2- 172.21.15.76:0/3567311976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5800c98e0 0x7fd58015b0d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.478+0000 7fd58630d640 1 -- 172.21.15.76:0/3567311976 >> 172.21.15.76:0/3567311976 conn(0x7fd5800cf740 msgr2=0x7fd5800b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:50.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.478+0000 7fd58630d640 1 -- 172.21.15.76:0/3567311976 shutdown_connections 2024-03-08T06:02:50.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.478+0000 7fd58630d640 1 -- 172.21.15.76:0/3567311976 wait complete. 2024-03-08T06:02:50.489 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-03-08T06:02:49.950304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":256360,"kb_used_data":11072,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754718360,"statfs":{"total":773094113280,"available":772831600640,"internally_reserved":0,"allocated":11337728,"data_stored":3019607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4198629,"num_objects":-1022,"num_object_clones":0,"num_object_copies":-2044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-2058,"num_write_kb":-6092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-14,"num_bytes_recovered":-42,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001347"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":214,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-03-08T06:02:47.482867+0000","last_change":"2024-03-08T06:00:26.178419+0000","last_active":"2024-03-08T06:02:47.482867+0000","last_peered":"2024-03-08T06:02:47.482867+0000","last_clean":"2024-03-08T06:02:47.482867+0000","last_became_active":"2024-03-08T06:00:26.175826+0000","last_became_peered":"2024-03-08T06:00:26.175826+0000","last_unstale":"2024-03-08T06:02:47.482867+0000","last_undegraded":"2024-03-08T06:02:47.482867+0000","last_fullsized":"2024-03-08T06:02:47.482867+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_clean_scrub_stamp":"2024-03-08T06:00:24.029492+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:36:00.682517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37496,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334344,"statfs":{"total":96636764160,"available":96598368256,"internally_reserved":0,"allocated":2207744,"data_stored":818971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28412,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343428,"statfs":{"total":96636764160,"available":96607670272,"internally_reserved":0,"allocated":1425408,"data_stored":228477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":228831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28428,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343412,"statfs":{"total":96636764160,"available":96607653888,"internally_reserved":0,"allocated":1441792,"data_stored":228494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35864,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335976,"statfs":{"total":96636764160,"available":96600039424,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31704,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28352,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343488,"statfs":{"total":96636764160,"available":96607731712,"internally_reserved":0,"allocated":1363968,"data_stored":228436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1912832,"data_stored":818776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:02:50.490 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:02:50.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.590+0000 7f52a1bfd640 1 Processor -- start 2024-03-08T06:02:50.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.590+0000 7f52a1bfd640 1 -- start start 2024-03-08T06:02:50.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.591+0000 7f52a1bfd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f529c071390 0x7f529c0cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.591+0000 7f52a1bfd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f529c0cd310 0x7f529c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.591+0000 7f52a1bfd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529c06d880 con 0x7f529c06d3f0 2024-03-08T06:02:50.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.591+0000 7f52a1bfd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529c06e010 con 0x7f529c071390 2024-03-08T06:02:50.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.591+0000 7f52a1bfd640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529c06e2f0 con 0x7f529c0cd310 2024-03-08T06:02:50.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.591+0000 7f529b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f529c071390 0x7f529c0cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.591+0000 7f529affd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f529c06d3f0 0x7f529c071770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57950/0 (socket says 172.21.15.76:57950) 2024-03-08T06:02:50.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.591+0000 7f529affd640 1 -- 172.21.15.76:0/2409421305 learned_addr learned my addr 172.21.15.76:0/2409421305 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:50.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.591+0000 7f529bfff640 1 --2- 172.21.15.76:0/2409421305 >> v2:172.21.15.81:3300/0 conn(0x7f529c0cd310 0x7f529c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.591+0000 7f529b7fe640 1 -- 172.21.15.76:0/2409421305 >> v2:172.21.15.81:3300/0 conn(0x7f529c0cd310 msgr2=0x7f529c0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:50.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.591+0000 7f529b7fe640 1 --2- 172.21.15.76:0/2409421305 >> v2:172.21.15.81:3300/0 conn(0x7f529c0cd310 0x7f529c0cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.591+0000 7f529b7fe640 1 -- 172.21.15.76:0/2409421305 >> v1:172.21.15.76:6789/0 conn(0x7f529c06d3f0 legacy=0x7f529c071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:50.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.591+0000 7f529b7fe640 1 -- 172.21.15.76:0/2409421305 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c0ce7b0 con 0x7f529c071390 2024-03-08T06:02:50.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.591+0000 7f529a7fc640 1 -- 172.21.15.76:0/2409421305 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432321409 0 0) 0x7f529c06d880 con 0x7f529c06d3f0 2024-03-08T06:02:50.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.592+0000 7f529b7fe640 1 --2- 172.21.15.76:0/2409421305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f529c071390 0x7f529c0cf0e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f529c0b27a0 tx=0x7f528c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce068fd4e99c9b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:50.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.592+0000 7f529a7fc640 1 -- 172.21.15.76:0/2409421305 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c035020 con 0x7f529c071390 2024-03-08T06:02:50.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.592+0000 7f529a7fc640 1 -- 172.21.15.76:0/2409421305 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c031830 con 0x7f529c071390 2024-03-08T06:02:50.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.592+0000 7f529a7fc640 1 -- 172.21.15.76:0/2409421305 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c031c00 con 0x7f529c071390 2024-03-08T06:02:50.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.593+0000 7f52a1bfd640 1 -- 172.21.15.76:0/2409421305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f529c071390 msgr2=0x7f529c0cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:50.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.593+0000 7f52a1bfd640 1 --2- 172.21.15.76:0/2409421305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f529c071390 0x7f529c0cf0e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f529c0b27a0 tx=0x7f528c02b8d0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.593+0000 7f52a1bfd640 1 -- 172.21.15.76:0/2409421305 shutdown_connections 2024-03-08T06:02:50.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.593+0000 7f52a1bfd640 1 --2- 172.21.15.76:0/2409421305 >> v2:172.21.15.81:3300/0 conn(0x7f529c0cd310 0x7f529c0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.593+0000 7f52a1bfd640 1 --2- 172.21.15.76:0/2409421305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f529c071390 0x7f529c0cf0e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.593+0000 7f52a1bfd640 1 -- 172.21.15.76:0/2409421305 >> 172.21.15.76:0/2409421305 conn(0x7f529c0cf740 msgr2=0x7f529c0cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:50.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.593+0000 7f52a1bfd640 1 -- 172.21.15.76:0/2409421305 shutdown_connections 2024-03-08T06:02:50.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.593+0000 7f52a1bfd640 1 -- 172.21.15.76:0/2409421305 wait complete. 2024-03-08T06:02:50.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.594+0000 7f52a1bfd640 1 Processor -- start 2024-03-08T06:02:50.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.594+0000 7f52a1bfd640 1 -- start start 2024-03-08T06:02:50.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f52a1bfd640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f529c071390 0x7f529c0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f52a1bfd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f529c0cd310 0x7f529c15ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f52a1bfd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529c15d2e0 0x7f529c14d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f52a1bfd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c15c060 con 0x7f529c0cd310 2024-03-08T06:02:50.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f52a1bfd640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c15bee0 con 0x7f529c15d2e0 2024-03-08T06:02:50.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f52a1bfd640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c0cdcc0 con 0x7f529c071390 2024-03-08T06:02:50.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f529affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f529c0cd310 0x7f529c15ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f529affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f529c0cd310 0x7f529c15ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48474/0 (socket says 172.21.15.76:48474) 2024-03-08T06:02:50.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f529affd640 1 -- 172.21.15.76:0/4057945044 learned_addr learned my addr 172.21.15.76:0/4057945044 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:50.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f529bfff640 1 --2- 172.21.15.76:0/4057945044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529c15d2e0 0x7f529c14d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f529b7fe640 1 --2- 172.21.15.76:0/4057945044 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f529c071390 0x7f529c0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f529affd640 1 -- 172.21.15.76:0/4057945044 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f529c071390 msgr2=0x7f529c0cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f529affd640 1 --2- 172.21.15.76:0/4057945044 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f529c071390 0x7f529c0cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f529affd640 1 -- 172.21.15.76:0/4057945044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529c15d2e0 msgr2=0x7f529c14d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f529affd640 1 --2- 172.21.15.76:0/4057945044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529c15d2e0 0x7f529c14d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f529affd640 1 -- 172.21.15.76:0/4057945044 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c15c520 con 0x7f529c0cd310 2024-03-08T06:02:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.595+0000 7f529bfff640 1 --2- 172.21.15.76:0/4057945044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529c15d2e0 0x7f529c14d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.596+0000 7f529affd640 1 --2- 172.21.15.76:0/4057945044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f529c0cd310 0x7f529c15ec40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f529000c4d0 tx=0x7f529000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.596+0000 7f5283fff640 1 -- 172.21.15.76:0/4057945044 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290018070 con 0x7f529c0cd310 2024-03-08T06:02:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.596+0000 7f5283fff640 1 -- 172.21.15.76:0/4057945044 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5290007500 con 0x7f529c0cd310 2024-03-08T06:02:50.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.596+0000 7f52a1bfd640 1 -- 172.21.15.76:0/4057945044 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c15c800 con 0x7f529c0cd310 2024-03-08T06:02:50.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.596+0000 7f5283fff640 1 -- 172.21.15.76:0/4057945044 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290004a20 con 0x7f529c0cd310 2024-03-08T06:02:50.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.596+0000 7f52a1bfd640 1 -- 172.21.15.76:0/4057945044 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c15cdc0 con 0x7f529c0cd310 2024-03-08T06:02:50.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.597+0000 7f52a1bfd640 1 -- 172.21.15.76:0/4057945044 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529c04f030 con 0x7f529c0cd310 2024-03-08T06:02:50.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.597+0000 7f5283fff640 1 -- 172.21.15.76:0/4057945044 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f529001b070 con 0x7f529c0cd310 2024-03-08T06:02:50.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.597+0000 7f5283fff640 1 --2- 172.21.15.76:0/4057945044 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f528806b4d0 0x7f528806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.598+0000 7f5283fff640 1 -- 172.21.15.76:0/4057945044 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5290089350 con 0x7f529c0cd310 2024-03-08T06:02:50.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.598+0000 7f529b7fe640 1 --2- 172.21.15.76:0/4057945044 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f528806b4d0 0x7f528806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.598+0000 7f529b7fe640 1 --2- 172.21.15.76:0/4057945044 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f528806b4d0 0x7f528806d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f528c036000 tx=0x7f528c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:50.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.602+0000 7f5283fff640 1 -- 172.21.15.76:0/4057945044 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f52900583d0 con 0x7f529c0cd310 2024-03-08T06:02:50.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.769+0000 7f52a1bfd640 1 -- 172.21.15.76:0/4057945044 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f529c0cbfa0 con 0x7f528806b4d0 2024-03-08T06:02:50.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.770+0000 7f5283fff640 1 -- 172.21.15.76:0/4057945044 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13517 (secure 0 0 0) 0x7f529c0cbfa0 con 0x7f528806b4d0 2024-03-08T06:02:50.771 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:02:50.771 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-08T06:02:50.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.772+0000 7f52a1bfd640 1 -- 172.21.15.76:0/4057945044 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f528806b4d0 msgr2=0x7f528806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:50.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.772+0000 7f52a1bfd640 1 --2- 172.21.15.76:0/4057945044 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f528806b4d0 0x7f528806d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f528c036000 tx=0x7f528c008030 comp rx=0 tx=0).stop 2024-03-08T06:02:50.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.772+0000 7f52a1bfd640 1 -- 172.21.15.76:0/4057945044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f529c0cd310 msgr2=0x7f529c15ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:50.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.772+0000 7f52a1bfd640 1 --2- 172.21.15.76:0/4057945044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f529c0cd310 0x7f529c15ec40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f529000c4d0 tx=0x7f529000c990 comp rx=0 tx=0).stop 2024-03-08T06:02:50.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.772+0000 7f52a1bfd640 1 -- 172.21.15.76:0/4057945044 shutdown_connections 2024-03-08T06:02:50.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.772+0000 7f52a1bfd640 1 --2- 172.21.15.76:0/4057945044 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f528806b4d0 0x7f528806d990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.772+0000 7f52a1bfd640 1 --2- 172.21.15.76:0/4057945044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529c15d2e0 0x7f529c14d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.772+0000 7f52a1bfd640 1 --2- 172.21.15.76:0/4057945044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f529c0cd310 0x7f529c15ec40 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.772+0000 7f52a1bfd640 1 --2- 172.21.15.76:0/4057945044 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f529c071390 0x7f529c0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.772+0000 7f52a1bfd640 1 -- 172.21.15.76:0/4057945044 >> 172.21.15.76:0/4057945044 conn(0x7f529c0cf740 msgr2=0x7f529c0b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:50.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.772+0000 7f52a1bfd640 1 -- 172.21.15.76:0/4057945044 shutdown_connections 2024-03-08T06:02:50.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.772+0000 7f52a1bfd640 1 -- 172.21.15.76:0/4057945044 wait complete. 2024-03-08T06:02:50.783 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-03-08T06:02:49.950304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":256360,"kb_used_data":11072,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754718360,"statfs":{"total":773094113280,"available":772831600640,"internally_reserved":0,"allocated":11337728,"data_stored":3019607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4198629,"num_objects":-1022,"num_object_clones":0,"num_object_copies":-2044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-2058,"num_write_kb":-6092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-14,"num_bytes_recovered":-42,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001347"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":214,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-03-08T06:02:47.482867+0000","last_change":"2024-03-08T06:00:26.178419+0000","last_active":"2024-03-08T06:02:47.482867+0000","last_peered":"2024-03-08T06:02:47.482867+0000","last_clean":"2024-03-08T06:02:47.482867+0000","last_became_active":"2024-03-08T06:00:26.175826+0000","last_became_peered":"2024-03-08T06:00:26.175826+0000","last_unstale":"2024-03-08T06:02:47.482867+0000","last_undegraded":"2024-03-08T06:02:47.482867+0000","last_fullsized":"2024-03-08T06:02:47.482867+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_clean_scrub_stamp":"2024-03-08T06:00:24.029492+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:36:00.682517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37496,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334344,"statfs":{"total":96636764160,"available":96598368256,"internally_reserved":0,"allocated":2207744,"data_stored":818971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28412,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343428,"statfs":{"total":96636764160,"available":96607670272,"internally_reserved":0,"allocated":1425408,"data_stored":228477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":228831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28428,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343412,"statfs":{"total":96636764160,"available":96607653888,"internally_reserved":0,"allocated":1441792,"data_stored":228494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35864,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335976,"statfs":{"total":96636764160,"available":96600039424,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31704,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28352,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343488,"statfs":{"total":96636764160,"available":96607731712,"internally_reserved":0,"allocated":1363968,"data_stored":228436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1912832,"data_stored":818776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:02:50.784 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-08T06:02:50.784 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:02:50.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.884+0000 7f5564e15640 1 Processor -- start 2024-03-08T06:02:50.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.884+0000 7f5564e15640 1 -- start start 2024-03-08T06:02:50.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.885+0000 7f5564e15640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f55600d0b90 0x7f5560168bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.885+0000 7f5564e15640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55600cf180 con 0x7f55600b6d00 2024-03-08T06:02:50.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.885+0000 7f5564e15640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55600d1ed0 con 0x7f556015dde0 2024-03-08T06:02:50.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.885+0000 7f5564e15640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55600d21b0 con 0x7f55600d0b90 2024-03-08T06:02:50.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.885+0000 7f555ed9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f55600d0b90 0x7f5560168bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.885+0000 7f5555d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f55600b6d00 0x7f55600d0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57958/0 (socket says 172.21.15.76:57958) 2024-03-08T06:02:50.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.885+0000 7f5555d9b640 1 -- 172.21.15.76:0/2962173706 learned_addr learned my addr 172.21.15.76:0/2962173706 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:50.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.885+0000 7f555ed9d640 1 --2- 172.21.15.76:0/2962173706 >> v2:172.21.15.76:3300/0 conn(0x7f55600d0b90 0x7f5560168bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48478/0 (socket says 172.21.15.76:48478) 2024-03-08T06:02:50.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.885+0000 7f555ed9d640 1 -- 172.21.15.76:0/2962173706 >> v1:172.21.15.81:6789/0 conn(0x7f556015dde0 legacy=0x7f5560162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:50.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.885+0000 7f555ed9d640 1 -- 172.21.15.76:0/2962173706 >> v1:172.21.15.76:6789/0 conn(0x7f55600b6d00 legacy=0x7f55600d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:50.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.885+0000 7f555ed9d640 1 -- 172.21.15.76:0/2962173706 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55600cec10 con 0x7f55600d0b90 2024-03-08T06:02:50.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.886+0000 7f555ed9d640 1 --2- 172.21.15.76:0/2962173706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55600d0b90 0x7f5560168bd0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5558009d90 tx=0x7f555802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcbf1bf85ab9d44 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:50.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.886+0000 7f555dd9b640 1 -- 172.21.15.76:0/2962173706 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5558039070 con 0x7f55600d0b90 2024-03-08T06:02:50.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.886+0000 7f555dd9b640 1 -- 172.21.15.76:0/2962173706 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555802cbd0 con 0x7f55600d0b90 2024-03-08T06:02:50.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.886+0000 7f555dd9b640 1 -- 172.21.15.76:0/2962173706 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5558034440 con 0x7f55600d0b90 2024-03-08T06:02:50.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.886+0000 7f5564e15640 1 -- 172.21.15.76:0/2962173706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55600d0b90 msgr2=0x7f5560168bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:50.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.886+0000 7f5564e15640 1 --2- 172.21.15.76:0/2962173706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55600d0b90 0x7f5560168bd0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5558009d90 tx=0x7f555802be20 comp rx=0 tx=0).stop 2024-03-08T06:02:50.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.887+0000 7f5564e15640 1 -- 172.21.15.76:0/2962173706 shutdown_connections 2024-03-08T06:02:50.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.887+0000 7f5564e15640 1 --2- 172.21.15.76:0/2962173706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55600d0b90 0x7f5560168bd0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.887+0000 7f5564e15640 1 -- 172.21.15.76:0/2962173706 >> 172.21.15.76:0/2962173706 conn(0x7f55600b56e0 msgr2=0x7f55600b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:50.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.887+0000 7f5564e15640 1 -- 172.21.15.76:0/2962173706 shutdown_connections 2024-03-08T06:02:50.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.887+0000 7f5564e15640 1 -- 172.21.15.76:0/2962173706 wait complete. 2024-03-08T06:02:50.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.888+0000 7f5564e15640 1 Processor -- start 2024-03-08T06:02:50.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.888+0000 7f5564e15640 1 -- start start 2024-03-08T06:02:50.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f5564e15640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f55600b6d00 0x7f55601590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f5564e15640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55600d0b90 0x7f556014ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f5564e15640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f556015dde0 0x7f556014f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f5564e15640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55601694a0 con 0x7f556015dde0 2024-03-08T06:02:50.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f5564e15640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5560169320 con 0x7f55600d0b90 2024-03-08T06:02:50.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f5564e15640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5560169620 con 0x7f55600b6d00 2024-03-08T06:02:50.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f555ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f556015dde0 0x7f556014f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f555ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f556015dde0 0x7f556014f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48480/0 (socket says 172.21.15.76:48480) 2024-03-08T06:02:50.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f555ed9d640 1 -- 172.21.15.76:0/2165898297 learned_addr learned my addr 172.21.15.76:0/2165898297 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:50.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f555e59c640 1 --2- 172.21.15.76:0/2165898297 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f55600b6d00 0x7f55601590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f5555d9b640 1 --2- 172.21.15.76:0/2165898297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55600d0b90 0x7f556014ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f555ed9d640 1 -- 172.21.15.76:0/2165898297 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f55600b6d00 msgr2=0x7f55601590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:50.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f555ed9d640 1 --2- 172.21.15.76:0/2165898297 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f55600b6d00 0x7f55601590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f555ed9d640 1 -- 172.21.15.76:0/2165898297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55600d0b90 msgr2=0x7f556014ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:50.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f555ed9d640 1 --2- 172.21.15.76:0/2165898297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55600d0b90 0x7f556014ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:50.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f555ed9d640 1 -- 172.21.15.76:0/2165898297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55601f7e80 con 0x7f556015dde0 2024-03-08T06:02:50.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.889+0000 7f555ed9d640 1 --2- 172.21.15.76:0/2165898297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f556015dde0 0x7f556014f280 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5558037040 tx=0x7f5558004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:50.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.890+0000 7f55577fe640 1 -- 172.21.15.76:0/2165898297 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5558039050 con 0x7f556015dde0 2024-03-08T06:02:50.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.890+0000 7f55577fe640 1 -- 172.21.15.76:0/2165898297 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555803a040 con 0x7f556015dde0 2024-03-08T06:02:50.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.890+0000 7f5564e15640 1 -- 172.21.15.76:0/2165898297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55600caef0 con 0x7f556015dde0 2024-03-08T06:02:50.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.890+0000 7f55577fe640 1 -- 172.21.15.76:0/2165898297 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555803e610 con 0x7f556015dde0 2024-03-08T06:02:50.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.890+0000 7f5564e15640 1 -- 172.21.15.76:0/2165898297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5560159600 con 0x7f556015dde0 2024-03-08T06:02:50.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.891+0000 7f5564e15640 1 -- 172.21.15.76:0/2165898297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556004f030 con 0x7f556015dde0 2024-03-08T06:02:50.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.891+0000 7f55577fe640 1 -- 172.21.15.76:0/2165898297 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5558047020 con 0x7f556015dde0 2024-03-08T06:02:50.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.891+0000 7f55577fe640 1 --2- 172.21.15.76:0/2165898297 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f554006b4d0 0x7f554006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:50.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.891+0000 7f55577fe640 1 -- 172.21.15.76:0/2165898297 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f55580ac240 con 0x7f556015dde0 2024-03-08T06:02:50.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.891+0000 7f555e59c640 1 --2- 172.21.15.76:0/2165898297 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f554006b4d0 0x7f554006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:50.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.892+0000 7f555e59c640 1 --2- 172.21.15.76:0/2165898297 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f554006b4d0 0x7f554006d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f55601f70b0 tx=0x7f5550009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:50.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:50.896+0000 7f55577fe640 1 -- 172.21.15.76:0/2165898297 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f555804e020 con 0x7f556015dde0 2024-03-08T06:02:51.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.063+0000 7f5564e15640 1 -- 172.21.15.76:0/2165898297 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55600d1870 con 0x7f554006b4d0 2024-03-08T06:02:51.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.064+0000 7f55577fe640 1 -- 172.21.15.76:0/2165898297 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13517 (secure 0 0 0) 0x7f55600d1870 con 0x7f554006b4d0 2024-03-08T06:02:51.065 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:02:51.065 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-08T06:02:51.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.066+0000 7f5564e15640 1 -- 172.21.15.76:0/2165898297 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f554006b4d0 msgr2=0x7f554006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:51.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.066+0000 7f5564e15640 1 --2- 172.21.15.76:0/2165898297 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f554006b4d0 0x7f554006d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f55601f70b0 tx=0x7f5550009060 comp rx=0 tx=0).stop 2024-03-08T06:02:51.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.066+0000 7f5564e15640 1 -- 172.21.15.76:0/2165898297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f556015dde0 msgr2=0x7f556014f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:51.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.066+0000 7f5564e15640 1 --2- 172.21.15.76:0/2165898297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f556015dde0 0x7f556014f280 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5558037040 tx=0x7f5558004a40 comp rx=0 tx=0).stop 2024-03-08T06:02:51.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.066+0000 7f5564e15640 1 -- 172.21.15.76:0/2165898297 shutdown_connections 2024-03-08T06:02:51.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.066+0000 7f5564e15640 1 --2- 172.21.15.76:0/2165898297 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f554006b4d0 0x7f554006d990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:51.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.066+0000 7f5564e15640 1 --2- 172.21.15.76:0/2165898297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f556015dde0 0x7f556014f280 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:51.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.066+0000 7f5564e15640 1 --2- 172.21.15.76:0/2165898297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55600d0b90 0x7f556014ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:51.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.066+0000 7f5564e15640 1 --2- 172.21.15.76:0/2165898297 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f55600b6d00 0x7f55601590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:51.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.066+0000 7f5564e15640 1 -- 172.21.15.76:0/2165898297 >> 172.21.15.76:0/2165898297 conn(0x7f55600b56e0 msgr2=0x7f556015d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:51.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.066+0000 7f5564e15640 1 -- 172.21.15.76:0/2165898297 shutdown_connections 2024-03-08T06:02:51.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.066+0000 7f5564e15640 1 -- 172.21.15.76:0/2165898297 wait complete. 2024-03-08T06:02:51.077 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-03-08T06:02:49.950304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":256360,"kb_used_data":11072,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754718360,"statfs":{"total":773094113280,"available":772831600640,"internally_reserved":0,"allocated":11337728,"data_stored":3019607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4198629,"num_objects":-1022,"num_object_clones":0,"num_object_copies":-2044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-2058,"num_write_kb":-6092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-14,"num_bytes_recovered":-42,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001347"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":214,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-03-08T06:02:47.482867+0000","last_change":"2024-03-08T06:00:26.178419+0000","last_active":"2024-03-08T06:02:47.482867+0000","last_peered":"2024-03-08T06:02:47.482867+0000","last_clean":"2024-03-08T06:02:47.482867+0000","last_became_active":"2024-03-08T06:00:26.175826+0000","last_became_peered":"2024-03-08T06:00:26.175826+0000","last_unstale":"2024-03-08T06:02:47.482867+0000","last_undegraded":"2024-03-08T06:02:47.482867+0000","last_fullsized":"2024-03-08T06:02:47.482867+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_clean_scrub_stamp":"2024-03-08T06:00:24.029492+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:36:00.682517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37496,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334344,"statfs":{"total":96636764160,"available":96598368256,"internally_reserved":0,"allocated":2207744,"data_stored":818971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28412,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343428,"statfs":{"total":96636764160,"available":96607670272,"internally_reserved":0,"allocated":1425408,"data_stored":228477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":228831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28428,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343412,"statfs":{"total":96636764160,"available":96607653888,"internally_reserved":0,"allocated":1441792,"data_stored":228494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35864,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335976,"statfs":{"total":96636764160,"available":96600039424,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31704,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28352,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343488,"statfs":{"total":96636764160,"available":96607731712,"internally_reserved":0,"allocated":1363968,"data_stored":228436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1912832,"data_stored":818776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:02:51.078 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:02:51.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.177+0000 7f995e823640 1 Processor -- start 2024-03-08T06:02:51.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.177+0000 7f995e823640 1 -- start start 2024-03-08T06:02:51.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.177+0000 7f995e823640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f995806d3f0 0x7f99580cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:51.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.177+0000 7f995e823640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995806d880 con 0x7f99580cd310 2024-03-08T06:02:51.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.177+0000 7f995e823640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995806e010 con 0x7f9958071390 2024-03-08T06:02:51.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.177+0000 7f995e823640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f995806e2f0 con 0x7f995806d3f0 2024-03-08T06:02:51.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.178+0000 7f99577fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f995806d3f0 0x7f99580cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:51.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.178+0000 7f995cdc0640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f99580cd310 0x7f99580cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57968/0 (socket says 172.21.15.76:57968) 2024-03-08T06:02:51.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.178+0000 7f995cdc0640 1 -- 172.21.15.76:0/1185540885 learned_addr learned my addr 172.21.15.76:0/1185540885 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:51.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.178+0000 7f99577fe640 1 -- 172.21.15.76:0/1185540885 >> v1:172.21.15.81:6789/0 conn(0x7f9958071390 legacy=0x7f9958071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:51.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.178+0000 7f9956ffd640 1 -- 172.21.15.76:0/1185540885 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537345702 0 0) 0x7f995806d880 con 0x7f99580cd310 2024-03-08T06:02:51.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.178+0000 7f99577fe640 1 -- 172.21.15.76:0/1185540885 >> v1:172.21.15.76:6789/0 conn(0x7f99580cd310 legacy=0x7f99580cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:51.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.178+0000 7f99577fe640 1 -- 172.21.15.76:0/1185540885 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99580ce7b0 con 0x7f995806d3f0 2024-03-08T06:02:51.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.179+0000 7f99577fe640 1 --2- 172.21.15.76:0/1185540885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f995806d3f0 0x7f99580cf0e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f994c0097a0 tx=0x7f994c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a1e183cb322676 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:51.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.179+0000 7f9956ffd640 1 -- 172.21.15.76:0/1185540885 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c034070 con 0x7f995806d3f0 2024-03-08T06:02:51.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.179+0000 7f9956ffd640 1 -- 172.21.15.76:0/1185540885 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994c030610 con 0x7f995806d3f0 2024-03-08T06:02:51.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.179+0000 7f9956ffd640 1 -- 172.21.15.76:0/1185540885 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c0309e0 con 0x7f995806d3f0 2024-03-08T06:02:51.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.179+0000 7f995e823640 1 -- 172.21.15.76:0/1185540885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f995806d3f0 msgr2=0x7f99580cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:51.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.179+0000 7f995e823640 1 --2- 172.21.15.76:0/1185540885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f995806d3f0 0x7f99580cf0e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f994c0097a0 tx=0x7f994c02b960 comp rx=0 tx=0).stop 2024-03-08T06:02:51.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.179+0000 7f995e823640 1 -- 172.21.15.76:0/1185540885 shutdown_connections 2024-03-08T06:02:51.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.179+0000 7f995e823640 1 --2- 172.21.15.76:0/1185540885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f995806d3f0 0x7f99580cf0e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:51.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.179+0000 7f995e823640 1 -- 172.21.15.76:0/1185540885 >> 172.21.15.76:0/1185540885 conn(0x7f99580cf740 msgr2=0x7f99580cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:51.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.179+0000 7f995e823640 1 -- 172.21.15.76:0/1185540885 shutdown_connections 2024-03-08T06:02:51.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.179+0000 7f995e823640 1 -- 172.21.15.76:0/1185540885 wait complete. 2024-03-08T06:02:51.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.180+0000 7f995e823640 1 Processor -- start 2024-03-08T06:02:51.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.181+0000 7f995e823640 1 -- start start 2024-03-08T06:02:51.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.181+0000 7f995e823640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9958071390 0x7f995815d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:51.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.181+0000 7f995e823640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f99580cd310 0x7f9958157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:51.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.181+0000 7f995e823640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99581584c0 0x7f99580c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:51.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.181+0000 7f995e823640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995815c060 con 0x7f99581584c0 2024-03-08T06:02:51.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.181+0000 7f995e823640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995815bee0 con 0x7f99580cd310 2024-03-08T06:02:51.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.181+0000 7f995e823640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99580cde90 con 0x7f9958071390 2024-03-08T06:02:51.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.181+0000 7f995cdc0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99581584c0 0x7f99580c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:51.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.181+0000 7f995cdc0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99581584c0 0x7f99580c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48498/0 (socket says 172.21.15.76:48498) 2024-03-08T06:02:51.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.182+0000 7f995cdc0640 1 -- 172.21.15.76:0/1353353898 learned_addr learned my addr 172.21.15.76:0/1353353898 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:51.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.182+0000 7f9957fff640 1 --2- 172.21.15.76:0/1353353898 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9958071390 0x7f995815d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:51.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.182+0000 7f99577fe640 1 --2- 172.21.15.76:0/1353353898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f99580cd310 0x7f9958157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:51.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.182+0000 7f995cdc0640 1 -- 172.21.15.76:0/1353353898 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9958071390 msgr2=0x7f995815d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:51.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.182+0000 7f995cdc0640 1 --2- 172.21.15.76:0/1353353898 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9958071390 0x7f995815d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:51.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.182+0000 7f995cdc0640 1 -- 172.21.15.76:0/1353353898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f99580cd310 msgr2=0x7f9958157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:51.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.182+0000 7f995cdc0640 1 --2- 172.21.15.76:0/1353353898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f99580cd310 0x7f9958157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:51.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.182+0000 7f995cdc0640 1 -- 172.21.15.76:0/1353353898 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995815d880 con 0x7f99581584c0 2024-03-08T06:02:51.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.182+0000 7f995cdc0640 1 --2- 172.21.15.76:0/1353353898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99581584c0 0x7f99580c8a90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f995000c700 tx=0x7f995000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:51.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.182+0000 7f9954ff9640 1 -- 172.21.15.76:0/1353353898 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950018070 con 0x7f99581584c0 2024-03-08T06:02:51.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.182+0000 7f9954ff9640 1 -- 172.21.15.76:0/1353353898 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995000ce10 con 0x7f99581584c0 2024-03-08T06:02:51.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.182+0000 7f995e823640 1 -- 172.21.15.76:0/1353353898 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99581589b0 con 0x7f99581584c0 2024-03-08T06:02:51.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.182+0000 7f9954ff9640 1 -- 172.21.15.76:0/1353353898 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995000a860 con 0x7f99581584c0 2024-03-08T06:02:51.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.182+0000 7f995e823640 1 -- 172.21.15.76:0/1353353898 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99580ca2c0 con 0x7f99581584c0 2024-03-08T06:02:51.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.183+0000 7f995e823640 1 -- 172.21.15.76:0/1353353898 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995804f030 con 0x7f99581584c0 2024-03-08T06:02:51.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.184+0000 7f9954ff9640 1 -- 172.21.15.76:0/1353353898 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f995001b070 con 0x7f99581584c0 2024-03-08T06:02:51.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.184+0000 7f9954ff9640 1 --2- 172.21.15.76:0/1353353898 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f994406b4d0 0x7f994406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:51.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.184+0000 7f9957fff640 1 --2- 172.21.15.76:0/1353353898 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f994406b4d0 0x7f994406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:51.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.184+0000 7f9954ff9640 1 -- 172.21.15.76:0/1353353898 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9950089610 con 0x7f99581584c0 2024-03-08T06:02:51.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.184+0000 7f9957fff640 1 --2- 172.21.15.76:0/1353353898 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f994406b4d0 0x7f994406d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9948000f60 tx=0x7f9948009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:51.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.188+0000 7f9954ff9640 1 -- 172.21.15.76:0/1353353898 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9950058600 con 0x7f99581584c0 2024-03-08T06:02:51.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.359+0000 7f995e823640 1 -- 172.21.15.76:0/1353353898 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9958072170 con 0x7f99581584c0 2024-03-08T06:02:51.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.360+0000 7f9954ff9640 1 -- 172.21.15.76:0/1353353898 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+13209 (secure 0 0 0) 0x7f9950057f00 con 0x7f99581584c0 2024-03-08T06:02:51.361 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:02:51.361 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":45,"fsid":"8a42235b-b7c3-45b2-aa98-336e6b1846d1","created":"2024-03-08T06:00:11.743672+0000","modified":"2024-03-08T06:02:48.086903+0000","last_up_change":"2024-03-08T06:00:23.027779+0000","last_in_change":"2024-03-08T06:00:13.990205+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:00:23.961657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b75bfb24-86a7-47d5-8a16-656c0f0d6f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6812","nonce":280957956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6816","nonce":280957956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6823","nonce":280957956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":280957956},{"type":"v1","addr":"172.21.15.76:6821","nonce":280957956}]},"public_addr":"172.21.15.76:6812/280957956","cluster_addr":"172.21.15.76:6816/280957956","heartbeat_back_addr":"172.21.15.76:6823/280957956","heartbeat_front_addr":"172.21.15.76:6821/280957956","state":["exists","up"]},{"osd":1,"uuid":"3bedb444-41a6-4622-b5b1-7ee426a7bd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6801","nonce":712186592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6803","nonce":712186592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6807","nonce":712186592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":712186592},{"type":"v1","addr":"172.21.15.76:6805","nonce":712186592}]},"public_addr":"172.21.15.76:6801/712186592","cluster_addr":"172.21.15.76:6803/712186592","heartbeat_back_addr":"172.21.15.76:6807/712186592","heartbeat_front_addr":"172.21.15.76:6805/712186592","state":["exists","up"]},{"osd":2,"uuid":"6cc45538-985d-4a82-ba35-0bc1019dcd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6825","nonce":1112475349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6827","nonce":1112475349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6831","nonce":1112475349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1112475349},{"type":"v1","addr":"172.21.15.76:6829","nonce":1112475349}]},"public_addr":"172.21.15.76:6825/1112475349","cluster_addr":"172.21.15.76:6827/1112475349","heartbeat_back_addr":"172.21.15.76:6831/1112475349","heartbeat_front_addr":"172.21.15.76:6829/1112475349","state":["exists","up"]},{"osd":3,"uuid":"60266e20-1c28-4910-9963-62fce37868f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6809","nonce":2501234044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6813","nonce":2501234044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6819","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6820","nonce":2501234044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6815","nonce":2501234044},{"type":"v1","addr":"172.21.15.76:6817","nonce":2501234044}]},"public_addr":"172.21.15.76:6809/2501234044","cluster_addr":"172.21.15.76:6813/2501234044","heartbeat_back_addr":"172.21.15.76:6820/2501234044","heartbeat_front_addr":"172.21.15.76:6817/2501234044","state":["exists","up"]},{"osd":4,"uuid":"f7dbb15c-028c-4e1f-94a4-4051041e3bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6801","nonce":3228626670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6803","nonce":3228626670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6807","nonce":3228626670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3228626670},{"type":"v1","addr":"172.21.15.81:6805","nonce":3228626670}]},"public_addr":"172.21.15.81:6801/3228626670","cluster_addr":"172.21.15.81:6803/3228626670","heartbeat_back_addr":"172.21.15.81:6807/3228626670","heartbeat_front_addr":"172.21.15.81:6805/3228626670","state":["exists","up"]},{"osd":5,"uuid":"694b04e6-f982-4600-a221-c1f0cd7d6d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6809","nonce":173715776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6811","nonce":173715776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6815","nonce":173715776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":173715776},{"type":"v1","addr":"172.21.15.81:6813","nonce":173715776}]},"public_addr":"172.21.15.81:6809/173715776","cluster_addr":"172.21.15.81:6811/173715776","heartbeat_back_addr":"172.21.15.81:6815/173715776","heartbeat_front_addr":"172.21.15.81:6813/173715776","state":["exists","up"]},{"osd":6,"uuid":"58469229-276a-4a60-99bc-2ab5ee99d9b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6825","nonce":3040851989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6827","nonce":3040851989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6831","nonce":3040851989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3040851989},{"type":"v1","addr":"172.21.15.81:6829","nonce":3040851989}]},"public_addr":"172.21.15.81:6825/3040851989","cluster_addr":"172.21.15.81:6827/3040851989","heartbeat_back_addr":"172.21.15.81:6831/3040851989","heartbeat_front_addr":"172.21.15.81:6829/3040851989","state":["exists","up"]},{"osd":7,"uuid":"685d3757-2cbd-4ff6-8ef1-8e391d9a64f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6817","nonce":298940182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6819","nonce":298940182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6823","nonce":298940182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":298940182},{"type":"v1","addr":"172.21.15.81:6821","nonce":298940182}]},"public_addr":"172.21.15.81:6817/298940182","cluster_addr":"172.21.15.81:6819/298940182","heartbeat_back_addr":"172.21.15.81:6823/298940182","heartbeat_front_addr":"172.21.15.81:6821/298940182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.129808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.074636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.178588+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.210059+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.095181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.089022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.163014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:00:19.194197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6834/1503895376":"2024-03-09T06:02:44.983686+0000","172.21.15.81:6832/1503895376":"2024-03-09T06:02:44.983686+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:02:51.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.362+0000 7f995e823640 1 -- 172.21.15.76:0/1353353898 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f994406b4d0 msgr2=0x7f994406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:51.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.362+0000 7f995e823640 1 --2- 172.21.15.76:0/1353353898 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f994406b4d0 0x7f994406d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9948000f60 tx=0x7f9948009040 comp rx=0 tx=0).stop 2024-03-08T06:02:51.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.362+0000 7f995e823640 1 -- 172.21.15.76:0/1353353898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99581584c0 msgr2=0x7f99580c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:51.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.362+0000 7f995e823640 1 --2- 172.21.15.76:0/1353353898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99581584c0 0x7f99580c8a90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f995000c700 tx=0x7f995000cbc0 comp rx=0 tx=0).stop 2024-03-08T06:02:51.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.362+0000 7f995e823640 1 -- 172.21.15.76:0/1353353898 shutdown_connections 2024-03-08T06:02:51.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.362+0000 7f995e823640 1 --2- 172.21.15.76:0/1353353898 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f994406b4d0 0x7f994406d990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:51.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.362+0000 7f995e823640 1 --2- 172.21.15.76:0/1353353898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99581584c0 0x7f99580c8a90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:51.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.362+0000 7f995e823640 1 --2- 172.21.15.76:0/1353353898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f99580cd310 0x7f9958157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:51.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.362+0000 7f995e823640 1 --2- 172.21.15.76:0/1353353898 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9958071390 0x7f995815d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:51.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.362+0000 7f995e823640 1 -- 172.21.15.76:0/1353353898 >> 172.21.15.76:0/1353353898 conn(0x7f99580cf740 msgr2=0x7f99580b5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:51.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.362+0000 7f995e823640 1 -- 172.21.15.76:0/1353353898 shutdown_connections 2024-03-08T06:02:51.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:51.362+0000 7f995e823640 1 -- 172.21.15.76:0/1353353898 wait complete. 2024-03-08T06:02:52.375 INFO:tasks.ceph:Scrubbing osd.0 2024-03-08T06:02:52.375 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-08T06:02:52.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.488+0000 7fa3dc348640 1 Processor -- start 2024-03-08T06:02:52.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.488+0000 7fa3dc348640 1 -- start start 2024-03-08T06:02:52.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.489+0000 7fa3dc348640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa3d40af310 0x7fa3d406ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.489+0000 7fa3dc348640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d40ad720 con 0x7fa3d406b1c0 2024-03-08T06:02:52.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.489+0000 7fa3dc348640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d40ad9c0 con 0x7fa3d415f510 2024-03-08T06:02:52.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.489+0000 7fa3dc348640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d406ea60 con 0x7fa3d40af310 2024-03-08T06:02:52.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.489+0000 7fa3d98e3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa3d40af310 0x7fa3d406ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.489+0000 7fa3da8e5640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa3d406b1c0 0x7fa3d415f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57976/0 (socket says 172.21.15.76:57976) 2024-03-08T06:02:52.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.489+0000 7fa3da8e5640 1 -- 172.21.15.76:0/1288227354 learned_addr learned my addr 172.21.15.76:0/1288227354 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:52.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.489+0000 7fa3d98e3640 1 -- 172.21.15.76:0/1288227354 >> v1:172.21.15.81:6789/0 conn(0x7fa3d415f510 legacy=0x7fa3d406f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.489+0000 7fa3d98e3640 1 -- 172.21.15.76:0/1288227354 >> v1:172.21.15.76:6789/0 conn(0x7fa3d406b1c0 legacy=0x7fa3d415f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.489+0000 7fa3d98e3640 1 -- 172.21.15.76:0/1288227354 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d40adca0 con 0x7fa3d40af310 2024-03-08T06:02:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.489+0000 7fa3d90e2640 1 -- 172.21.15.76:0/1288227354 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494586146 0 0) 0x7fa3d40ad720 con 0x7fa3d406b1c0 2024-03-08T06:02:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.490+0000 7fa3d98e3640 1 --2- 172.21.15.76:0/1288227354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3d40af310 0x7fa3d406ac80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40097a0 tx=0x7fa3c402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10cb5d64eb940113 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.490+0000 7fa3d90e2640 1 -- 172.21.15.76:0/1288227354 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4034070 con 0x7fa3d40af310 2024-03-08T06:02:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.490+0000 7fa3d90e2640 1 -- 172.21.15.76:0/1288227354 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c4030610 con 0x7fa3d40af310 2024-03-08T06:02:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.490+0000 7fa3d90e2640 1 -- 172.21.15.76:0/1288227354 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c40309e0 con 0x7fa3d40af310 2024-03-08T06:02:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.491+0000 7fa3dc348640 1 -- 172.21.15.76:0/1288227354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3d40af310 msgr2=0x7fa3d406ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.491+0000 7fa3dc348640 1 --2- 172.21.15.76:0/1288227354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3d40af310 0x7fa3d406ac80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40097a0 tx=0x7fa3c402b960 comp rx=0 tx=0).stop 2024-03-08T06:02:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.491+0000 7fa3dc348640 1 -- 172.21.15.76:0/1288227354 shutdown_connections 2024-03-08T06:02:52.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.491+0000 7fa3dc348640 1 --2- 172.21.15.76:0/1288227354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3d40af310 0x7fa3d406ac80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.491+0000 7fa3dc348640 1 -- 172.21.15.76:0/1288227354 >> 172.21.15.76:0/1288227354 conn(0x7fa3d40cfed0 msgr2=0x7fa3d40d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:52.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.491+0000 7fa3dc348640 1 -- 172.21.15.76:0/1288227354 shutdown_connections 2024-03-08T06:02:52.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.491+0000 7fa3dc348640 1 -- 172.21.15.76:0/1288227354 wait complete. 2024-03-08T06:02:52.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.492+0000 7fa3dc348640 1 Processor -- start 2024-03-08T06:02:52.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.492+0000 7fa3dc348640 1 -- start start 2024-03-08T06:02:52.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3dc348640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3d406b1c0 0x7fa3d414de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3dc348640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3d40af310 0x7fa3d414f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3dc348640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3d415f510 0x7fa3d414fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3dc348640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40af790 con 0x7fa3d40af310 2024-03-08T06:02:52.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3dc348640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d406ed40 con 0x7fa3d406b1c0 2024-03-08T06:02:52.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3dc348640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d406f8e0 con 0x7fa3d415f510 2024-03-08T06:02:52.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3d98e3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3d40af310 0x7fa3d414f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3d98e3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3d40af310 0x7fa3d414f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48510/0 (socket says 172.21.15.76:48510) 2024-03-08T06:02:52.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3d98e3640 1 -- 172.21.15.76:0/743014730 learned_addr learned my addr 172.21.15.76:0/743014730 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:52.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3da0e4640 1 --2- 172.21.15.76:0/743014730 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3d406b1c0 0x7fa3d414de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3da8e5640 1 --2- 172.21.15.76:0/743014730 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3d415f510 0x7fa3d414fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3d98e3640 1 -- 172.21.15.76:0/743014730 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3d415f510 msgr2=0x7fa3d414fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3d98e3640 1 --2- 172.21.15.76:0/743014730 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3d415f510 0x7fa3d414fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3d98e3640 1 -- 172.21.15.76:0/743014730 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3d406b1c0 msgr2=0x7fa3d414de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3d98e3640 1 --2- 172.21.15.76:0/743014730 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3d406b1c0 0x7fa3d414de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3d98e3640 1 -- 172.21.15.76:0/743014730 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d40b4ea0 con 0x7fa3d40af310 2024-03-08T06:02:52.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3da0e4640 1 --2- 172.21.15.76:0/743014730 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3d406b1c0 0x7fa3d414de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:52.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3da8e5640 1 --2- 172.21.15.76:0/743014730 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3d415f510 0x7fa3d414fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:52.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3d98e3640 1 --2- 172.21.15.76:0/743014730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3d40af310 0x7fa3d414f940 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa3c402bf70 tx=0x7fa3c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:52.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3caffd640 1 -- 172.21.15.76:0/743014730 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4034040 con 0x7fa3d40af310 2024-03-08T06:02:52.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3caffd640 1 -- 172.21.15.76:0/743014730 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c4039a10 con 0x7fa3d40af310 2024-03-08T06:02:52.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3dc348640 1 -- 172.21.15.76:0/743014730 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d40b2f60 con 0x7fa3d40af310 2024-03-08T06:02:52.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3caffd640 1 -- 172.21.15.76:0/743014730 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4039e20 con 0x7fa3d40af310 2024-03-08T06:02:52.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.493+0000 7fa3dc348640 1 -- 172.21.15.76:0/743014730 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d40b3520 con 0x7fa3d40af310 2024-03-08T06:02:52.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.494+0000 7fa3dc348640 1 -- 172.21.15.76:0/743014730 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa3d404f030 con 0x7fa3d40af310 2024-03-08T06:02:52.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.495+0000 7fa3caffd640 1 -- 172.21.15.76:0/743014730 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3c4037070 con 0x7fa3d40af310 2024-03-08T06:02:52.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.495+0000 7fa3caffd640 1 --2- 172.21.15.76:0/743014730 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa3b806b500 0x7fa3b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.495+0000 7fa3caffd640 1 -- 172.21.15.76:0/743014730 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa3c40ac190 con 0x7fa3d40af310 2024-03-08T06:02:52.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.495+0000 7fa3da0e4640 1 --2- 172.21.15.76:0/743014730 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa3b806b500 0x7fa3b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.495+0000 7fa3caffd640 1 --2- 172.21.15.76:0/743014730 >> [v2:172.21.15.76:6810/280957956,v1:172.21.15.76:6812/280957956] conn(0x7fa3b80719a0 0x7fa3b8073de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.495+0000 7fa3caffd640 1 -- 172.21.15.76:0/743014730 --> [v2:172.21.15.76:6810/280957956,v1:172.21.15.76:6812/280957956] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3c4004490 con 0x7fa3b80719a0 2024-03-08T06:02:52.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.495+0000 7fa3caffd640 1 -- 172.21.15.76:0/743014730 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3c40b3050 con 0x7fa3d40af310 2024-03-08T06:02:52.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.495+0000 7fa3da8e5640 1 --2- 172.21.15.76:0/743014730 >> [v2:172.21.15.76:6810/280957956,v1:172.21.15.76:6812/280957956] conn(0x7fa3b80719a0 0x7fa3b8073de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.496+0000 7fa3da0e4640 1 --2- 172.21.15.76:0/743014730 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa3b806b500 0x7fa3b806d9c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0007fd0 tx=0x7fa3d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:52.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.496+0000 7fa3da8e5640 1 --2- 172.21.15.76:0/743014730 >> [v2:172.21.15.76:6810/280957956,v1:172.21.15.76:6812/280957956] conn(0x7fa3b80719a0 0x7fa3b8073de0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:52.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.496+0000 7fa3caffd640 1 -- 172.21.15.76:0/743014730 <== osd.0 v2:172.21.15.76:6810/280957956 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa3c4004490 con 0x7fa3b80719a0 2024-03-08T06:02:52.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.509+0000 7fa3dc348640 1 -- 172.21.15.76:0/743014730 --> [v2:172.21.15.76:6810/280957956,v1:172.21.15.76:6812/280957956] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa3d4066f10 con 0x7fa3b80719a0 2024-03-08T06:02:52.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.510+0000 7fa3caffd640 1 -- 172.21.15.76:0/743014730 <== osd.0 v2:172.21.15.76:6810/280957956 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa3d4066f10 con 0x7fa3b80719a0 2024-03-08T06:02:52.510 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-08T06:02:52.511 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T06:02:52.511 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-08T06:02:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.510+0000 7fa3dc348640 1 -- 172.21.15.76:0/743014730 >> [v2:172.21.15.76:6810/280957956,v1:172.21.15.76:6812/280957956] conn(0x7fa3b80719a0 msgr2=0x7fa3b8073de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.510+0000 7fa3dc348640 1 --2- 172.21.15.76:0/743014730 >> [v2:172.21.15.76:6810/280957956,v1:172.21.15.76:6812/280957956] conn(0x7fa3b80719a0 0x7fa3b8073de0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.510+0000 7fa3dc348640 1 -- 172.21.15.76:0/743014730 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa3b806b500 msgr2=0x7fa3b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.510+0000 7fa3dc348640 1 --2- 172.21.15.76:0/743014730 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa3b806b500 0x7fa3b806d9c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0007fd0 tx=0x7fa3d0009040 comp rx=0 tx=0).stop 2024-03-08T06:02:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.510+0000 7fa3dc348640 1 -- 172.21.15.76:0/743014730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3d40af310 msgr2=0x7fa3d414f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.510+0000 7fa3dc348640 1 --2- 172.21.15.76:0/743014730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3d40af310 0x7fa3d414f940 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa3c402bf70 tx=0x7fa3c40046c0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.510+0000 7fa3da8e5640 1 -- 172.21.15.76:0/743014730 reap_dead start 2024-03-08T06:02:52.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.510+0000 7fa3dc348640 1 -- 172.21.15.76:0/743014730 shutdown_connections 2024-03-08T06:02:52.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.510+0000 7fa3dc348640 1 -- 172.21.15.76:0/743014730 >> 172.21.15.76:0/743014730 conn(0x7fa3d40cfed0 msgr2=0x7fa3d40ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:52.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.510+0000 7fa3dc348640 1 -- 172.21.15.76:0/743014730 shutdown_connections 2024-03-08T06:02:52.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.510+0000 7fa3dc348640 1 -- 172.21.15.76:0/743014730 wait complete. 2024-03-08T06:02:52.521 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-08T06:02:52.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.621+0000 7fbee73a1640 1 Processor -- start 2024-03-08T06:02:52.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.621+0000 7fbee73a1640 1 -- start start 2024-03-08T06:02:52.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.622+0000 7fbee73a1640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbee015dbc0 0x7fbee00d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.622+0000 7fbee73a1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbee00b5cf0 0x7fbee00b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.622+0000 7fbee73a1640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee00d27d0 con 0x7fbee00b9840 2024-03-08T06:02:52.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.622+0000 7fbee73a1640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee00d2ab0 con 0x7fbee00b5cf0 2024-03-08T06:02:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.622+0000 7fbee73a1640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00d2d90 con 0x7fbee015dbc0 2024-03-08T06:02:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.622+0000 7fbee593e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbee00b5cf0 0x7fbee00b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.622+0000 7fbee493c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbee00b9840 0x7fbee00b55e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57982/0 (socket says 172.21.15.76:57982) 2024-03-08T06:02:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.622+0000 7fbee493c640 1 -- 172.21.15.76:0/3831799567 learned_addr learned my addr 172.21.15.76:0/3831799567 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.623+0000 7fbee513d640 1 --2- 172.21.15.76:0/3831799567 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbee015dbc0 0x7fbee00d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.623+0000 7fbee593e640 1 -- 172.21.15.76:0/3831799567 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbee015dbc0 msgr2=0x7fbee00d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.623+0000 7fbee593e640 1 --2- 172.21.15.76:0/3831799567 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbee015dbc0 0x7fbee00d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.623+0000 7fbee593e640 1 -- 172.21.15.76:0/3831799567 >> v1:172.21.15.76:6789/0 conn(0x7fbee00b9840 legacy=0x7fbee00b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.623+0000 7fbed7fff640 1 -- 172.21.15.76:0/3831799567 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102041620 0 0) 0x7fbee00d27d0 con 0x7fbee00b9840 2024-03-08T06:02:52.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.623+0000 7fbee593e640 1 -- 172.21.15.76:0/3831799567 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee00d3070 con 0x7fbee00b5cf0 2024-03-08T06:02:52.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.623+0000 7fbee513d640 1 --2- 172.21.15.76:0/3831799567 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbee015dbc0 0x7fbee00d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:52.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.623+0000 7fbee593e640 1 --2- 172.21.15.76:0/3831799567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee00b5cf0 0x7fbee00b6c90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009d90 tx=0x7fbed802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3f3f5a07b6d06c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:52.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.624+0000 7fbed7fff640 1 -- 172.21.15.76:0/3831799567 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8039070 con 0x7fbee00b5cf0 2024-03-08T06:02:52.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.624+0000 7fbed7fff640 1 -- 172.21.15.76:0/3831799567 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed802cc50 con 0x7fbee00b5cf0 2024-03-08T06:02:52.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.624+0000 7fbed7fff640 1 -- 172.21.15.76:0/3831799567 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8034440 con 0x7fbee00b5cf0 2024-03-08T06:02:52.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.624+0000 7fbee73a1640 1 -- 172.21.15.76:0/3831799567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee00b5cf0 msgr2=0x7fbee00b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.624+0000 7fbee73a1640 1 --2- 172.21.15.76:0/3831799567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee00b5cf0 0x7fbee00b6c90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009d90 tx=0x7fbed802bea0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.624+0000 7fbee73a1640 1 -- 172.21.15.76:0/3831799567 shutdown_connections 2024-03-08T06:02:52.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.624+0000 7fbee73a1640 1 --2- 172.21.15.76:0/3831799567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee00b5cf0 0x7fbee00b6c90 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.624+0000 7fbee73a1640 1 --2- 172.21.15.76:0/3831799567 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbee015dbc0 0x7fbee00d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.624+0000 7fbee73a1640 1 -- 172.21.15.76:0/3831799567 >> 172.21.15.76:0/3831799567 conn(0x7fbee00670d0 msgr2=0x7fbee00d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:52.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.624+0000 7fbee73a1640 1 -- 172.21.15.76:0/3831799567 shutdown_connections 2024-03-08T06:02:52.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.624+0000 7fbee73a1640 1 -- 172.21.15.76:0/3831799567 wait complete. 2024-03-08T06:02:52.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.625+0000 7fbee73a1640 1 Processor -- start 2024-03-08T06:02:52.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.625+0000 7fbee73a1640 1 -- start start 2024-03-08T06:02:52.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.626+0000 7fbee73a1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee00b5cf0 0x7fbee014c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.626+0000 7fbee73a1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbee00b9840 0x7fbee00ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.626+0000 7fbee73a1640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbee015dbc0 0x7fbee00cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.626+0000 7fbee73a1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00d25b0 con 0x7fbee00b5cf0 2024-03-08T06:02:52.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.626+0000 7fbee73a1640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00d2430 con 0x7fbee00b9840 2024-03-08T06:02:52.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.626+0000 7fbee73a1640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee015dfa0 con 0x7fbee015dbc0 2024-03-08T06:02:52.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.626+0000 7fbee513d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee00b5cf0 0x7fbee014c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.626+0000 7fbee513d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee00b5cf0 0x7fbee014c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48522/0 (socket says 172.21.15.76:48522) 2024-03-08T06:02:52.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.626+0000 7fbee513d640 1 -- 172.21.15.76:0/687492540 learned_addr learned my addr 172.21.15.76:0/687492540 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:52.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.626+0000 7fbee513d640 1 -- 172.21.15.76:0/687492540 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbee015dbc0 msgr2=0x7fbee00cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.626+0000 7fbee513d640 1 --2- 172.21.15.76:0/687492540 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbee015dbc0 0x7fbee00cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.626+0000 7fbee513d640 1 -- 172.21.15.76:0/687492540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbee00b9840 msgr2=0x7fbee00ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.626+0000 7fbee513d640 1 --2- 172.21.15.76:0/687492540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbee00b9840 0x7fbee00ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.626+0000 7fbee513d640 1 -- 172.21.15.76:0/687492540 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee00cbb90 con 0x7fbee00b5cf0 2024-03-08T06:02:52.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.627+0000 7fbee513d640 1 --2- 172.21.15.76:0/687492540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee00b5cf0 0x7fbee014c0f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00d7d0 tx=0x7fbedc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:52.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.627+0000 7fbed5ffb640 1 -- 172.21.15.76:0/687492540 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc004030 con 0x7fbee00b5cf0 2024-03-08T06:02:52.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.627+0000 7fbed5ffb640 1 -- 172.21.15.76:0/687492540 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc0041d0 con 0x7fbee00b5cf0 2024-03-08T06:02:52.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.627+0000 7fbee73a1640 1 -- 172.21.15.76:0/687492540 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee00cbe70 con 0x7fbee00b5cf0 2024-03-08T06:02:52.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.627+0000 7fbed5ffb640 1 -- 172.21.15.76:0/687492540 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0096a0 con 0x7fbee00b5cf0 2024-03-08T06:02:52.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.627+0000 7fbee73a1640 1 -- 172.21.15.76:0/687492540 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee00cc430 con 0x7fbee00b5cf0 2024-03-08T06:02:52.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.628+0000 7fbee73a1640 1 -- 172.21.15.76:0/687492540 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee004f030 con 0x7fbee00b5cf0 2024-03-08T06:02:52.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.628+0000 7fbed5ffb640 1 -- 172.21.15.76:0/687492540 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbedc009840 con 0x7fbee00b5cf0 2024-03-08T06:02:52.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.629+0000 7fbed5ffb640 1 --2- 172.21.15.76:0/687492540 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbec406b4d0 0x7fbec406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.629+0000 7fbee493c640 1 --2- 172.21.15.76:0/687492540 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbec406b4d0 0x7fbec406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.629+0000 7fbed5ffb640 1 -- 172.21.15.76:0/687492540 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbedc08a3c0 con 0x7fbee00b5cf0 2024-03-08T06:02:52.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.629+0000 7fbee493c640 1 --2- 172.21.15.76:0/687492540 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbec406b4d0 0x7fbec406d990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbed0007fd0 tx=0x7fbed0007900 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:52.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.633+0000 7fbed5ffb640 1 -- 172.21.15.76:0/687492540 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbedc0593b0 con 0x7fbee00b5cf0 2024-03-08T06:02:52.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.804+0000 7fbee73a1640 1 -- 172.21.15.76:0/687492540 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fbee00cd170 con 0x7fbec406b4d0 2024-03-08T06:02:52.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.804+0000 7fbed5ffb640 1 -- 172.21.15.76:0/687492540 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbed00070d0 con 0x7fbec406b4d0 2024-03-08T06:02:52.805 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 0 to deep-scrub 2024-03-08T06:02:52.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.806+0000 7fbee73a1640 1 -- 172.21.15.76:0/687492540 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbec406b4d0 msgr2=0x7fbec406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.806+0000 7fbee73a1640 1 --2- 172.21.15.76:0/687492540 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbec406b4d0 0x7fbec406d990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbed0007fd0 tx=0x7fbed0007900 comp rx=0 tx=0).stop 2024-03-08T06:02:52.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.806+0000 7fbee73a1640 1 -- 172.21.15.76:0/687492540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee00b5cf0 msgr2=0x7fbee014c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.806+0000 7fbee73a1640 1 --2- 172.21.15.76:0/687492540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee00b5cf0 0x7fbee014c0f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00d7d0 tx=0x7fbedc00dc90 comp rx=0 tx=0).stop 2024-03-08T06:02:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.806+0000 7fbee73a1640 1 -- 172.21.15.76:0/687492540 shutdown_connections 2024-03-08T06:02:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.806+0000 7fbee73a1640 1 --2- 172.21.15.76:0/687492540 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fbec406b4d0 0x7fbec406d990 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.806+0000 7fbee73a1640 1 --2- 172.21.15.76:0/687492540 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbee015dbc0 0x7fbee00cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.806+0000 7fbee73a1640 1 --2- 172.21.15.76:0/687492540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbee00b9840 0x7fbee00ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.806+0000 7fbee73a1640 1 --2- 172.21.15.76:0/687492540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee00b5cf0 0x7fbee014c0f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.806+0000 7fbee73a1640 1 -- 172.21.15.76:0/687492540 >> 172.21.15.76:0/687492540 conn(0x7fbee00670d0 msgr2=0x7fbee00c7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:52.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.806+0000 7fbee73a1640 1 -- 172.21.15.76:0/687492540 shutdown_connections 2024-03-08T06:02:52.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.806+0000 7fbee73a1640 1 -- 172.21.15.76:0/687492540 wait complete. 2024-03-08T06:02:52.818 INFO:tasks.ceph:Scrubbing osd.1 2024-03-08T06:02:52.818 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-08T06:02:52.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.922+0000 7fd029c6f640 1 Processor -- start 2024-03-08T06:02:52.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.922+0000 7fd029c6f640 1 -- start start 2024-03-08T06:02:52.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.923+0000 7fd029c6f640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd02406ac80 0x7fd0240cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.923+0000 7fd029c6f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0240b31a0 con 0x7fd0240cc080 2024-03-08T06:02:52.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.923+0000 7fd029c6f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02406b310 con 0x7fd0240cc9d0 2024-03-08T06:02:52.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.923+0000 7fd029c6f640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd02406b0f0 con 0x7fd02406ac80 2024-03-08T06:02:52.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.923+0000 7fd0237fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd0240cc080 0x7fd02406ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57986/0 (socket says 172.21.15.76:57986) 2024-03-08T06:02:52.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.923+0000 7fd0237fe640 1 -- 172.21.15.76:0/3613286507 learned_addr learned my addr 172.21.15.76:0/3613286507 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:52.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.923+0000 7fd0227fc640 1 -- 172.21.15.76:0/3613286507 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075648690 0 0) 0x7fd0240b31a0 con 0x7fd0240cc080 2024-03-08T06:02:52.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.923+0000 7fd0227fc640 1 -- 172.21.15.76:0/3613286507 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd00c003520 con 0x7fd0240cc080 2024-03-08T06:02:52.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.923+0000 7fd023fff640 1 --2- 172.21.15.76:0/3613286507 >> v2:172.21.15.81:3300/0 conn(0x7fd02406ac80 0x7fd0240cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.923+0000 7fd0227fc640 1 -- 172.21.15.76:0/3613286507 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 482462943 0 0) 0x7fd00c003520 con 0x7fd0240cc080 2024-03-08T06:02:52.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.923+0000 7fd0227fc640 1 -- 172.21.15.76:0/3613286507 >> v2:172.21.15.81:3300/0 conn(0x7fd02406ac80 msgr2=0x7fd0240cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:52.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.923+0000 7fd0227fc640 1 --2- 172.21.15.76:0/3613286507 >> v2:172.21.15.81:3300/0 conn(0x7fd02406ac80 0x7fd0240cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.923+0000 7fd0227fc640 1 -- 172.21.15.76:0/3613286507 >> v1:172.21.15.81:6789/0 conn(0x7fd0240cc9d0 legacy=0x7fd0240cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.923+0000 7fd0227fc640 1 -- 172.21.15.76:0/3613286507 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02406b5f0 con 0x7fd0240cc080 2024-03-08T06:02:52.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.923+0000 7fd0227fc640 1 -- 172.21.15.76:0/3613286507 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fd014001690 con 0x7fd0240cc080 2024-03-08T06:02:52.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.924+0000 7fd0227fc640 1 -- 172.21.15.76:0/3613286507 >> v1:172.21.15.76:6789/0 conn(0x7fd0240cc080 legacy=0x7fd02406ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.924+0000 7fd0227fc640 1 --2- 172.21.15.76:0/3613286507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd00c0039b0 0x7fd00c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.924+0000 7fd0227fc640 1 --2- 172.21.15.76:0/3613286507 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd00c004800 0x7fd00c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.924+0000 7fd0227fc640 1 --2- 172.21.15.76:0/3613286507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd00c0051c0 0x7fd00c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.924+0000 7fd0227fc640 1 -- 172.21.15.76:0/3613286507 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02406b0f0 con 0x7fd00c0051c0 2024-03-08T06:02:52.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.924+0000 7fd0227fc640 1 -- 172.21.15.76:0/3613286507 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c00c060 con 0x7fd00c0039b0 2024-03-08T06:02:52.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.924+0000 7fd0227fc640 1 -- 172.21.15.76:0/3613286507 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c00c260 con 0x7fd00c004800 2024-03-08T06:02:52.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.924+0000 7fd023fff640 1 --2- 172.21.15.76:0/3613286507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd00c0051c0 0x7fd00c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.924+0000 7fd0237fe640 1 --2- 172.21.15.76:0/3613286507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd00c0039b0 0x7fd00c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.924+0000 7fd022ffd640 1 --2- 172.21.15.76:0/3613286507 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd00c004800 0x7fd00c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.924+0000 7fd023fff640 1 -- 172.21.15.76:0/3613286507 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd00c004800 msgr2=0x7fd00c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:52.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.924+0000 7fd023fff640 1 --2- 172.21.15.76:0/3613286507 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd00c004800 0x7fd00c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.924+0000 7fd023fff640 1 -- 172.21.15.76:0/3613286507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd00c0039b0 msgr2=0x7fd00c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:52.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.924+0000 7fd023fff640 1 --2- 172.21.15.76:0/3613286507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd00c0039b0 0x7fd00c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.924+0000 7fd023fff640 1 -- 172.21.15.76:0/3613286507 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c003520 con 0x7fd00c0051c0 2024-03-08T06:02:52.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.925+0000 7fd023fff640 1 -- 172.21.15.76:0/3613286507 reap_dead start 2024-03-08T06:02:52.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.925+0000 7fd023fff640 1 --2- 172.21.15.76:0/3613286507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd00c0051c0 0x7fd00c00b9c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd01c009fc0 tx=0x7fd00c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca77512815084dbb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:52.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.925+0000 7fd0227fc640 1 -- 172.21.15.76:0/3613286507 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c009c10 con 0x7fd00c0051c0 2024-03-08T06:02:52.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.925+0000 7fd0227fc640 1 -- 172.21.15.76:0/3613286507 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01c02cca0 con 0x7fd00c0051c0 2024-03-08T06:02:52.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.925+0000 7fd0227fc640 1 -- 172.21.15.76:0/3613286507 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c003bd0 con 0x7fd00c0051c0 2024-03-08T06:02:52.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.925+0000 7fd029c6f640 1 -- 172.21.15.76:0/3613286507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd00c0051c0 msgr2=0x7fd00c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.925+0000 7fd029c6f640 1 --2- 172.21.15.76:0/3613286507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd00c0051c0 0x7fd00c00b9c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd01c009fc0 tx=0x7fd00c005830 comp rx=0 tx=0).stop 2024-03-08T06:02:52.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.925+0000 7fd029c6f640 1 -- 172.21.15.76:0/3613286507 shutdown_connections 2024-03-08T06:02:52.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.925+0000 7fd029c6f640 1 --2- 172.21.15.76:0/3613286507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd00c0051c0 0x7fd00c00b9c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.925+0000 7fd029c6f640 1 -- 172.21.15.76:0/3613286507 >> 172.21.15.76:0/3613286507 conn(0x7fd0240b22b0 msgr2=0x7fd0240b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:52.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.925+0000 7fd029c6f640 1 -- 172.21.15.76:0/3613286507 shutdown_connections 2024-03-08T06:02:52.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.925+0000 7fd029c6f640 1 -- 172.21.15.76:0/3613286507 wait complete. 2024-03-08T06:02:52.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.926+0000 7fd029c6f640 1 Processor -- start 2024-03-08T06:02:52.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.927+0000 7fd029c6f640 1 -- start start 2024-03-08T06:02:52.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.927+0000 7fd029c6f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd00c0051c0 0x7fd0240c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.927+0000 7fd029c6f640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd0240ae5e0 0x7fd0240aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.927+0000 7fd029c6f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0241f3e60 0x7fd0241f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.927+0000 7fd029c6f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02406e790 con 0x7fd00c0051c0 2024-03-08T06:02:52.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.927+0000 7fd029c6f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02406e4b0 con 0x7fd0241f3e60 2024-03-08T06:02:52.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.927+0000 7fd029c6f640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240cc460 con 0x7fd0240ae5e0 2024-03-08T06:02:52.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.927+0000 7fd0237fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd00c0051c0 0x7fd0240c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.927+0000 7fd0237fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd00c0051c0 0x7fd0240c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48548/0 (socket says 172.21.15.76:48548) 2024-03-08T06:02:52.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.927+0000 7fd0237fe640 1 -- 172.21.15.76:0/3119471334 learned_addr learned my addr 172.21.15.76:0/3119471334 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:52.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.928+0000 7fd023fff640 1 --2- 172.21.15.76:0/3119471334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0241f3e60 0x7fd0241f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.928+0000 7fd022ffd640 1 --2- 172.21.15.76:0/3119471334 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd0240ae5e0 0x7fd0240aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.928+0000 7fd0237fe640 1 -- 172.21.15.76:0/3119471334 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd0240ae5e0 msgr2=0x7fd0240aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.928+0000 7fd0237fe640 1 --2- 172.21.15.76:0/3119471334 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd0240ae5e0 0x7fd0240aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.928+0000 7fd0237fe640 1 -- 172.21.15.76:0/3119471334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0241f3e60 msgr2=0x7fd0241f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.928+0000 7fd0237fe640 1 --2- 172.21.15.76:0/3119471334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0241f3e60 0x7fd0241f42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.928+0000 7fd0237fe640 1 -- 172.21.15.76:0/3119471334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0241f4820 con 0x7fd00c0051c0 2024-03-08T06:02:52.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.928+0000 7fd0237fe640 1 --2- 172.21.15.76:0/3119471334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd00c0051c0 0x7fd0240c7810 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd014001200 tx=0x7fd014007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:52.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.928+0000 7fd00bfff640 1 -- 172.21.15.76:0/3119471334 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01400f960 con 0x7fd00c0051c0 2024-03-08T06:02:52.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.928+0000 7fd00bfff640 1 -- 172.21.15.76:0/3119471334 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd014018070 con 0x7fd00c0051c0 2024-03-08T06:02:52.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.928+0000 7fd029c6f640 1 -- 172.21.15.76:0/3119471334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0240c7fb0 con 0x7fd00c0051c0 2024-03-08T06:02:52.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.928+0000 7fd00bfff640 1 -- 172.21.15.76:0/3119471334 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01400fd40 con 0x7fd00c0051c0 2024-03-08T06:02:52.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.928+0000 7fd029c6f640 1 -- 172.21.15.76:0/3119471334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0240c8570 con 0x7fd00c0051c0 2024-03-08T06:02:52.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.929+0000 7fd029c6f640 1 -- 172.21.15.76:0/3119471334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd0240c7d50 con 0x7fd00c0051c0 2024-03-08T06:02:52.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.929+0000 7fd00bfff640 1 -- 172.21.15.76:0/3119471334 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd01401d460 con 0x7fd00c0051c0 2024-03-08T06:02:52.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.930+0000 7fd00bfff640 1 --2- 172.21.15.76:0/3119471334 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd01006b4d0 0x7fd01006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.930+0000 7fd00bfff640 1 -- 172.21.15.76:0/3119471334 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd01408a040 con 0x7fd00c0051c0 2024-03-08T06:02:52.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.930+0000 7fd022ffd640 1 --2- 172.21.15.76:0/3119471334 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd01006b4d0 0x7fd01006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.930+0000 7fd00bfff640 1 --2- 172.21.15.76:0/3119471334 >> [v2:172.21.15.76:6800/712186592,v1:172.21.15.76:6801/712186592] conn(0x7fd010071970 0x7fd010073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:52.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.930+0000 7fd00bfff640 1 -- 172.21.15.76:0/3119471334 --> [v2:172.21.15.76:6800/712186592,v1:172.21.15.76:6801/712186592] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd01401be50 con 0x7fd010071970 2024-03-08T06:02:52.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.930+0000 7fd00bfff640 1 -- 172.21.15.76:0/3119471334 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd014091050 con 0x7fd00c0051c0 2024-03-08T06:02:52.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.930+0000 7fd023fff640 1 --2- 172.21.15.76:0/3119471334 >> [v2:172.21.15.76:6800/712186592,v1:172.21.15.76:6801/712186592] conn(0x7fd010071970 0x7fd010073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:52.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.930+0000 7fd022ffd640 1 --2- 172.21.15.76:0/3119471334 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd01006b4d0 0x7fd01006d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd0240ceb70 tx=0x7fd018009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:52.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.930+0000 7fd023fff640 1 --2- 172.21.15.76:0/3119471334 >> [v2:172.21.15.76:6800/712186592,v1:172.21.15.76:6801/712186592] conn(0x7fd010071970 0x7fd010073db0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:52.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.931+0000 7fd00bfff640 1 -- 172.21.15.76:0/3119471334 <== osd.1 v2:172.21.15.76:6800/712186592 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd01401be50 con 0x7fd010071970 2024-03-08T06:02:52.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.944+0000 7fd029c6f640 1 -- 172.21.15.76:0/3119471334 --> [v2:172.21.15.76:6800/712186592,v1:172.21.15.76:6801/712186592] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd0240c8a00 con 0x7fd010071970 2024-03-08T06:02:52.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.944+0000 7fd00bfff640 1 -- 172.21.15.76:0/3119471334 <== osd.1 v2:172.21.15.76:6800/712186592 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd0240c8a00 con 0x7fd010071970 2024-03-08T06:02:52.945 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-08T06:02:52.945 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T06:02:52.945 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-08T06:02:52.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.945+0000 7fd029c6f640 1 -- 172.21.15.76:0/3119471334 >> [v2:172.21.15.76:6800/712186592,v1:172.21.15.76:6801/712186592] conn(0x7fd010071970 msgr2=0x7fd010073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.945+0000 7fd029c6f640 1 --2- 172.21.15.76:0/3119471334 >> [v2:172.21.15.76:6800/712186592,v1:172.21.15.76:6801/712186592] conn(0x7fd010071970 0x7fd010073db0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:52.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.945+0000 7fd029c6f640 1 -- 172.21.15.76:0/3119471334 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd01006b4d0 msgr2=0x7fd01006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.945+0000 7fd029c6f640 1 --2- 172.21.15.76:0/3119471334 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fd01006b4d0 0x7fd01006d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd0240ceb70 tx=0x7fd018009040 comp rx=0 tx=0).stop 2024-03-08T06:02:52.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.945+0000 7fd029c6f640 1 -- 172.21.15.76:0/3119471334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd00c0051c0 msgr2=0x7fd0240c7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:52.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.945+0000 7fd029c6f640 1 --2- 172.21.15.76:0/3119471334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd00c0051c0 0x7fd0240c7810 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd014001200 tx=0x7fd014007720 comp rx=0 tx=0).stop 2024-03-08T06:02:52.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.945+0000 7fd023fff640 1 -- 172.21.15.76:0/3119471334 reap_dead start 2024-03-08T06:02:52.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.945+0000 7fd029c6f640 1 -- 172.21.15.76:0/3119471334 shutdown_connections 2024-03-08T06:02:52.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.945+0000 7fd029c6f640 1 -- 172.21.15.76:0/3119471334 >> 172.21.15.76:0/3119471334 conn(0x7fd0240b22b0 msgr2=0x7fd0240b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:52.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.945+0000 7fd029c6f640 1 -- 172.21.15.76:0/3119471334 shutdown_connections 2024-03-08T06:02:52.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:52.945+0000 7fd029c6f640 1 -- 172.21.15.76:0/3119471334 wait complete. 2024-03-08T06:02:52.955 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-08T06:02:53.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.056+0000 7fc45153e640 1 Processor -- start 2024-03-08T06:02:53.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.056+0000 7fc45153e640 1 -- start start 2024-03-08T06:02:53.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.057+0000 7fc45153e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc44c0cfbb0 0x7fc44c06b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.057+0000 7fc45153e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc44c0caee0 0x7fc44c0cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.057+0000 7fc45153e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c0d04d0 con 0x7fc44c15e1b0 2024-03-08T06:02:53.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.057+0000 7fc45153e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44c0cb360 con 0x7fc44c0caee0 2024-03-08T06:02:53.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.057+0000 7fc45153e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c15e9d0 con 0x7fc44c0cfbb0 2024-03-08T06:02:53.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.057+0000 7fc44b7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc44c15e1b0 0x7fc44c0cd4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57990/0 (socket says 172.21.15.76:57990) 2024-03-08T06:02:53.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.057+0000 7fc44b7fe640 1 -- 172.21.15.76:0/3675269685 learned_addr learned my addr 172.21.15.76:0/3675269685 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:53.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc44a7fc640 1 --2- 172.21.15.76:0/3675269685 >> v2:172.21.15.81:3300/0 conn(0x7fc44c0caee0 0x7fc44c0cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc44affd640 1 --2- 172.21.15.76:0/3675269685 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc44c0cfbb0 0x7fc44c06b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc449ffb640 1 -- 172.21.15.76:0/3675269685 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947245633 0 0) 0x7fc44c0d04d0 con 0x7fc44c15e1b0 2024-03-08T06:02:53.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc449ffb640 1 -- 172.21.15.76:0/3675269685 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc434003520 con 0x7fc44c15e1b0 2024-03-08T06:02:53.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc449ffb640 1 -- 172.21.15.76:0/3675269685 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 499386384 0 0) 0x7fc434003520 con 0x7fc44c15e1b0 2024-03-08T06:02:53.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc449ffb640 1 -- 172.21.15.76:0/3675269685 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc44c0cfbb0 msgr2=0x7fc44c06b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:53.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc449ffb640 1 --2- 172.21.15.76:0/3675269685 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc44c0cfbb0 0x7fc44c06b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc449ffb640 1 -- 172.21.15.76:0/3675269685 >> v2:172.21.15.81:3300/0 conn(0x7fc44c0caee0 msgr2=0x7fc44c0cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:53.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc449ffb640 1 --2- 172.21.15.76:0/3675269685 >> v2:172.21.15.81:3300/0 conn(0x7fc44c0caee0 0x7fc44c0cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc449ffb640 1 -- 172.21.15.76:0/3675269685 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c15ecb0 con 0x7fc44c15e1b0 2024-03-08T06:02:53.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc449ffb640 1 -- 172.21.15.76:0/3675269685 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fc4440019f0 con 0x7fc44c15e1b0 2024-03-08T06:02:53.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc449ffb640 1 -- 172.21.15.76:0/3675269685 >> v1:172.21.15.76:6789/0 conn(0x7fc44c15e1b0 legacy=0x7fc44c0cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc449ffb640 1 --2- 172.21.15.76:0/3675269685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc434003a20 0x7fc434003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc449ffb640 1 --2- 172.21.15.76:0/3675269685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc434004800 0x7fc434004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc449ffb640 1 --2- 172.21.15.76:0/3675269685 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc44c0cfbb0 0x7fc4340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc449ffb640 1 -- 172.21.15.76:0/3675269685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0cb360 con 0x7fc434003a20 2024-03-08T06:02:53.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc449ffb640 1 -- 172.21.15.76:0/3675269685 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c15e9d0 con 0x7fc434004800 2024-03-08T06:02:53.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc449ffb640 1 -- 172.21.15.76:0/3675269685 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43400b9c0 con 0x7fc44c0cfbb0 2024-03-08T06:02:53.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc44affd640 1 --2- 172.21.15.76:0/3675269685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc434003a20 0x7fc434003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc44a7fc640 1 --2- 172.21.15.76:0/3675269685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc434004800 0x7fc434004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc44b7fe640 1 --2- 172.21.15.76:0/3675269685 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc44c0cfbb0 0x7fc4340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc44affd640 1 -- 172.21.15.76:0/3675269685 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc44c0cfbb0 msgr2=0x7fc4340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:53.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc44affd640 1 --2- 172.21.15.76:0/3675269685 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc44c0cfbb0 0x7fc4340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc44affd640 1 -- 172.21.15.76:0/3675269685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc434004800 msgr2=0x7fc434004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:53.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc44affd640 1 --2- 172.21.15.76:0/3675269685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc434004800 0x7fc434004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.058+0000 7fc44affd640 1 -- 172.21.15.76:0/3675269685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc434003520 con 0x7fc434003a20 2024-03-08T06:02:53.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.059+0000 7fc44affd640 1 --2- 172.21.15.76:0/3675269685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc434003a20 0x7fc434003e40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009af0 tx=0x7fc434005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177d17a54d42e016 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.059+0000 7fc449ffb640 1 -- 172.21.15.76:0/3675269685 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c039070 con 0x7fc434003a20 2024-03-08T06:02:53.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.059+0000 7fc449ffb640 1 -- 172.21.15.76:0/3675269685 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c02c6b0 con 0x7fc434003a20 2024-03-08T06:02:53.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.059+0000 7fc449ffb640 1 -- 172.21.15.76:0/3675269685 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c009690 con 0x7fc434003a20 2024-03-08T06:02:53.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.060+0000 7fc45153e640 1 -- 172.21.15.76:0/3675269685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc434003a20 msgr2=0x7fc434003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.060+0000 7fc45153e640 1 --2- 172.21.15.76:0/3675269685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc434003a20 0x7fc434003e40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009af0 tx=0x7fc434005830 comp rx=0 tx=0).stop 2024-03-08T06:02:53.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.060+0000 7fc44b7fe640 1 -- 172.21.15.76:0/3675269685 reap_dead start 2024-03-08T06:02:53.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.060+0000 7fc45153e640 1 -- 172.21.15.76:0/3675269685 shutdown_connections 2024-03-08T06:02:53.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.060+0000 7fc45153e640 1 -- 172.21.15.76:0/3675269685 >> 172.21.15.76:0/3675269685 conn(0x7fc44c0d0890 msgr2=0x7fc44c0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:53.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.060+0000 7fc45153e640 1 -- 172.21.15.76:0/3675269685 shutdown_connections 2024-03-08T06:02:53.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.060+0000 7fc45153e640 1 -- 172.21.15.76:0/3675269685 wait complete. 2024-03-08T06:02:53.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.061+0000 7fc45153e640 1 Processor -- start 2024-03-08T06:02:53.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.061+0000 7fc45153e640 1 -- start start 2024-03-08T06:02:53.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc45153e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc434003a20 0x7fc44c06f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc45153e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc44c157300 0x7fc44c159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc45153e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc44c157760 0x7fc44c1597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc45153e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c15e610 con 0x7fc434003a20 2024-03-08T06:02:53.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc45153e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c15f270 con 0x7fc44c157300 2024-03-08T06:02:53.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc45153e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c15e790 con 0x7fc44c157760 2024-03-08T06:02:53.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc44affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc434003a20 0x7fc44c06f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc44affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc434003a20 0x7fc44c06f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48564/0 (socket says 172.21.15.76:48564) 2024-03-08T06:02:53.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc44affd640 1 -- 172.21.15.76:0/3099292859 learned_addr learned my addr 172.21.15.76:0/3099292859 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:53.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc44b7fe640 1 --2- 172.21.15.76:0/3099292859 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc44c157760 0x7fc44c1597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc44a7fc640 1 --2- 172.21.15.76:0/3099292859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc44c157300 0x7fc44c159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc44affd640 1 -- 172.21.15.76:0/3099292859 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc44c157760 msgr2=0x7fc44c1597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc44affd640 1 --2- 172.21.15.76:0/3099292859 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc44c157760 0x7fc44c1597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc44affd640 1 -- 172.21.15.76:0/3099292859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc44c157300 msgr2=0x7fc44c159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc44affd640 1 --2- 172.21.15.76:0/3099292859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc44c157300 0x7fc44c159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc44affd640 1 -- 172.21.15.76:0/3099292859 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c14ab40 con 0x7fc434003a20 2024-03-08T06:02:53.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc44a7fc640 1 --2- 172.21.15.76:0/3099292859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc44c157300 0x7fc44c159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:53.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc44affd640 1 --2- 172.21.15.76:0/3099292859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc434003a20 0x7fc44c06f210 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc43c037040 tx=0x7fc43c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc4337fe640 1 -- 172.21.15.76:0/3099292859 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c042020 con 0x7fc434003a20 2024-03-08T06:02:53.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc4337fe640 1 -- 172.21.15.76:0/3099292859 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c03a040 con 0x7fc434003a20 2024-03-08T06:02:53.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc45153e640 1 -- 172.21.15.76:0/3099292859 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c14ae20 con 0x7fc434003a20 2024-03-08T06:02:53.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.062+0000 7fc4337fe640 1 -- 172.21.15.76:0/3099292859 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c039070 con 0x7fc434003a20 2024-03-08T06:02:53.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.063+0000 7fc45153e640 1 -- 172.21.15.76:0/3099292859 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c14b3e0 con 0x7fc434003a20 2024-03-08T06:02:53.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.064+0000 7fc45153e640 1 -- 172.21.15.76:0/3099292859 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44c04f030 con 0x7fc434003a20 2024-03-08T06:02:53.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.064+0000 7fc4337fe640 1 -- 172.21.15.76:0/3099292859 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc43c045020 con 0x7fc434003a20 2024-03-08T06:02:53.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.064+0000 7fc4337fe640 1 --2- 172.21.15.76:0/3099292859 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc43806b4d0 0x7fc43806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.064+0000 7fc4337fe640 1 -- 172.21.15.76:0/3099292859 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc43c0ab3e0 con 0x7fc434003a20 2024-03-08T06:02:53.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.064+0000 7fc44a7fc640 1 --2- 172.21.15.76:0/3099292859 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc43806b4d0 0x7fc43806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.065+0000 7fc44a7fc640 1 --2- 172.21.15.76:0/3099292859 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc43806b4d0 0x7fc43806d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0703f0 tx=0x7fc440008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.069+0000 7fc4337fe640 1 -- 172.21.15.76:0/3099292859 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc43c04c070 con 0x7fc434003a20 2024-03-08T06:02:53.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.239+0000 7fc45153e640 1 -- 172.21.15.76:0/3099292859 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc44c0ccef0 con 0x7fc43806b4d0 2024-03-08T06:02:53.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.239+0000 7fc4337fe640 1 -- 172.21.15.76:0/3099292859 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc44c0ccef0 con 0x7fc43806b4d0 2024-03-08T06:02:53.240 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 1 to deep-scrub 2024-03-08T06:02:53.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.241+0000 7fc45153e640 1 -- 172.21.15.76:0/3099292859 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc43806b4d0 msgr2=0x7fc43806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.241+0000 7fc45153e640 1 --2- 172.21.15.76:0/3099292859 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc43806b4d0 0x7fc43806d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0703f0 tx=0x7fc440008040 comp rx=0 tx=0).stop 2024-03-08T06:02:53.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.241+0000 7fc45153e640 1 -- 172.21.15.76:0/3099292859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc434003a20 msgr2=0x7fc44c06f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.241+0000 7fc45153e640 1 --2- 172.21.15.76:0/3099292859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc434003a20 0x7fc44c06f210 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc43c037040 tx=0x7fc43c004730 comp rx=0 tx=0).stop 2024-03-08T06:02:53.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.241+0000 7fc45153e640 1 -- 172.21.15.76:0/3099292859 shutdown_connections 2024-03-08T06:02:53.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.241+0000 7fc45153e640 1 --2- 172.21.15.76:0/3099292859 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fc43806b4d0 0x7fc43806d990 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.241+0000 7fc45153e640 1 --2- 172.21.15.76:0/3099292859 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc44c157760 0x7fc44c1597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.241+0000 7fc45153e640 1 --2- 172.21.15.76:0/3099292859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc44c157300 0x7fc44c159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.241+0000 7fc45153e640 1 --2- 172.21.15.76:0/3099292859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc434003a20 0x7fc44c06f210 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.241+0000 7fc45153e640 1 -- 172.21.15.76:0/3099292859 >> 172.21.15.76:0/3099292859 conn(0x7fc44c0d0890 msgr2=0x7fc44c06e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:53.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.242+0000 7fc45153e640 1 -- 172.21.15.76:0/3099292859 shutdown_connections 2024-03-08T06:02:53.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.242+0000 7fc45153e640 1 -- 172.21.15.76:0/3099292859 wait complete. 2024-03-08T06:02:53.253 INFO:tasks.ceph:Scrubbing osd.2 2024-03-08T06:02:53.253 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-08T06:02:53.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.357+0000 7f435b781640 1 Processor -- start 2024-03-08T06:02:53.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.357+0000 7f435b781640 1 -- start start 2024-03-08T06:02:53.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.358+0000 7f435b781640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f435415dd10 0x7f43540ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.358+0000 7f435b781640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f435406fef0 0x7f435406c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.358+0000 7f435b781640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43540cfd00 con 0x7f43540cf800 2024-03-08T06:02:53.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.358+0000 7f435b781640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435415e0f0 con 0x7f435415dd10 2024-03-08T06:02:53.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.358+0000 7f435b781640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43540d02c0 con 0x7f435406fef0 2024-03-08T06:02:53.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.358+0000 7f435951d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f435415dd10 0x7f43540ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.358+0000 7f435951d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f435415dd10 0x7f43540ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48576/0 (socket says 172.21.15.76:48576) 2024-03-08T06:02:53.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.358+0000 7f435951d640 1 -- 172.21.15.76:0/3437727233 learned_addr learned my addr 172.21.15.76:0/3437727233 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:53.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.358+0000 7f4359d1e640 1 --2- 172.21.15.76:0/3437727233 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f435406fef0 0x7f435406c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.358+0000 7f435951d640 1 -- 172.21.15.76:0/3437727233 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f435406fef0 msgr2=0x7f435406c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:53.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.358+0000 7f435951d640 1 --2- 172.21.15.76:0/3437727233 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f435406fef0 0x7f435406c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.358+0000 7f435951d640 1 -- 172.21.15.76:0/3437727233 >> v1:172.21.15.81:6789/0 conn(0x7f43540cf800 legacy=0x7f435406f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:53.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.359+0000 7f435951d640 1 -- 172.21.15.76:0/3437727233 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43540cffe0 con 0x7f435415dd10 2024-03-08T06:02:53.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.359+0000 7f4359d1e640 1 --2- 172.21.15.76:0/3437727233 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f435406fef0 0x7f435406c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:53.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.359+0000 7f435951d640 1 --2- 172.21.15.76:0/3437727233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f435415dd10 0x7f43540ccda0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4350009870 tx=0x7f435002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32be24d3f80de79 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.359+0000 7f434bfff640 1 -- 172.21.15.76:0/3437727233 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350035020 con 0x7f435415dd10 2024-03-08T06:02:53.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.359+0000 7f434bfff640 1 -- 172.21.15.76:0/3437727233 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4350031640 con 0x7f435415dd10 2024-03-08T06:02:53.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.359+0000 7f434bfff640 1 -- 172.21.15.76:0/3437727233 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435002d040 con 0x7f435415dd10 2024-03-08T06:02:53.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.360+0000 7f435b781640 1 -- 172.21.15.76:0/3437727233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f435415dd10 msgr2=0x7f43540ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.360+0000 7f435b781640 1 --2- 172.21.15.76:0/3437727233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f435415dd10 0x7f43540ccda0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4350009870 tx=0x7f435002b8f0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.360+0000 7f435b781640 1 -- 172.21.15.76:0/3437727233 shutdown_connections 2024-03-08T06:02:53.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.360+0000 7f435b781640 1 --2- 172.21.15.76:0/3437727233 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f435406fef0 0x7f435406c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.360+0000 7f435b781640 1 --2- 172.21.15.76:0/3437727233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f435415dd10 0x7f43540ccda0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.360+0000 7f435b781640 1 -- 172.21.15.76:0/3437727233 >> 172.21.15.76:0/3437727233 conn(0x7f435406ae60 msgr2=0x7f435406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:53.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.360+0000 7f435b781640 1 -- 172.21.15.76:0/3437727233 shutdown_connections 2024-03-08T06:02:53.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.360+0000 7f435b781640 1 -- 172.21.15.76:0/3437727233 wait complete. 2024-03-08T06:02:53.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.361+0000 7f435b781640 1 Processor -- start 2024-03-08T06:02:53.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.361+0000 7f435b781640 1 -- start start 2024-03-08T06:02:53.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f435b781640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f435406fef0 0x7f43540c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f435b781640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43540cf800 0x7f43540cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f435b781640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f435415dd10 0x7f4354155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f435b781640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43540cd2e0 con 0x7f435415dd10 2024-03-08T06:02:53.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f435b781640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435406c2a0 con 0x7f435406fef0 2024-03-08T06:02:53.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f435b781640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43540cd460 con 0x7f43540cf800 2024-03-08T06:02:53.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f4359d1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f435415dd10 0x7f4354155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f4359d1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f435415dd10 0x7f4354155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48590/0 (socket says 172.21.15.76:48590) 2024-03-08T06:02:53.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f4359d1e640 1 -- 172.21.15.76:0/2572570565 learned_addr learned my addr 172.21.15.76:0/2572570565 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:53.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f435951d640 1 --2- 172.21.15.76:0/2572570565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f435406fef0 0x7f43540c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f4358d1c640 1 --2- 172.21.15.76:0/2572570565 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43540cf800 0x7f43540cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f4359d1e640 1 -- 172.21.15.76:0/2572570565 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43540cf800 msgr2=0x7f43540cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f4359d1e640 1 --2- 172.21.15.76:0/2572570565 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43540cf800 0x7f43540cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f4359d1e640 1 -- 172.21.15.76:0/2572570565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f435406fef0 msgr2=0x7f43540c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f4359d1e640 1 --2- 172.21.15.76:0/2572570565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f435406fef0 0x7f43540c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f4359d1e640 1 -- 172.21.15.76:0/2572570565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43540ca3d0 con 0x7f435415dd10 2024-03-08T06:02:53.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f435951d640 1 --2- 172.21.15.76:0/2572570565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f435406fef0 0x7f43540c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:53.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f4359d1e640 1 --2- 172.21.15.76:0/2572570565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f435415dd10 0x7f4354155c40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f434c00cef0 tx=0x7f434c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f4349ffb640 1 -- 172.21.15.76:0/2572570565 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c01a070 con 0x7f435415dd10 2024-03-08T06:02:53.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f4349ffb640 1 -- 172.21.15.76:0/2572570565 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434c00d860 con 0x7f435415dd10 2024-03-08T06:02:53.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f435b781640 1 -- 172.21.15.76:0/2572570565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43540ca6b0 con 0x7f435415dd10 2024-03-08T06:02:53.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.362+0000 7f4349ffb640 1 -- 172.21.15.76:0/2572570565 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c00dc50 con 0x7f435415dd10 2024-03-08T06:02:53.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.363+0000 7f435b781640 1 -- 172.21.15.76:0/2572570565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43540cac70 con 0x7f435415dd10 2024-03-08T06:02:53.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.363+0000 7f435b781640 1 -- 172.21.15.76:0/2572570565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f435404f030 con 0x7f435415dd10 2024-03-08T06:02:53.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.364+0000 7f4349ffb640 1 -- 172.21.15.76:0/2572570565 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f434c01d070 con 0x7f435415dd10 2024-03-08T06:02:53.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.364+0000 7f4349ffb640 1 --2- 172.21.15.76:0/2572570565 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f433806b4d0 0x7f433806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.364+0000 7f4349ffb640 1 -- 172.21.15.76:0/2572570565 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f434c08a910 con 0x7f435415dd10 2024-03-08T06:02:53.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.364+0000 7f435951d640 1 --2- 172.21.15.76:0/2572570565 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f433806b4d0 0x7f433806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.364+0000 7f4349ffb640 1 --2- 172.21.15.76:0/2572570565 >> [v2:172.21.15.76:6824/1112475349,v1:172.21.15.76:6825/1112475349] conn(0x7f4338071970 0x7f4338073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.364+0000 7f4349ffb640 1 -- 172.21.15.76:0/2572570565 --> [v2:172.21.15.76:6824/1112475349,v1:172.21.15.76:6825/1112475349] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4338074480 con 0x7f4338071970 2024-03-08T06:02:53.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.364+0000 7f4349ffb640 1 -- 172.21.15.76:0/2572570565 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f434c08f2c0 con 0x7f435415dd10 2024-03-08T06:02:53.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.365+0000 7f4358d1c640 1 --2- 172.21.15.76:0/2572570565 >> [v2:172.21.15.76:6824/1112475349,v1:172.21.15.76:6825/1112475349] conn(0x7f4338071970 0x7f4338073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.365+0000 7f435951d640 1 --2- 172.21.15.76:0/2572570565 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f433806b4d0 0x7f433806d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4350036000 tx=0x7f435003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.365+0000 7f4358d1c640 1 --2- 172.21.15.76:0/2572570565 >> [v2:172.21.15.76:6824/1112475349,v1:172.21.15.76:6825/1112475349] conn(0x7f4338071970 0x7f4338073db0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.366+0000 7f4349ffb640 1 -- 172.21.15.76:0/2572570565 <== osd.2 v2:172.21.15.76:6824/1112475349 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4338074480 con 0x7f4338071970 2024-03-08T06:02:53.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.378+0000 7f435b781640 1 -- 172.21.15.76:0/2572570565 --> [v2:172.21.15.76:6824/1112475349,v1:172.21.15.76:6825/1112475349] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f43540ccb80 con 0x7f4338071970 2024-03-08T06:02:53.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.379+0000 7f4349ffb640 1 -- 172.21.15.76:0/2572570565 <== osd.2 v2:172.21.15.76:6824/1112475349 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f43540ccb80 con 0x7f4338071970 2024-03-08T06:02:53.379 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-08T06:02:53.380 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T06:02:53.380 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-08T06:02:53.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.379+0000 7f435b781640 1 -- 172.21.15.76:0/2572570565 >> [v2:172.21.15.76:6824/1112475349,v1:172.21.15.76:6825/1112475349] conn(0x7f4338071970 msgr2=0x7f4338073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.379+0000 7f435b781640 1 --2- 172.21.15.76:0/2572570565 >> [v2:172.21.15.76:6824/1112475349,v1:172.21.15.76:6825/1112475349] conn(0x7f4338071970 0x7f4338073db0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.379+0000 7f435b781640 1 -- 172.21.15.76:0/2572570565 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f433806b4d0 msgr2=0x7f433806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.379+0000 7f435b781640 1 --2- 172.21.15.76:0/2572570565 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f433806b4d0 0x7f433806d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4350036000 tx=0x7f435003c040 comp rx=0 tx=0).stop 2024-03-08T06:02:53.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.379+0000 7f435b781640 1 -- 172.21.15.76:0/2572570565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f435415dd10 msgr2=0x7f4354155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.379+0000 7f435b781640 1 --2- 172.21.15.76:0/2572570565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f435415dd10 0x7f4354155c40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f434c00cef0 tx=0x7f434c00a710 comp rx=0 tx=0).stop 2024-03-08T06:02:53.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.379+0000 7f4359d1e640 1 -- 172.21.15.76:0/2572570565 reap_dead start 2024-03-08T06:02:53.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.379+0000 7f435b781640 1 -- 172.21.15.76:0/2572570565 shutdown_connections 2024-03-08T06:02:53.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.379+0000 7f435b781640 1 -- 172.21.15.76:0/2572570565 >> 172.21.15.76:0/2572570565 conn(0x7f435406ae60 msgr2=0x7f43540b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:53.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.379+0000 7f435b781640 1 -- 172.21.15.76:0/2572570565 shutdown_connections 2024-03-08T06:02:53.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.379+0000 7f435b781640 1 -- 172.21.15.76:0/2572570565 wait complete. 2024-03-08T06:02:53.390 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-08T06:02:53.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.490+0000 7f06f53e2640 1 Processor -- start 2024-03-08T06:02:53.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.490+0000 7f06f53e2640 1 -- start start 2024-03-08T06:02:53.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.491+0000 7f06f53e2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f06f00b9840 0x7f06f00b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.491+0000 7f06f53e2640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f06f00b5b20 0x7f06f00b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.491+0000 7f06f53e2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f00b5f40 con 0x7f06f015dbc0 2024-03-08T06:02:53.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.491+0000 7f06f53e2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f00d27d0 con 0x7f06f00b9840 2024-03-08T06:02:53.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.491+0000 7f06f53e2640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f00d2ab0 con 0x7f06f00b5b20 2024-03-08T06:02:53.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.491+0000 7f06e67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f06f00b9840 0x7f06f00b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.491+0000 7f06e67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f06f00b9840 0x7f06f00b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48598/0 (socket says 172.21.15.76:48598) 2024-03-08T06:02:53.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.491+0000 7f06e67fc640 1 -- 172.21.15.76:0/1345672437 learned_addr learned my addr 172.21.15.76:0/1345672437 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:53.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.491+0000 7f06e67fc640 1 -- 172.21.15.76:0/1345672437 >> v2:172.21.15.81:3300/0 conn(0x7f06f00b5b20 msgr2=0x7f06f00b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:53.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.491+0000 7f06e67fc640 1 --2- 172.21.15.76:0/1345672437 >> v2:172.21.15.81:3300/0 conn(0x7f06f00b5b20 0x7f06f00b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.491+0000 7f06e67fc640 1 -- 172.21.15.76:0/1345672437 >> v1:172.21.15.81:6789/0 conn(0x7f06f015dbc0 legacy=0x7f06f00d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:53.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.491+0000 7f06e67fc640 1 -- 172.21.15.76:0/1345672437 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f00d2d90 con 0x7f06f00b9840 2024-03-08T06:02:53.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.492+0000 7f06e67fc640 1 --2- 172.21.15.76:0/1345672437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06f00b9840 0x7f06f00b55e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f06f006d410 tx=0x7f06d802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe430be5ce7e485 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.492+0000 7f06ee7fc640 1 -- 172.21.15.76:0/1345672437 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d8035020 con 0x7f06f00b9840 2024-03-08T06:02:53.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.492+0000 7f06ee7fc640 1 -- 172.21.15.76:0/1345672437 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d8031870 con 0x7f06f00b9840 2024-03-08T06:02:53.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.492+0000 7f06ee7fc640 1 -- 172.21.15.76:0/1345672437 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d8031c40 con 0x7f06f00b9840 2024-03-08T06:02:53.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.493+0000 7f06f53e2640 1 -- 172.21.15.76:0/1345672437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06f00b9840 msgr2=0x7f06f00b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.493+0000 7f06f53e2640 1 --2- 172.21.15.76:0/1345672437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06f00b9840 0x7f06f00b55e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f06f006d410 tx=0x7f06d802b910 comp rx=0 tx=0).stop 2024-03-08T06:02:53.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.493+0000 7f06f53e2640 1 -- 172.21.15.76:0/1345672437 shutdown_connections 2024-03-08T06:02:53.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.493+0000 7f06f53e2640 1 --2- 172.21.15.76:0/1345672437 >> v2:172.21.15.81:3300/0 conn(0x7f06f00b5b20 0x7f06f00b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.493+0000 7f06f53e2640 1 --2- 172.21.15.76:0/1345672437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06f00b9840 0x7f06f00b55e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.493+0000 7f06f53e2640 1 -- 172.21.15.76:0/1345672437 >> 172.21.15.76:0/1345672437 conn(0x7f06f00670d0 msgr2=0x7f06f00d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:53.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.493+0000 7f06f53e2640 1 -- 172.21.15.76:0/1345672437 shutdown_connections 2024-03-08T06:02:53.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.493+0000 7f06f53e2640 1 -- 172.21.15.76:0/1345672437 wait complete. 2024-03-08T06:02:53.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.494+0000 7f06f53e2640 1 Processor -- start 2024-03-08T06:02:53.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.494+0000 7f06f53e2640 1 -- start start 2024-03-08T06:02:53.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.495+0000 7f06f53e2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06f00b5b20 0x7f06f01f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.495+0000 7f06f53e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06f00b9840 0x7f06f01496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.495+0000 7f06f53e2640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f06f015dbc0 0x7f06f0149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.495+0000 7f06f53e2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f015dfa0 con 0x7f06f00b9840 2024-03-08T06:02:53.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.495+0000 7f06f53e2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f00d3870 con 0x7f06f00b5b20 2024-03-08T06:02:53.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.495+0000 7f06f53e2640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f015e120 con 0x7f06f015dbc0 2024-03-08T06:02:53.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.495+0000 7f06e67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06f00b9840 0x7f06f01496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.495+0000 7f06e67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06f00b9840 0x7f06f01496e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48612/0 (socket says 172.21.15.76:48612) 2024-03-08T06:02:53.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.495+0000 7f06e67fc640 1 -- 172.21.15.76:0/3557643517 learned_addr learned my addr 172.21.15.76:0/3557643517 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:53.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.495+0000 7f06e67fc640 1 -- 172.21.15.76:0/3557643517 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f06f015dbc0 msgr2=0x7f06f0149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.495+0000 7f06e67fc640 1 --2- 172.21.15.76:0/3557643517 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f06f015dbc0 0x7f06f0149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.495+0000 7f06e67fc640 1 -- 172.21.15.76:0/3557643517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06f00b5b20 msgr2=0x7f06f01f32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.495+0000 7f06eeffd640 1 --2- 172.21.15.76:0/3557643517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06f00b5b20 0x7f06f01f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.495+0000 7f06e67fc640 1 --2- 172.21.15.76:0/3557643517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06f00b5b20 0x7f06f01f32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.495+0000 7f06e67fc640 1 -- 172.21.15.76:0/3557643517 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f0159740 con 0x7f06f00b9840 2024-03-08T06:02:53.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.496+0000 7f06e67fc640 1 --2- 172.21.15.76:0/3557643517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06f00b9840 0x7f06f01496e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f06d8000c00 tx=0x7f06d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.496+0000 7f06e7fff640 1 -- 172.21.15.76:0/3557643517 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d803ad60 con 0x7f06f00b9840 2024-03-08T06:02:53.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.496+0000 7f06e7fff640 1 -- 172.21.15.76:0/3557643517 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d80443f0 con 0x7f06f00b9840 2024-03-08T06:02:53.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.496+0000 7f06e7fff640 1 -- 172.21.15.76:0/3557643517 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d80447e0 con 0x7f06f00b9840 2024-03-08T06:02:53.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.496+0000 7f06f53e2640 1 -- 172.21.15.76:0/3557643517 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f0158b30 con 0x7f06f00b9840 2024-03-08T06:02:53.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.496+0000 7f06f53e2640 1 -- 172.21.15.76:0/3557643517 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f01590f0 con 0x7f06f00b9840 2024-03-08T06:02:53.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.497+0000 7f06f53e2640 1 -- 172.21.15.76:0/3557643517 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f004f030 con 0x7f06f00b9840 2024-03-08T06:02:53.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.497+0000 7f06e7fff640 1 -- 172.21.15.76:0/3557643517 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06d804b060 con 0x7f06f00b9840 2024-03-08T06:02:53.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.498+0000 7f06e7fff640 1 --2- 172.21.15.76:0/3557643517 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f06d006b4d0 0x7f06d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.498+0000 7f06eeffd640 1 --2- 172.21.15.76:0/3557643517 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f06d006b4d0 0x7f06d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.498+0000 7f06e7fff640 1 -- 172.21.15.76:0/3557643517 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f06d80ac360 con 0x7f06f00b9840 2024-03-08T06:02:53.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.498+0000 7f06eeffd640 1 --2- 172.21.15.76:0/3557643517 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f06d006b4d0 0x7f06d006d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f06e000a730 tx=0x7f06e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.502+0000 7f06e7fff640 1 -- 172.21.15.76:0/3557643517 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06d803f070 con 0x7f06f00b9840 2024-03-08T06:02:53.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.671+0000 7f06f53e2640 1 -- 172.21.15.76:0/3557643517 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f06f00cac10 con 0x7f06d006b4d0 2024-03-08T06:02:53.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.671+0000 7f06e7fff640 1 -- 172.21.15.76:0/3557643517 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f06e000a380 con 0x7f06d006b4d0 2024-03-08T06:02:53.672 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 2 to deep-scrub 2024-03-08T06:02:53.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.673+0000 7f06f53e2640 1 -- 172.21.15.76:0/3557643517 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f06d006b4d0 msgr2=0x7f06d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.673+0000 7f06f53e2640 1 --2- 172.21.15.76:0/3557643517 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f06d006b4d0 0x7f06d006d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f06e000a730 tx=0x7f06e0008040 comp rx=0 tx=0).stop 2024-03-08T06:02:53.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.673+0000 7f06f53e2640 1 -- 172.21.15.76:0/3557643517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06f00b9840 msgr2=0x7f06f01496e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.673+0000 7f06f53e2640 1 --2- 172.21.15.76:0/3557643517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06f00b9840 0x7f06f01496e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f06d8000c00 tx=0x7f06d80046c0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.673+0000 7f06f53e2640 1 -- 172.21.15.76:0/3557643517 shutdown_connections 2024-03-08T06:02:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.673+0000 7f06f53e2640 1 --2- 172.21.15.76:0/3557643517 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f06d006b4d0 0x7f06d006d990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.673+0000 7f06f53e2640 1 --2- 172.21.15.76:0/3557643517 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f06f015dbc0 0x7f06f0149c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.673+0000 7f06f53e2640 1 --2- 172.21.15.76:0/3557643517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06f00b9840 0x7f06f01496e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.673+0000 7f06f53e2640 1 --2- 172.21.15.76:0/3557643517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06f00b5b20 0x7f06f01f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.673+0000 7f06f53e2640 1 -- 172.21.15.76:0/3557643517 >> 172.21.15.76:0/3557643517 conn(0x7f06f00670d0 msgr2=0x7f06f00b8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.673+0000 7f06f53e2640 1 -- 172.21.15.76:0/3557643517 shutdown_connections 2024-03-08T06:02:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.673+0000 7f06f53e2640 1 -- 172.21.15.76:0/3557643517 wait complete. 2024-03-08T06:02:53.685 INFO:tasks.ceph:Scrubbing osd.3 2024-03-08T06:02:53.685 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-08T06:02:53.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.790+0000 7fa5f7081640 1 Processor -- start 2024-03-08T06:02:53.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.790+0000 7fa5f7081640 1 -- start start 2024-03-08T06:02:53.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.791+0000 7fa5f7081640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa5f00cf800 0x7fa5f006f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.791+0000 7fa5f7081640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa5f006fd20 0x7fa5f006c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.791+0000 7fa5f7081640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f00cfd00 con 0x7fa5f015dd10 2024-03-08T06:02:53.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.791+0000 7fa5f7081640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f015e0f0 con 0x7fa5f00cf800 2024-03-08T06:02:53.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.791+0000 7fa5f7081640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f00d02c0 con 0x7fa5f006fd20 2024-03-08T06:02:53.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.791+0000 7fa5effff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa5f00cf800 0x7fa5f006f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.791+0000 7fa5effff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa5f00cf800 0x7fa5f006f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48624/0 (socket says 172.21.15.76:48624) 2024-03-08T06:02:53.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.791+0000 7fa5effff640 1 -- 172.21.15.76:0/1605837875 learned_addr learned my addr 172.21.15.76:0/1605837875 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:53.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.791+0000 7fa5f561e640 1 --2- 172.21.15.76:0/1605837875 >> v2:172.21.15.81:3300/0 conn(0x7fa5f006fd20 0x7fa5f006c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.791+0000 7fa5effff640 1 -- 172.21.15.76:0/1605837875 >> v2:172.21.15.81:3300/0 conn(0x7fa5f006fd20 msgr2=0x7fa5f006c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:53.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.791+0000 7fa5effff640 1 --2- 172.21.15.76:0/1605837875 >> v2:172.21.15.81:3300/0 conn(0x7fa5f006fd20 0x7fa5f006c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.791+0000 7fa5effff640 1 -- 172.21.15.76:0/1605837875 >> v1:172.21.15.81:6789/0 conn(0x7fa5f015dd10 legacy=0x7fa5f00ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:53.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.791+0000 7fa5effff640 1 -- 172.21.15.76:0/1605837875 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f00cffe0 con 0x7fa5f00cf800 2024-03-08T06:02:53.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.791+0000 7fa5f561e640 1 --2- 172.21.15.76:0/1605837875 >> v2:172.21.15.81:3300/0 conn(0x7fa5f006fd20 0x7fa5f006c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:53.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.792+0000 7fa5effff640 1 --2- 172.21.15.76:0/1605837875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f00cf800 0x7fa5f006f7e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00d0c70 tx=0x7fa5e002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656e1bc6aa0c4b2a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.792+0000 7fa5ef7fe640 1 -- 172.21.15.76:0/1605837875 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e002ca80 con 0x7fa5f00cf800 2024-03-08T06:02:53.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.792+0000 7fa5ef7fe640 1 -- 172.21.15.76:0/1605837875 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e002cc20 con 0x7fa5f00cf800 2024-03-08T06:02:53.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.792+0000 7fa5ef7fe640 1 -- 172.21.15.76:0/1605837875 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e00343f0 con 0x7fa5f00cf800 2024-03-08T06:02:53.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.793+0000 7fa5f7081640 1 -- 172.21.15.76:0/1605837875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f00cf800 msgr2=0x7fa5f006f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.793+0000 7fa5f7081640 1 --2- 172.21.15.76:0/1605837875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f00cf800 0x7fa5f006f7e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00d0c70 tx=0x7fa5e002bee0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.793+0000 7fa5f7081640 1 -- 172.21.15.76:0/1605837875 shutdown_connections 2024-03-08T06:02:53.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.793+0000 7fa5f7081640 1 --2- 172.21.15.76:0/1605837875 >> v2:172.21.15.81:3300/0 conn(0x7fa5f006fd20 0x7fa5f006c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.793+0000 7fa5f7081640 1 --2- 172.21.15.76:0/1605837875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f00cf800 0x7fa5f006f7e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.793+0000 7fa5f7081640 1 -- 172.21.15.76:0/1605837875 >> 172.21.15.76:0/1605837875 conn(0x7fa5f006ae60 msgr2=0x7fa5f006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:53.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.793+0000 7fa5f7081640 1 -- 172.21.15.76:0/1605837875 shutdown_connections 2024-03-08T06:02:53.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.793+0000 7fa5f7081640 1 -- 172.21.15.76:0/1605837875 wait complete. 2024-03-08T06:02:53.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.794+0000 7fa5f7081640 1 Processor -- start 2024-03-08T06:02:53.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.794+0000 7fa5f7081640 1 -- start start 2024-03-08T06:02:53.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f7081640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f006fd20 0x7fa5f01488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f7081640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa5f00cf800 0x7fa5f00c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f7081640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5f015dd10 0x7fa5f0146ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f7081640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f006cc70 con 0x7fa5f006fd20 2024-03-08T06:02:53.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f7081640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f006c2a0 con 0x7fa5f015dd10 2024-03-08T06:02:53.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f7081640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f006cdf0 con 0x7fa5f00cf800 2024-03-08T06:02:53.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f4e1d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f006fd20 0x7fa5f01488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f4e1d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f006fd20 0x7fa5f01488b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48628/0 (socket says 172.21.15.76:48628) 2024-03-08T06:02:53.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f4e1d640 1 -- 172.21.15.76:0/3296685288 learned_addr learned my addr 172.21.15.76:0/3296685288 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:53.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f561e640 1 --2- 172.21.15.76:0/3296685288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5f015dd10 0x7fa5f0146ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5effff640 1 --2- 172.21.15.76:0/3296685288 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa5f00cf800 0x7fa5f00c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f4e1d640 1 -- 172.21.15.76:0/3296685288 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa5f00cf800 msgr2=0x7fa5f00c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f4e1d640 1 --2- 172.21.15.76:0/3296685288 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa5f00cf800 0x7fa5f00c4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f4e1d640 1 -- 172.21.15.76:0/3296685288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5f015dd10 msgr2=0x7fa5f0146ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f4e1d640 1 --2- 172.21.15.76:0/3296685288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5f015dd10 0x7fa5f0146ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f4e1d640 1 -- 172.21.15.76:0/3296685288 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f0147810 con 0x7fa5f006fd20 2024-03-08T06:02:53.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f4e1d640 1 --2- 172.21.15.76:0/3296685288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f006fd20 0x7fa5f01488b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800b3f0 tx=0x7fa5e800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5ed7fa640 1 -- 172.21.15.76:0/3296685288 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8015020 con 0x7fa5f006fd20 2024-03-08T06:02:53.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5ed7fa640 1 -- 172.21.15.76:0/3296685288 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e8011830 con 0x7fa5f006fd20 2024-03-08T06:02:53.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f7081640 1 -- 172.21.15.76:0/3296685288 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f0147af0 con 0x7fa5f006fd20 2024-03-08T06:02:53.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5ed7fa640 1 -- 172.21.15.76:0/3296685288 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8011c20 con 0x7fa5f006fd20 2024-03-08T06:02:53.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.795+0000 7fa5f7081640 1 -- 172.21.15.76:0/3296685288 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f01480b0 con 0x7fa5f006fd20 2024-03-08T06:02:53.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.796+0000 7fa5f7081640 1 -- 172.21.15.76:0/3296685288 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa5f004f030 con 0x7fa5f006fd20 2024-03-08T06:02:53.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.797+0000 7fa5ed7fa640 1 -- 172.21.15.76:0/3296685288 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5e8018020 con 0x7fa5f006fd20 2024-03-08T06:02:53.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.797+0000 7fa5ed7fa640 1 --2- 172.21.15.76:0/3296685288 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa5d006b500 0x7fa5d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.797+0000 7fa5ed7fa640 1 -- 172.21.15.76:0/3296685288 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa5e8089bd0 con 0x7fa5f006fd20 2024-03-08T06:02:53.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.797+0000 7fa5effff640 1 --2- 172.21.15.76:0/3296685288 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa5d006b500 0x7fa5d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.797+0000 7fa5ed7fa640 1 --2- 172.21.15.76:0/3296685288 >> [v2:172.21.15.76:6808/2501234044,v1:172.21.15.76:6809/2501234044] conn(0x7fa5d00719a0 0x7fa5d0073de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.797+0000 7fa5ed7fa640 1 -- 172.21.15.76:0/3296685288 --> [v2:172.21.15.76:6808/2501234044,v1:172.21.15.76:6809/2501234044] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5e8007e50 con 0x7fa5d00719a0 2024-03-08T06:02:53.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.797+0000 7fa5ed7fa640 1 -- 172.21.15.76:0/3296685288 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5e808f2a0 con 0x7fa5f006fd20 2024-03-08T06:02:53.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.797+0000 7fa5f561e640 1 --2- 172.21.15.76:0/3296685288 >> [v2:172.21.15.76:6808/2501234044,v1:172.21.15.76:6809/2501234044] conn(0x7fa5d00719a0 0x7fa5d0073de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.798+0000 7fa5effff640 1 --2- 172.21.15.76:0/3296685288 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa5d006b500 0x7fa5d006d9c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0035000 tx=0x7fa5e0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.798+0000 7fa5f561e640 1 --2- 172.21.15.76:0/3296685288 >> [v2:172.21.15.76:6808/2501234044,v1:172.21.15.76:6809/2501234044] conn(0x7fa5d00719a0 0x7fa5d0073de0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.798+0000 7fa5ed7fa640 1 -- 172.21.15.76:0/3296685288 <== osd.3 v2:172.21.15.76:6808/2501234044 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa5e8007e50 con 0x7fa5d00719a0 2024-03-08T06:02:53.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.811+0000 7fa5f7081640 1 -- 172.21.15.76:0/3296685288 --> [v2:172.21.15.76:6808/2501234044,v1:172.21.15.76:6809/2501234044] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa5f00ccb80 con 0x7fa5d00719a0 2024-03-08T06:02:53.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.811+0000 7fa5ed7fa640 1 -- 172.21.15.76:0/3296685288 <== osd.3 v2:172.21.15.76:6808/2501234044 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa5f00ccb80 con 0x7fa5d00719a0 2024-03-08T06:02:53.812 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-08T06:02:53.812 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T06:02:53.813 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-08T06:02:53.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.812+0000 7fa5f7081640 1 -- 172.21.15.76:0/3296685288 >> [v2:172.21.15.76:6808/2501234044,v1:172.21.15.76:6809/2501234044] conn(0x7fa5d00719a0 msgr2=0x7fa5d0073de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.812+0000 7fa5f7081640 1 --2- 172.21.15.76:0/3296685288 >> [v2:172.21.15.76:6808/2501234044,v1:172.21.15.76:6809/2501234044] conn(0x7fa5d00719a0 0x7fa5d0073de0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.812+0000 7fa5f7081640 1 -- 172.21.15.76:0/3296685288 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa5d006b500 msgr2=0x7fa5d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.812+0000 7fa5f7081640 1 --2- 172.21.15.76:0/3296685288 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa5d006b500 0x7fa5d006d9c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0035000 tx=0x7fa5e0030000 comp rx=0 tx=0).stop 2024-03-08T06:02:53.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.812+0000 7fa5f7081640 1 -- 172.21.15.76:0/3296685288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f006fd20 msgr2=0x7fa5f01488b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.812+0000 7fa5f7081640 1 --2- 172.21.15.76:0/3296685288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f006fd20 0x7fa5f01488b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800b3f0 tx=0x7fa5e800b8b0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.812+0000 7fa5f561e640 1 -- 172.21.15.76:0/3296685288 reap_dead start 2024-03-08T06:02:53.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.812+0000 7fa5f7081640 1 -- 172.21.15.76:0/3296685288 shutdown_connections 2024-03-08T06:02:53.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.812+0000 7fa5f7081640 1 -- 172.21.15.76:0/3296685288 >> 172.21.15.76:0/3296685288 conn(0x7fa5f006ae60 msgr2=0x7fa5f00ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:53.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.812+0000 7fa5f7081640 1 -- 172.21.15.76:0/3296685288 shutdown_connections 2024-03-08T06:02:53.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.812+0000 7fa5f7081640 1 -- 172.21.15.76:0/3296685288 wait complete. 2024-03-08T06:02:53.822 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-08T06:02:53.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.923+0000 7f12a9143640 1 Processor -- start 2024-03-08T06:02:53.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.923+0000 7f12a9143640 1 -- start start 2024-03-08T06:02:53.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.924+0000 7f12a9143640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f12a40b3c50 0x7f12a40b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.924+0000 7f12a9143640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12a406b0f0 0x7f12a40adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.924+0000 7f12a9143640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a40b4570 con 0x7f12a406ac80 2024-03-08T06:02:53.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.924+0000 7f12a9143640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a406b720 con 0x7f12a40b3c50 2024-03-08T06:02:53.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.924+0000 7f12a9143640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a406ba00 con 0x7f12a406b0f0 2024-03-08T06:02:53.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.924+0000 7f12a2d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f12a40b3c50 0x7f12a40b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.924+0000 7f129bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f12a406ac80 0x7f12a406f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57996/0 (socket says 172.21.15.76:57996) 2024-03-08T06:02:53.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.924+0000 7f129bfff640 1 -- 172.21.15.76:0/894450502 learned_addr learned my addr 172.21.15.76:0/894450502 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:53.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.924+0000 7f12a359e640 1 --2- 172.21.15.76:0/894450502 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12a406b0f0 0x7f12a40adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.924+0000 7f12a2d9d640 1 -- 172.21.15.76:0/894450502 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12a406b0f0 msgr2=0x7f12a40adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:53.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.924+0000 7f12a2d9d640 1 --2- 172.21.15.76:0/894450502 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12a406b0f0 0x7f12a40adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.924+0000 7f12a2d9d640 1 -- 172.21.15.76:0/894450502 >> v1:172.21.15.76:6789/0 conn(0x7f12a406ac80 legacy=0x7f12a406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.924+0000 7f12a2d9d640 1 -- 172.21.15.76:0/894450502 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a406bce0 con 0x7f12a40b3c50 2024-03-08T06:02:53.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.924+0000 7f12a259c640 1 -- 172.21.15.76:0/894450502 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251297802 0 0) 0x7f12a40b4570 con 0x7f12a406ac80 2024-03-08T06:02:53.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.924+0000 7f12a359e640 1 --2- 172.21.15.76:0/894450502 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12a406b0f0 0x7f12a40adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:53.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.925+0000 7f12a2d9d640 1 --2- 172.21.15.76:0/894450502 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12a40b3c50 0x7f12a40b4030 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f12940097d0 tx=0x7f129402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dea0af8e944f750 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.925+0000 7f12a259c640 1 -- 172.21.15.76:0/894450502 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294035020 con 0x7f12a40b3c50 2024-03-08T06:02:53.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.925+0000 7f12a259c640 1 -- 172.21.15.76:0/894450502 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1294031800 con 0x7f12a40b3c50 2024-03-08T06:02:53.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.925+0000 7f12a259c640 1 -- 172.21.15.76:0/894450502 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294031bd0 con 0x7f12a40b3c50 2024-03-08T06:02:53.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.926+0000 7f12a9143640 1 -- 172.21.15.76:0/894450502 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12a40b3c50 msgr2=0x7f12a40b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.926+0000 7f12a9143640 1 --2- 172.21.15.76:0/894450502 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12a40b3c50 0x7f12a40b4030 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f12940097d0 tx=0x7f129402b8a0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.926+0000 7f12a9143640 1 -- 172.21.15.76:0/894450502 shutdown_connections 2024-03-08T06:02:53.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.926+0000 7f12a9143640 1 --2- 172.21.15.76:0/894450502 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12a406b0f0 0x7f12a40adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.926+0000 7f12a9143640 1 --2- 172.21.15.76:0/894450502 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12a40b3c50 0x7f12a40b4030 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.926+0000 7f12a9143640 1 -- 172.21.15.76:0/894450502 >> 172.21.15.76:0/894450502 conn(0x7f12a406dd40 msgr2=0x7f12a406e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:53.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.926+0000 7f12a9143640 1 -- 172.21.15.76:0/894450502 shutdown_connections 2024-03-08T06:02:53.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.926+0000 7f12a9143640 1 -- 172.21.15.76:0/894450502 wait complete. 2024-03-08T06:02:53.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.927+0000 7f12a9143640 1 Processor -- start 2024-03-08T06:02:53.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.927+0000 7f12a9143640 1 -- start start 2024-03-08T06:02:53.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f12a9143640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a406ac80 0x7f12a415faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f12a9143640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12a406b0f0 0x7f12a41552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f12a9143640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12a40b3c50 0x7f12a4152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f12a9143640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a4070500 con 0x7f12a406b0f0 2024-03-08T06:02:53.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f12a9143640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a406c2a0 con 0x7f12a406ac80 2024-03-08T06:02:53.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f12a9143640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a41612c0 con 0x7f12a40b3c50 2024-03-08T06:02:53.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f129bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12a406b0f0 0x7f12a41552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f129bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12a406b0f0 0x7f12a41552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48638/0 (socket says 172.21.15.76:48638) 2024-03-08T06:02:53.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f129bfff640 1 -- 172.21.15.76:0/4189249499 learned_addr learned my addr 172.21.15.76:0/4189249499 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:53.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f12a359e640 1 --2- 172.21.15.76:0/4189249499 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12a40b3c50 0x7f12a4152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f12a2d9d640 1 --2- 172.21.15.76:0/4189249499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a406ac80 0x7f12a415faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f129bfff640 1 -- 172.21.15.76:0/4189249499 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12a40b3c50 msgr2=0x7f12a4152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f129bfff640 1 --2- 172.21.15.76:0/4189249499 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12a40b3c50 0x7f12a4152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f129bfff640 1 -- 172.21.15.76:0/4189249499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a406ac80 msgr2=0x7f12a415faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:53.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f129bfff640 1 --2- 172.21.15.76:0/4189249499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a406ac80 0x7f12a415faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:53.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f129bfff640 1 -- 172.21.15.76:0/4189249499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a415ea00 con 0x7f12a406b0f0 2024-03-08T06:02:53.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f12a359e640 1 --2- 172.21.15.76:0/4189249499 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12a40b3c50 0x7f12a4152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:53.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f12a2d9d640 1 --2- 172.21.15.76:0/4189249499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a406ac80 0x7f12a415faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:53.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f129bfff640 1 --2- 172.21.15.76:0/4189249499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12a406b0f0 0x7f12a41552a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f128c00c500 tx=0x7f128c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f129b7fe640 1 -- 172.21.15.76:0/4189249499 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c018070 con 0x7f12a406b0f0 2024-03-08T06:02:53.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f129b7fe640 1 -- 172.21.15.76:0/4189249499 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128c007580 con 0x7f12a406b0f0 2024-03-08T06:02:53.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f12a9143640 1 -- 172.21.15.76:0/4189249499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a415ece0 con 0x7f12a406b0f0 2024-03-08T06:02:53.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f129b7fe640 1 -- 172.21.15.76:0/4189249499 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c004a20 con 0x7f12a406b0f0 2024-03-08T06:02:53.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.928+0000 7f12a9143640 1 -- 172.21.15.76:0/4189249499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a415f2a0 con 0x7f12a406b0f0 2024-03-08T06:02:53.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.929+0000 7f12a9143640 1 -- 172.21.15.76:0/4189249499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a404f030 con 0x7f12a406b0f0 2024-03-08T06:02:53.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.930+0000 7f129b7fe640 1 -- 172.21.15.76:0/4189249499 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f128c01b070 con 0x7f12a406b0f0 2024-03-08T06:02:53.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.930+0000 7f129b7fe640 1 --2- 172.21.15.76:0/4189249499 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f128406b4d0 0x7f128406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:53.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.930+0000 7f129b7fe640 1 -- 172.21.15.76:0/4189249499 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f128c089350 con 0x7f12a406b0f0 2024-03-08T06:02:53.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.930+0000 7f12a2d9d640 1 --2- 172.21.15.76:0/4189249499 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f128406b4d0 0x7f128406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:53.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.931+0000 7f12a2d9d640 1 --2- 172.21.15.76:0/4189249499 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f128406b4d0 0x7f128406d990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1294036000 tx=0x7f1294008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:53.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:53.935+0000 7f129b7fe640 1 -- 172.21.15.76:0/4189249499 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f128c0583d0 con 0x7f12a406b0f0 2024-03-08T06:02:54.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.106+0000 7f12a9143640 1 -- 172.21.15.76:0/4189249499 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f12a406f9e0 con 0x7f128406b4d0 2024-03-08T06:02:54.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.106+0000 7f129b7fe640 1 -- 172.21.15.76:0/4189249499 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f12a406f9e0 con 0x7f128406b4d0 2024-03-08T06:02:54.107 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 3 to deep-scrub 2024-03-08T06:02:54.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.109+0000 7f12a9143640 1 -- 172.21.15.76:0/4189249499 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f128406b4d0 msgr2=0x7f128406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.109+0000 7f12a9143640 1 --2- 172.21.15.76:0/4189249499 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f128406b4d0 0x7f128406d990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1294036000 tx=0x7f1294008030 comp rx=0 tx=0).stop 2024-03-08T06:02:54.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.109+0000 7f12a9143640 1 -- 172.21.15.76:0/4189249499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12a406b0f0 msgr2=0x7f12a41552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.109+0000 7f12a9143640 1 --2- 172.21.15.76:0/4189249499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12a406b0f0 0x7f12a41552a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f128c00c500 tx=0x7f128c00c9c0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.109+0000 7f12a9143640 1 -- 172.21.15.76:0/4189249499 shutdown_connections 2024-03-08T06:02:54.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.109+0000 7f12a9143640 1 --2- 172.21.15.76:0/4189249499 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f128406b4d0 0x7f128406d990 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.109+0000 7f12a9143640 1 --2- 172.21.15.76:0/4189249499 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f12a40b3c50 0x7f12a4152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.109+0000 7f12a9143640 1 --2- 172.21.15.76:0/4189249499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12a406b0f0 0x7f12a41552a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.109+0000 7f12a9143640 1 --2- 172.21.15.76:0/4189249499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a406ac80 0x7f12a415faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.109+0000 7f12a9143640 1 -- 172.21.15.76:0/4189249499 >> 172.21.15.76:0/4189249499 conn(0x7f12a406dd40 msgr2=0x7f12a40b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:54.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.109+0000 7f12a9143640 1 -- 172.21.15.76:0/4189249499 shutdown_connections 2024-03-08T06:02:54.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.109+0000 7f12a9143640 1 -- 172.21.15.76:0/4189249499 wait complete. 2024-03-08T06:02:54.120 INFO:tasks.ceph:Scrubbing osd.4 2024-03-08T06:02:54.120 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-08T06:02:54.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.225+0000 7ffbe95e9640 1 Processor -- start 2024-03-08T06:02:54.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.225+0000 7ffbe95e9640 1 -- start start 2024-03-08T06:02:54.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.226+0000 7ffbe95e9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffbe4154d90 0x7ffbe40b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.226+0000 7ffbe95e9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ffbe406eaa0 0x7ffbe406ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.226+0000 7ffbe95e9640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe415e740 con 0x7ffbe4155170 2024-03-08T06:02:54.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.226+0000 7ffbe95e9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe406f450 con 0x7ffbe4154d90 2024-03-08T06:02:54.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.226+0000 7ffbe95e9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe40b2d60 con 0x7ffbe406eaa0 2024-03-08T06:02:54.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.226+0000 7ffbe2ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffbe4154d90 0x7ffbe40b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.226+0000 7ffbe2ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffbe4154d90 0x7ffbe40b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48652/0 (socket says 172.21.15.76:48652) 2024-03-08T06:02:54.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.226+0000 7ffbe2ffd640 1 -- 172.21.15.76:0/1235058872 learned_addr learned my addr 172.21.15.76:0/1235058872 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:54.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.226+0000 7ffbe2ffd640 1 -- 172.21.15.76:0/1235058872 >> v2:172.21.15.81:3300/0 conn(0x7ffbe406eaa0 msgr2=0x7ffbe406ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:54.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.226+0000 7ffbe2ffd640 1 --2- 172.21.15.76:0/1235058872 >> v2:172.21.15.81:3300/0 conn(0x7ffbe406eaa0 0x7ffbe406ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.226+0000 7ffbe2ffd640 1 -- 172.21.15.76:0/1235058872 >> v1:172.21.15.81:6789/0 conn(0x7ffbe4155170 legacy=0x7ffbe415af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:54.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.226+0000 7ffbe2ffd640 1 -- 172.21.15.76:0/1235058872 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe415ba90 con 0x7ffbe4154d90 2024-03-08T06:02:54.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.227+0000 7ffbe2ffd640 1 --2- 172.21.15.76:0/1235058872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbe4154d90 0x7ffbe40b31f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4009e50 tx=0x7ffbd402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba9b85af86791fc server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:54.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.227+0000 7ffbe1ffb640 1 -- 172.21.15.76:0/1235058872 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd402ca80 con 0x7ffbe4154d90 2024-03-08T06:02:54.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.227+0000 7ffbe1ffb640 1 -- 172.21.15.76:0/1235058872 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd402cc20 con 0x7ffbe4154d90 2024-03-08T06:02:54.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.227+0000 7ffbe1ffb640 1 -- 172.21.15.76:0/1235058872 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd40343f0 con 0x7ffbe4154d90 2024-03-08T06:02:54.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.228+0000 7ffbe95e9640 1 -- 172.21.15.76:0/1235058872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbe4154d90 msgr2=0x7ffbe40b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.228+0000 7ffbe95e9640 1 --2- 172.21.15.76:0/1235058872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbe4154d90 0x7ffbe40b31f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4009e50 tx=0x7ffbd402bee0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.228+0000 7ffbe95e9640 1 -- 172.21.15.76:0/1235058872 shutdown_connections 2024-03-08T06:02:54.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.228+0000 7ffbe95e9640 1 --2- 172.21.15.76:0/1235058872 >> v2:172.21.15.81:3300/0 conn(0x7ffbe406eaa0 0x7ffbe406ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.228+0000 7ffbe95e9640 1 --2- 172.21.15.76:0/1235058872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbe4154d90 0x7ffbe40b31f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.228+0000 7ffbe95e9640 1 -- 172.21.15.76:0/1235058872 >> 172.21.15.76:0/1235058872 conn(0x7ffbe406ac80 msgr2=0x7ffbe406b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:54.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.228+0000 7ffbe95e9640 1 -- 172.21.15.76:0/1235058872 shutdown_connections 2024-03-08T06:02:54.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.228+0000 7ffbe95e9640 1 -- 172.21.15.76:0/1235058872 wait complete. 2024-03-08T06:02:54.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.229+0000 7ffbe95e9640 1 Processor -- start 2024-03-08T06:02:54.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.229+0000 7ffbe95e9640 1 -- start start 2024-03-08T06:02:54.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.230+0000 7ffbe95e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbe406eaa0 0x7ffbe41f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.230+0000 7ffbe95e9640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffbe4154d90 0x7ffbe40cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.230+0000 7ffbe95e9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbe4155170 0x7ffbe40b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.230+0000 7ffbe95e9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe40b29f0 con 0x7ffbe4155170 2024-03-08T06:02:54.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.230+0000 7ffbe95e9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe40b2710 con 0x7ffbe406eaa0 2024-03-08T06:02:54.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.230+0000 7ffbe95e9640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe40b3730 con 0x7ffbe4154d90 2024-03-08T06:02:54.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.230+0000 7ffbe37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbe4155170 0x7ffbe40b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.230+0000 7ffbe37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbe4155170 0x7ffbe40b1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48662/0 (socket says 172.21.15.76:48662) 2024-03-08T06:02:54.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.230+0000 7ffbe37fe640 1 -- 172.21.15.76:0/2998281574 learned_addr learned my addr 172.21.15.76:0/2998281574 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:54.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.230+0000 7ffbe2ffd640 1 --2- 172.21.15.76:0/2998281574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbe406eaa0 0x7ffbe41f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.230+0000 7ffbe27fc640 1 --2- 172.21.15.76:0/2998281574 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffbe4154d90 0x7ffbe40cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.230+0000 7ffbe37fe640 1 -- 172.21.15.76:0/2998281574 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffbe4154d90 msgr2=0x7ffbe40cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.230+0000 7ffbe37fe640 1 --2- 172.21.15.76:0/2998281574 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ffbe4154d90 0x7ffbe40cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.230+0000 7ffbe37fe640 1 -- 172.21.15.76:0/2998281574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbe406eaa0 msgr2=0x7ffbe41f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.230+0000 7ffbe37fe640 1 --2- 172.21.15.76:0/2998281574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbe406eaa0 0x7ffbe41f1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.230+0000 7ffbe37fe640 1 -- 172.21.15.76:0/2998281574 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe41e4380 con 0x7ffbe4155170 2024-03-08T06:02:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.231+0000 7ffbe37fe640 1 --2- 172.21.15.76:0/2998281574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbe4155170 0x7ffbe40b1b00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009fd0 tx=0x7ffbdc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.231+0000 7ffbcb7fe640 1 -- 172.21.15.76:0/2998281574 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc011830 con 0x7ffbe4155170 2024-03-08T06:02:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.231+0000 7ffbcb7fe640 1 -- 172.21.15.76:0/2998281574 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbdc016070 con 0x7ffbe4155170 2024-03-08T06:02:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.231+0000 7ffbe95e9640 1 -- 172.21.15.76:0/2998281574 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe41e4660 con 0x7ffbe4155170 2024-03-08T06:02:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.231+0000 7ffbcb7fe640 1 -- 172.21.15.76:0/2998281574 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc011bf0 con 0x7ffbe4155170 2024-03-08T06:02:54.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.231+0000 7ffbe95e9640 1 -- 172.21.15.76:0/2998281574 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe41e4c20 con 0x7ffbe4155170 2024-03-08T06:02:54.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.232+0000 7ffbe95e9640 1 -- 172.21.15.76:0/2998281574 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffbe41e5000 con 0x7ffbe4155170 2024-03-08T06:02:54.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.232+0000 7ffbcb7fe640 1 -- 172.21.15.76:0/2998281574 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffbdc01b460 con 0x7ffbe4155170 2024-03-08T06:02:54.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.233+0000 7ffbcb7fe640 1 --2- 172.21.15.76:0/2998281574 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffbd006b4d0 0x7ffbd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.233+0000 7ffbcb7fe640 1 -- 172.21.15.76:0/2998281574 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ffbdc08a7e0 con 0x7ffbe4155170 2024-03-08T06:02:54.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.233+0000 7ffbe2ffd640 1 --2- 172.21.15.76:0/2998281574 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffbd006b4d0 0x7ffbd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.233+0000 7ffbcb7fe640 1 --2- 172.21.15.76:0/2998281574 >> [v2:172.21.15.81:6800/3228626670,v1:172.21.15.81:6801/3228626670] conn(0x7ffbd0071970 0x7ffbd0073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.233+0000 7ffbcb7fe640 1 -- 172.21.15.76:0/2998281574 --> [v2:172.21.15.81:6800/3228626670,v1:172.21.15.81:6801/3228626670] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffbd0074480 con 0x7ffbd0071970 2024-03-08T06:02:54.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.233+0000 7ffbcb7fe640 1 -- 172.21.15.76:0/2998281574 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbdc08ac70 con 0x7ffbe4155170 2024-03-08T06:02:54.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.233+0000 7ffbe2ffd640 1 --2- 172.21.15.76:0/2998281574 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffbd006b4d0 0x7ffbd006d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4035000 tx=0x7ffbd4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:54.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.233+0000 7ffbe27fc640 1 --2- 172.21.15.76:0/2998281574 >> [v2:172.21.15.81:6800/3228626670,v1:172.21.15.81:6801/3228626670] conn(0x7ffbd0071970 0x7ffbd0073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.233+0000 7ffbe27fc640 1 --2- 172.21.15.76:0/2998281574 >> [v2:172.21.15.81:6800/3228626670,v1:172.21.15.81:6801/3228626670] conn(0x7ffbd0071970 0x7ffbd0073db0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:54.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.234+0000 7ffbcb7fe640 1 -- 172.21.15.76:0/2998281574 <== osd.4 v2:172.21.15.81:6800/3228626670 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ffbd0074480 con 0x7ffbd0071970 2024-03-08T06:02:54.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.247+0000 7ffbe95e9640 1 -- 172.21.15.76:0/2998281574 --> [v2:172.21.15.81:6800/3228626670,v1:172.21.15.81:6801/3228626670] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ffbe404f030 con 0x7ffbd0071970 2024-03-08T06:02:54.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.248+0000 7ffbcb7fe640 1 -- 172.21.15.76:0/2998281574 <== osd.4 v2:172.21.15.81:6800/3228626670 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ffbe404f030 con 0x7ffbd0071970 2024-03-08T06:02:54.248 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-08T06:02:54.248 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T06:02:54.249 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-08T06:02:54.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.248+0000 7ffbe95e9640 1 -- 172.21.15.76:0/2998281574 >> [v2:172.21.15.81:6800/3228626670,v1:172.21.15.81:6801/3228626670] conn(0x7ffbd0071970 msgr2=0x7ffbd0073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.248+0000 7ffbe95e9640 1 --2- 172.21.15.76:0/2998281574 >> [v2:172.21.15.81:6800/3228626670,v1:172.21.15.81:6801/3228626670] conn(0x7ffbd0071970 0x7ffbd0073db0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.248+0000 7ffbe95e9640 1 -- 172.21.15.76:0/2998281574 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffbd006b4d0 msgr2=0x7ffbd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.248+0000 7ffbe95e9640 1 --2- 172.21.15.76:0/2998281574 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7ffbd006b4d0 0x7ffbd006d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4035000 tx=0x7ffbd4030000 comp rx=0 tx=0).stop 2024-03-08T06:02:54.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.248+0000 7ffbe95e9640 1 -- 172.21.15.76:0/2998281574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbe4155170 msgr2=0x7ffbe40b1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.248+0000 7ffbe95e9640 1 --2- 172.21.15.76:0/2998281574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbe4155170 0x7ffbe40b1b00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009fd0 tx=0x7ffbdc00b8e0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.248+0000 7ffbe37fe640 1 -- 172.21.15.76:0/2998281574 reap_dead start 2024-03-08T06:02:54.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.248+0000 7ffbe95e9640 1 -- 172.21.15.76:0/2998281574 shutdown_connections 2024-03-08T06:02:54.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.248+0000 7ffbe95e9640 1 -- 172.21.15.76:0/2998281574 >> 172.21.15.76:0/2998281574 conn(0x7ffbe406ac80 msgr2=0x7ffbe4071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:54.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.248+0000 7ffbe95e9640 1 -- 172.21.15.76:0/2998281574 shutdown_connections 2024-03-08T06:02:54.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.248+0000 7ffbe95e9640 1 -- 172.21.15.76:0/2998281574 wait complete. 2024-03-08T06:02:54.259 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-08T06:02:54.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.358+0000 7fb8dbb43640 1 Processor -- start 2024-03-08T06:02:54.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.359+0000 7fb8dbb43640 1 -- start start 2024-03-08T06:02:54.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.359+0000 7fb8dbb43640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb8d4154df0 0x7fb8d40b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.359+0000 7fb8dbb43640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb8d40cc7e0 0x7fb8d415af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.359+0000 7fb8dbb43640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d415f450 con 0x7fb8d40b3640 2024-03-08T06:02:54.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8dbb43640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d40af1e0 con 0x7fb8d4154df0 2024-03-08T06:02:54.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8dbb43640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40b3c80 con 0x7fb8d40cc7e0 2024-03-08T06:02:54.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d90de640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb8d40b3640 0x7fb8d40cc0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:58010/0 (socket says 172.21.15.76:58010) 2024-03-08T06:02:54.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d90de640 1 -- 172.21.15.76:0/953589491 learned_addr learned my addr 172.21.15.76:0/953589491 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:54.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d88dd640 1 -- 172.21.15.76:0/953589491 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334741879 0 0) 0x7fb8d415f450 con 0x7fb8d40b3640 2024-03-08T06:02:54.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d98df640 1 --2- 172.21.15.76:0/953589491 >> v2:172.21.15.81:3300/0 conn(0x7fb8d4154df0 0x7fb8d40b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d88dd640 1 -- 172.21.15.76:0/953589491 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8c0003520 con 0x7fb8d40b3640 2024-03-08T06:02:54.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8da0e0640 1 --2- 172.21.15.76:0/953589491 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb8d40cc7e0 0x7fb8d415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d88dd640 1 -- 172.21.15.76:0/953589491 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2148521237 0 0) 0x7fb8c0003520 con 0x7fb8d40b3640 2024-03-08T06:02:54.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d88dd640 1 -- 172.21.15.76:0/953589491 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb8d40cc7e0 msgr2=0x7fb8d415af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:54.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d88dd640 1 --2- 172.21.15.76:0/953589491 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb8d40cc7e0 0x7fb8d415af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d88dd640 1 -- 172.21.15.76:0/953589491 >> v2:172.21.15.81:3300/0 conn(0x7fb8d4154df0 msgr2=0x7fb8d40b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:54.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d88dd640 1 --2- 172.21.15.76:0/953589491 >> v2:172.21.15.81:3300/0 conn(0x7fb8d4154df0 0x7fb8d40b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d88dd640 1 -- 172.21.15.76:0/953589491 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d415e690 con 0x7fb8d40b3640 2024-03-08T06:02:54.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d88dd640 1 -- 172.21.15.76:0/953589491 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fb8c4001670 con 0x7fb8d40b3640 2024-03-08T06:02:54.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d88dd640 1 -- 172.21.15.76:0/953589491 >> v1:172.21.15.76:6789/0 conn(0x7fb8d40b3640 legacy=0x7fb8d40cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d88dd640 1 --2- 172.21.15.76:0/953589491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c0003a20 0x7fb8c0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d88dd640 1 --2- 172.21.15.76:0/953589491 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb8d40cc7e0 0x7fb8c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d88dd640 1 --2- 172.21.15.76:0/953589491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8c0004d40 0x7fb8c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d88dd640 1 -- 172.21.15.76:0/953589491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40af1e0 con 0x7fb8c0003a20 2024-03-08T06:02:54.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d88dd640 1 -- 172.21.15.76:0/953589491 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40b3c80 con 0x7fb8c0004d40 2024-03-08T06:02:54.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d88dd640 1 -- 172.21.15.76:0/953589491 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c000ba40 con 0x7fb8d40cc7e0 2024-03-08T06:02:54.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d98df640 1 --2- 172.21.15.76:0/953589491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c0003a20 0x7fb8c0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8da0e0640 1 --2- 172.21.15.76:0/953589491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8c0004d40 0x7fb8c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.360+0000 7fb8d90de640 1 --2- 172.21.15.76:0/953589491 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb8d40cc7e0 0x7fb8c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.361+0000 7fb8d98df640 1 -- 172.21.15.76:0/953589491 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb8d40cc7e0 msgr2=0x7fb8c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:54.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.361+0000 7fb8d98df640 1 --2- 172.21.15.76:0/953589491 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb8d40cc7e0 0x7fb8c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.361+0000 7fb8d98df640 1 -- 172.21.15.76:0/953589491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8c0004d40 msgr2=0x7fb8c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:54.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.361+0000 7fb8d98df640 1 --2- 172.21.15.76:0/953589491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8c0004d40 0x7fb8c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.361+0000 7fb8d98df640 1 -- 172.21.15.76:0/953589491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c0003520 con 0x7fb8c0003a20 2024-03-08T06:02:54.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.361+0000 7fb8d90de640 1 --2- 172.21.15.76:0/953589491 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb8d40cc7e0 0x7fb8c0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:54.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.361+0000 7fb8d98df640 1 --2- 172.21.15.76:0/953589491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c0003a20 0x7fb8c0003e40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000bfc0 tx=0x7fb8c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4edc8787a7d6ae server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:54.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.361+0000 7fb8d88dd640 1 -- 172.21.15.76:0/953589491 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d003b070 con 0x7fb8c0003a20 2024-03-08T06:02:54.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.361+0000 7fb8d88dd640 1 -- 172.21.15.76:0/953589491 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d002e890 con 0x7fb8c0003a20 2024-03-08T06:02:54.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.361+0000 7fb8d88dd640 1 -- 172.21.15.76:0/953589491 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d000bc10 con 0x7fb8c0003a20 2024-03-08T06:02:54.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.362+0000 7fb8dbb43640 1 -- 172.21.15.76:0/953589491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c0003a20 msgr2=0x7fb8c0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.362+0000 7fb8dbb43640 1 --2- 172.21.15.76:0/953589491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c0003a20 0x7fb8c0003e40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000bfc0 tx=0x7fb8c0005830 comp rx=0 tx=0).stop 2024-03-08T06:02:54.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.362+0000 7fb8da0e0640 1 -- 172.21.15.76:0/953589491 reap_dead start 2024-03-08T06:02:54.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.362+0000 7fb8dbb43640 1 -- 172.21.15.76:0/953589491 shutdown_connections 2024-03-08T06:02:54.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.362+0000 7fb8dbb43640 1 -- 172.21.15.76:0/953589491 >> 172.21.15.76:0/953589491 conn(0x7fb8d406ac80 msgr2=0x7fb8d406b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:54.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.362+0000 7fb8dbb43640 1 -- 172.21.15.76:0/953589491 shutdown_connections 2024-03-08T06:02:54.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.362+0000 7fb8dbb43640 1 -- 172.21.15.76:0/953589491 wait complete. 2024-03-08T06:02:54.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.363+0000 7fb8dbb43640 1 Processor -- start 2024-03-08T06:02:54.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.363+0000 7fb8dbb43640 1 -- start start 2024-03-08T06:02:54.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8dbb43640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8c0003a20 0x7fb8d40a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8dbb43640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb8d4150c70 0x7fb8d4151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8dbb43640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8d41510d0 0x7fb8d4151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8dbb43640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d415ef30 con 0x7fb8d41510d0 2024-03-08T06:02:54.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8dbb43640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d415ec50 con 0x7fb8c0003a20 2024-03-08T06:02:54.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8dbb43640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d41551d0 con 0x7fb8d4150c70 2024-03-08T06:02:54.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8da0e0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8d41510d0 0x7fb8d4151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8da0e0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8d41510d0 0x7fb8d4151ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48674/0 (socket says 172.21.15.76:48674) 2024-03-08T06:02:54.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8da0e0640 1 -- 172.21.15.76:0/1693086504 learned_addr learned my addr 172.21.15.76:0/1693086504 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:54.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8d98df640 1 --2- 172.21.15.76:0/1693086504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8c0003a20 0x7fb8d40a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8d90de640 1 --2- 172.21.15.76:0/1693086504 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb8d4150c70 0x7fb8d4151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8da0e0640 1 -- 172.21.15.76:0/1693086504 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb8d4150c70 msgr2=0x7fb8d4151990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8da0e0640 1 --2- 172.21.15.76:0/1693086504 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb8d4150c70 0x7fb8d4151990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8da0e0640 1 -- 172.21.15.76:0/1693086504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8c0003a20 msgr2=0x7fb8d40a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8da0e0640 1 --2- 172.21.15.76:0/1693086504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8c0003a20 0x7fb8d40a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8da0e0640 1 -- 172.21.15.76:0/1693086504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d40a86d0 con 0x7fb8d41510d0 2024-03-08T06:02:54.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8d98df640 1 --2- 172.21.15.76:0/1693086504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8c0003a20 0x7fb8d40a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:54.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.364+0000 7fb8d90de640 1 --2- 172.21.15.76:0/1693086504 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb8d4150c70 0x7fb8d4151990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:54.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.365+0000 7fb8da0e0640 1 --2- 172.21.15.76:0/1693086504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8d41510d0 0x7fb8d4151ed0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb8d406bae0 tx=0x7fb8cc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:54.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.365+0000 7fb8ca7fc640 1 -- 172.21.15.76:0/1693086504 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc00fe10 con 0x7fb8d41510d0 2024-03-08T06:02:54.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.365+0000 7fb8ca7fc640 1 -- 172.21.15.76:0/1693086504 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8cc01d070 con 0x7fb8d41510d0 2024-03-08T06:02:54.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.365+0000 7fb8dbb43640 1 -- 172.21.15.76:0/1693086504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d40a89b0 con 0x7fb8d41510d0 2024-03-08T06:02:54.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.365+0000 7fb8ca7fc640 1 -- 172.21.15.76:0/1693086504 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc015670 con 0x7fb8d41510d0 2024-03-08T06:02:54.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.365+0000 7fb8dbb43640 1 -- 172.21.15.76:0/1693086504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d40a8f70 con 0x7fb8d41510d0 2024-03-08T06:02:54.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.366+0000 7fb8dbb43640 1 -- 172.21.15.76:0/1693086504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d40a9170 con 0x7fb8d41510d0 2024-03-08T06:02:54.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.366+0000 7fb8ca7fc640 1 -- 172.21.15.76:0/1693086504 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8cc015810 con 0x7fb8d41510d0 2024-03-08T06:02:54.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.366+0000 7fb8ca7fc640 1 --2- 172.21.15.76:0/1693086504 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb8b806b4d0 0x7fb8b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.367+0000 7fb8ca7fc640 1 -- 172.21.15.76:0/1693086504 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb8cc08aa70 con 0x7fb8d41510d0 2024-03-08T06:02:54.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.367+0000 7fb8d98df640 1 --2- 172.21.15.76:0/1693086504 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb8b806b4d0 0x7fb8b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.367+0000 7fb8d98df640 1 --2- 172.21.15.76:0/1693086504 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb8b806b4d0 0x7fb8b806d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0038000 tx=0x7fb8d0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:54.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.371+0000 7fb8ca7fc640 1 -- 172.21.15.76:0/1693086504 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8d40a9170 con 0x7fb8d41510d0 2024-03-08T06:02:54.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.540+0000 7fb8dbb43640 1 -- 172.21.15.76:0/1693086504 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8d404f030 con 0x7fb8b806b4d0 2024-03-08T06:02:54.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.540+0000 7fb8ca7fc640 1 -- 172.21.15.76:0/1693086504 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb8d404f030 con 0x7fb8b806b4d0 2024-03-08T06:02:54.541 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 4 to deep-scrub 2024-03-08T06:02:54.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.542+0000 7fb8dbb43640 1 -- 172.21.15.76:0/1693086504 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb8b806b4d0 msgr2=0x7fb8b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.542+0000 7fb8dbb43640 1 --2- 172.21.15.76:0/1693086504 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb8b806b4d0 0x7fb8b806d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0038000 tx=0x7fb8d0030000 comp rx=0 tx=0).stop 2024-03-08T06:02:54.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.542+0000 7fb8dbb43640 1 -- 172.21.15.76:0/1693086504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8d41510d0 msgr2=0x7fb8d4151ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.542+0000 7fb8dbb43640 1 --2- 172.21.15.76:0/1693086504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8d41510d0 0x7fb8d4151ed0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb8d406bae0 tx=0x7fb8cc00f580 comp rx=0 tx=0).stop 2024-03-08T06:02:54.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.542+0000 7fb8dbb43640 1 -- 172.21.15.76:0/1693086504 shutdown_connections 2024-03-08T06:02:54.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.542+0000 7fb8dbb43640 1 --2- 172.21.15.76:0/1693086504 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb8b806b4d0 0x7fb8b806d990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.542+0000 7fb8dbb43640 1 --2- 172.21.15.76:0/1693086504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8d41510d0 0x7fb8d4151ed0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.542+0000 7fb8dbb43640 1 --2- 172.21.15.76:0/1693086504 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb8d4150c70 0x7fb8d4151990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.542+0000 7fb8dbb43640 1 --2- 172.21.15.76:0/1693086504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8c0003a20 0x7fb8d40a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.542+0000 7fb8dbb43640 1 -- 172.21.15.76:0/1693086504 >> 172.21.15.76:0/1693086504 conn(0x7fb8d406ac80 msgr2=0x7fb8d40719b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:54.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.542+0000 7fb8dbb43640 1 -- 172.21.15.76:0/1693086504 shutdown_connections 2024-03-08T06:02:54.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.542+0000 7fb8dbb43640 1 -- 172.21.15.76:0/1693086504 wait complete. 2024-03-08T06:02:54.553 INFO:tasks.ceph:Scrubbing osd.5 2024-03-08T06:02:54.553 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-08T06:02:54.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.656+0000 7fa4a413e640 1 Processor -- start 2024-03-08T06:02:54.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.656+0000 7fa4a413e640 1 -- start start 2024-03-08T06:02:54.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.657+0000 7fa4a413e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa49c0b3510 0x7fa49c0adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.657+0000 7fa4a413e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49c0ace50 con 0x7fa49c15da30 2024-03-08T06:02:54.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.657+0000 7fa4a413e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49c066ea0 con 0x7fa49c0b30d0 2024-03-08T06:02:54.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.657+0000 7fa4a413e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0ad440 con 0x7fa49c0b3510 2024-03-08T06:02:54.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.657+0000 7fa4a1eda640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa49c15da30 0x7fa49c06b690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:58022/0 (socket says 172.21.15.76:58022) 2024-03-08T06:02:54.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.657+0000 7fa4a1eda640 1 -- 172.21.15.76:0/2300283228 learned_addr learned my addr 172.21.15.76:0/2300283228 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:54.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.657+0000 7fa4a26db640 1 --2- 172.21.15.76:0/2300283228 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa49c0b3510 0x7fa49c0adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.657+0000 7fa4a0ed8640 1 -- 172.21.15.76:0/2300283228 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106401727 0 0) 0x7fa48c001110 con 0x7fa49c15da30 2024-03-08T06:02:54.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.657+0000 7fa4a0ed8640 1 -- 172.21.15.76:0/2300283228 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa488003520 con 0x7fa49c15da30 2024-03-08T06:02:54.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a0ed8640 1 -- 172.21.15.76:0/2300283228 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 322051576 0 0) 0x7fa488003520 con 0x7fa49c15da30 2024-03-08T06:02:54.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a0ed8640 1 -- 172.21.15.76:0/2300283228 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa49c0b3510 msgr2=0x7fa49c0adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:54.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a0ed8640 1 --2- 172.21.15.76:0/2300283228 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa49c0b3510 0x7fa49c0adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a0ed8640 1 -- 172.21.15.76:0/2300283228 >> v1:172.21.15.81:6789/0 conn(0x7fa49c0b30d0 legacy=0x7fa49c0ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a0ed8640 1 -- 172.21.15.76:0/2300283228 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c06c150 con 0x7fa49c15da30 2024-03-08T06:02:54.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a0ed8640 1 -- 172.21.15.76:0/2300283228 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fa48c001840 con 0x7fa49c15da30 2024-03-08T06:02:54.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a0ed8640 1 -- 172.21.15.76:0/2300283228 >> v1:172.21.15.76:6789/0 conn(0x7fa49c15da30 legacy=0x7fa49c06b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a0ed8640 1 --2- 172.21.15.76:0/2300283228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa4880039e0 0x7fa488003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a0ed8640 1 --2- 172.21.15.76:0/2300283228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa488004390 0x7fa488004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a0ed8640 1 --2- 172.21.15.76:0/2300283228 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa49c0b3510 0x7fa4880050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a0ed8640 1 -- 172.21.15.76:0/2300283228 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0ad440 con 0x7fa488004390 2024-03-08T06:02:54.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a0ed8640 1 -- 172.21.15.76:0/2300283228 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48800b9c0 con 0x7fa4880039e0 2024-03-08T06:02:54.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a0ed8640 1 -- 172.21.15.76:0/2300283228 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48800bbc0 con 0x7fa49c0b3510 2024-03-08T06:02:54.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a16d9640 1 --2- 172.21.15.76:0/2300283228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa488004390 0x7fa488004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a1eda640 1 --2- 172.21.15.76:0/2300283228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa4880039e0 0x7fa488003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a26db640 1 --2- 172.21.15.76:0/2300283228 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa49c0b3510 0x7fa4880050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a16d9640 1 -- 172.21.15.76:0/2300283228 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa49c0b3510 msgr2=0x7fa4880050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:54.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a16d9640 1 --2- 172.21.15.76:0/2300283228 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa49c0b3510 0x7fa4880050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a16d9640 1 -- 172.21.15.76:0/2300283228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa4880039e0 msgr2=0x7fa488003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:54.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a16d9640 1 --2- 172.21.15.76:0/2300283228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa4880039e0 0x7fa488003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.658+0000 7fa4a16d9640 1 -- 172.21.15.76:0/2300283228 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa488003520 con 0x7fa488004390 2024-03-08T06:02:54.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.659+0000 7fa4a16d9640 1 --2- 172.21.15.76:0/2300283228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa488004390 0x7fa488004860 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa498001860 tx=0x7fa488005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4256dd402054ff25 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:54.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.659+0000 7fa4a0ed8640 1 -- 172.21.15.76:0/2300283228 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498038070 con 0x7fa488004390 2024-03-08T06:02:54.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.659+0000 7fa4a0ed8640 1 -- 172.21.15.76:0/2300283228 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa498033410 con 0x7fa488004390 2024-03-08T06:02:54.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.659+0000 7fa4a0ed8640 1 -- 172.21.15.76:0/2300283228 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498001430 con 0x7fa488004390 2024-03-08T06:02:54.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.659+0000 7fa4a413e640 1 -- 172.21.15.76:0/2300283228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa488004390 msgr2=0x7fa488004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.659+0000 7fa4a413e640 1 --2- 172.21.15.76:0/2300283228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa488004390 0x7fa488004860 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa498001860 tx=0x7fa488005830 comp rx=0 tx=0).stop 2024-03-08T06:02:54.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.659+0000 7fa4a26db640 1 -- 172.21.15.76:0/2300283228 reap_dead start 2024-03-08T06:02:54.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.660+0000 7fa4a413e640 1 -- 172.21.15.76:0/2300283228 shutdown_connections 2024-03-08T06:02:54.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.660+0000 7fa4a413e640 1 -- 172.21.15.76:0/2300283228 >> 172.21.15.76:0/2300283228 conn(0x7fa49c0cfed0 msgr2=0x7fa49c0d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:54.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.660+0000 7fa4a413e640 1 -- 172.21.15.76:0/2300283228 shutdown_connections 2024-03-08T06:02:54.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.660+0000 7fa4a413e640 1 -- 172.21.15.76:0/2300283228 wait complete. 2024-03-08T06:02:54.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.661+0000 7fa4a413e640 1 Processor -- start 2024-03-08T06:02:54.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.661+0000 7fa4a413e640 1 -- start start 2024-03-08T06:02:54.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.661+0000 7fa4a413e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa488004390 0x7fa49c15f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.661+0000 7fa4a413e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa49c156c90 0x7fa49c1499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.661+0000 7fa4a413e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa49c15a470 0x7fa49c15a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.661+0000 7fa4a413e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0b3960 con 0x7fa49c156c90 2024-03-08T06:02:54.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.661+0000 7fa4a413e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0ad0d0 con 0x7fa49c15a470 2024-03-08T06:02:54.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.661+0000 7fa4a413e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c06bda0 con 0x7fa488004390 2024-03-08T06:02:54.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.662+0000 7fa4a16d9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa49c156c90 0x7fa49c1499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.662+0000 7fa4a16d9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa49c156c90 0x7fa49c1499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48686/0 (socket says 172.21.15.76:48686) 2024-03-08T06:02:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.662+0000 7fa4a16d9640 1 -- 172.21.15.76:0/2788918217 learned_addr learned my addr 172.21.15.76:0/2788918217 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.662+0000 7fa4a1eda640 1 --2- 172.21.15.76:0/2788918217 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa488004390 0x7fa49c15f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.662+0000 7fa4a26db640 1 --2- 172.21.15.76:0/2788918217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa49c15a470 0x7fa49c15a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.662+0000 7fa4a16d9640 1 -- 172.21.15.76:0/2788918217 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa488004390 msgr2=0x7fa49c15f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.662+0000 7fa4a16d9640 1 --2- 172.21.15.76:0/2788918217 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa488004390 0x7fa49c15f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.662+0000 7fa4a16d9640 1 -- 172.21.15.76:0/2788918217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa49c15a470 msgr2=0x7fa49c15a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.662+0000 7fa4a16d9640 1 --2- 172.21.15.76:0/2788918217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa49c15a470 0x7fa49c15a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.662+0000 7fa4a16d9640 1 -- 172.21.15.76:0/2788918217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c149f30 con 0x7fa49c156c90 2024-03-08T06:02:54.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.662+0000 7fa4a16d9640 1 --2- 172.21.15.76:0/2788918217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa49c156c90 0x7fa49c1499f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa498001980 tx=0x7fa498001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:54.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.662+0000 7fa4927fc640 1 -- 172.21.15.76:0/2788918217 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498041020 con 0x7fa49c156c90 2024-03-08T06:02:54.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.662+0000 7fa4927fc640 1 -- 172.21.15.76:0/2788918217 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa498006e00 con 0x7fa49c156c90 2024-03-08T06:02:54.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.662+0000 7fa4a413e640 1 -- 172.21.15.76:0/2788918217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c159fb0 con 0x7fa49c156c90 2024-03-08T06:02:54.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.662+0000 7fa4927fc640 1 -- 172.21.15.76:0/2788918217 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498038040 con 0x7fa49c156c90 2024-03-08T06:02:54.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.662+0000 7fa4a413e640 1 -- 172.21.15.76:0/2788918217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c15feb0 con 0x7fa49c156c90 2024-03-08T06:02:54.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.663+0000 7fa4a413e640 1 -- 172.21.15.76:0/2788918217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa49c04f030 con 0x7fa49c156c90 2024-03-08T06:02:54.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.664+0000 7fa4927fc640 1 -- 172.21.15.76:0/2788918217 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa498044020 con 0x7fa49c156c90 2024-03-08T06:02:54.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.664+0000 7fa4927fc640 1 --2- 172.21.15.76:0/2788918217 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa48006b4d0 0x7fa48006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.664+0000 7fa4927fc640 1 -- 172.21.15.76:0/2788918217 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa4980ab0a0 con 0x7fa49c156c90 2024-03-08T06:02:54.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.664+0000 7fa4a1eda640 1 --2- 172.21.15.76:0/2788918217 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa48006b4d0 0x7fa48006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.664+0000 7fa4927fc640 1 --2- 172.21.15.76:0/2788918217 >> [v2:172.21.15.81:6808/173715776,v1:172.21.15.81:6809/173715776] conn(0x7fa480071970 0x7fa480073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.664+0000 7fa4927fc640 1 -- 172.21.15.76:0/2788918217 --> [v2:172.21.15.81:6808/173715776,v1:172.21.15.81:6809/173715776] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa498043e50 con 0x7fa480071970 2024-03-08T06:02:54.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.664+0000 7fa4927fc640 1 -- 172.21.15.76:0/2788918217 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4980b2050 con 0x7fa49c156c90 2024-03-08T06:02:54.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.664+0000 7fa4a1eda640 1 --2- 172.21.15.76:0/2788918217 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa48006b4d0 0x7fa48006d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa48c000f60 tx=0x7fa48c000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:54.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.664+0000 7fa4a26db640 1 --2- 172.21.15.76:0/2788918217 >> [v2:172.21.15.81:6808/173715776,v1:172.21.15.81:6809/173715776] conn(0x7fa480071970 0x7fa480073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.665+0000 7fa4a26db640 1 --2- 172.21.15.76:0/2788918217 >> [v2:172.21.15.81:6808/173715776,v1:172.21.15.81:6809/173715776] conn(0x7fa480071970 0x7fa480073db0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:54.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.666+0000 7fa4927fc640 1 -- 172.21.15.76:0/2788918217 <== osd.5 v2:172.21.15.81:6808/173715776 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa498043e50 con 0x7fa480071970 2024-03-08T06:02:54.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.679+0000 7fa4a413e640 1 -- 172.21.15.76:0/2788918217 --> [v2:172.21.15.81:6808/173715776,v1:172.21.15.81:6809/173715776] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa49c0cbfb0 con 0x7fa480071970 2024-03-08T06:02:54.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.679+0000 7fa4927fc640 1 -- 172.21.15.76:0/2788918217 <== osd.5 v2:172.21.15.81:6808/173715776 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa49c0cbfb0 con 0x7fa480071970 2024-03-08T06:02:54.680 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-08T06:02:54.680 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T06:02:54.680 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-08T06:02:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.679+0000 7fa4a413e640 1 -- 172.21.15.76:0/2788918217 >> [v2:172.21.15.81:6808/173715776,v1:172.21.15.81:6809/173715776] conn(0x7fa480071970 msgr2=0x7fa480073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.679+0000 7fa4a413e640 1 --2- 172.21.15.76:0/2788918217 >> [v2:172.21.15.81:6808/173715776,v1:172.21.15.81:6809/173715776] conn(0x7fa480071970 0x7fa480073db0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.679+0000 7fa4a413e640 1 -- 172.21.15.76:0/2788918217 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa48006b4d0 msgr2=0x7fa48006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.679+0000 7fa4a413e640 1 --2- 172.21.15.76:0/2788918217 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fa48006b4d0 0x7fa48006d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa48c000f60 tx=0x7fa48c000ed0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.679+0000 7fa4a413e640 1 -- 172.21.15.76:0/2788918217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa49c156c90 msgr2=0x7fa49c1499f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.679+0000 7fa4a413e640 1 --2- 172.21.15.76:0/2788918217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa49c156c90 0x7fa49c1499f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa498001980 tx=0x7fa498001230 comp rx=0 tx=0).stop 2024-03-08T06:02:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.679+0000 7fa4a26db640 1 -- 172.21.15.76:0/2788918217 reap_dead start 2024-03-08T06:02:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.679+0000 7fa4a413e640 1 -- 172.21.15.76:0/2788918217 shutdown_connections 2024-03-08T06:02:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.679+0000 7fa4a413e640 1 -- 172.21.15.76:0/2788918217 >> 172.21.15.76:0/2788918217 conn(0x7fa49c0cfed0 msgr2=0x7fa49c0ad7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.679+0000 7fa4a413e640 1 -- 172.21.15.76:0/2788918217 shutdown_connections 2024-03-08T06:02:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.679+0000 7fa4a413e640 1 -- 172.21.15.76:0/2788918217 wait complete. 2024-03-08T06:02:54.690 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-08T06:02:54.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.791+0000 7fb5d392c640 1 Processor -- start 2024-03-08T06:02:54.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.791+0000 7fb5d392c640 1 -- start start 2024-03-08T06:02:54.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.792+0000 7fb5d392c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb5cc0ce5e0 0x7fb5cc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.792+0000 7fb5d392c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb5cc0ce9c0 0x7fb5cc162020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.792+0000 7fb5d392c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5cc161840 con 0x7fb5cc162560 2024-03-08T06:02:54.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.792+0000 7fb5d392c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5cc161b90 con 0x7fb5cc0ce9c0 2024-03-08T06:02:54.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.792+0000 7fb5d392c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5cc15eb30 con 0x7fb5cc0ce5e0 2024-03-08T06:02:54.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.792+0000 7fb5d0ec7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb5cc0ce9c0 0x7fb5cc162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.792+0000 7fb5d1ec9640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb5cc162560 0x7fb5cc15dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:58028/0 (socket says 172.21.15.76:58028) 2024-03-08T06:02:54.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.792+0000 7fb5d1ec9640 1 -- 172.21.15.76:0/2915794107 learned_addr learned my addr 172.21.15.76:0/2915794107 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:54.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.793+0000 7fb5d16c8640 1 --2- 172.21.15.76:0/2915794107 >> v2:172.21.15.81:3300/0 conn(0x7fb5cc0ce5e0 0x7fb5cc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.793+0000 7fb5d0ec7640 1 -- 172.21.15.76:0/2915794107 >> v2:172.21.15.81:3300/0 conn(0x7fb5cc0ce5e0 msgr2=0x7fb5cc06d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:54.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.793+0000 7fb5d0ec7640 1 --2- 172.21.15.76:0/2915794107 >> v2:172.21.15.81:3300/0 conn(0x7fb5cc0ce5e0 0x7fb5cc06d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.793+0000 7fb5d0ec7640 1 -- 172.21.15.76:0/2915794107 >> v1:172.21.15.76:6789/0 conn(0x7fb5cc162560 legacy=0x7fb5cc15dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.793+0000 7fb5d0ec7640 1 -- 172.21.15.76:0/2915794107 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc15ee10 con 0x7fb5cc0ce9c0 2024-03-08T06:02:54.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.793+0000 7fb5d0ec7640 1 --2- 172.21.15.76:0/2915794107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc0ce9c0 0x7fb5cc162020 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0b3e90 tx=0x7fb5bc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8839555837410ab6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:54.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.793+0000 7fb5c3fff640 1 -- 172.21.15.76:0/2915794107 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc034070 con 0x7fb5cc0ce9c0 2024-03-08T06:02:54.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.794+0000 7fb5c3fff640 1 -- 172.21.15.76:0/2915794107 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc030610 con 0x7fb5cc0ce9c0 2024-03-08T06:02:54.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.794+0000 7fb5c3fff640 1 -- 172.21.15.76:0/2915794107 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc0309e0 con 0x7fb5cc0ce9c0 2024-03-08T06:02:54.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.794+0000 7fb5d392c640 1 -- 172.21.15.76:0/2915794107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc0ce9c0 msgr2=0x7fb5cc162020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.794+0000 7fb5d392c640 1 --2- 172.21.15.76:0/2915794107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc0ce9c0 0x7fb5cc162020 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0b3e90 tx=0x7fb5bc02b960 comp rx=0 tx=0).stop 2024-03-08T06:02:54.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.794+0000 7fb5d392c640 1 -- 172.21.15.76:0/2915794107 shutdown_connections 2024-03-08T06:02:54.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.794+0000 7fb5d392c640 1 --2- 172.21.15.76:0/2915794107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc0ce9c0 0x7fb5cc162020 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.794+0000 7fb5d392c640 1 --2- 172.21.15.76:0/2915794107 >> v2:172.21.15.81:3300/0 conn(0x7fb5cc0ce5e0 0x7fb5cc06d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.794+0000 7fb5d392c640 1 -- 172.21.15.76:0/2915794107 >> 172.21.15.76:0/2915794107 conn(0x7fb5cc06ae60 msgr2=0x7fb5cc06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:54.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.794+0000 7fb5d392c640 1 -- 172.21.15.76:0/2915794107 shutdown_connections 2024-03-08T06:02:54.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.794+0000 7fb5d392c640 1 -- 172.21.15.76:0/2915794107 wait complete. 2024-03-08T06:02:54.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.795+0000 7fb5d392c640 1 Processor -- start 2024-03-08T06:02:54.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.795+0000 7fb5d392c640 1 -- start start 2024-03-08T06:02:54.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.796+0000 7fb5d392c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc0ce5e0 0x7fb5cc0b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.796+0000 7fb5d392c640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb5cc0ce9c0 0x7fb5cc1f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.796+0000 7fb5d392c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5cc162560 0x7fb5cc14e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.796+0000 7fb5d392c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc15e6c0 con 0x7fb5cc0ce5e0 2024-03-08T06:02:54.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.796+0000 7fb5d392c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc0cee20 con 0x7fb5cc162560 2024-03-08T06:02:54.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.796+0000 7fb5d392c640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc15e840 con 0x7fb5cc0ce9c0 2024-03-08T06:02:54.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.796+0000 7fb5d16c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc0ce5e0 0x7fb5cc0b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.796+0000 7fb5d16c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc0ce5e0 0x7fb5cc0b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48716/0 (socket says 172.21.15.76:48716) 2024-03-08T06:02:54.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.796+0000 7fb5d16c8640 1 -- 172.21.15.76:0/1255051807 learned_addr learned my addr 172.21.15.76:0/1255051807 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:54.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.796+0000 7fb5d0ec7640 1 --2- 172.21.15.76:0/1255051807 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb5cc0ce9c0 0x7fb5cc1f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.796+0000 7fb5d1ec9640 1 --2- 172.21.15.76:0/1255051807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5cc162560 0x7fb5cc14e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.796+0000 7fb5d16c8640 1 -- 172.21.15.76:0/1255051807 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb5cc0ce9c0 msgr2=0x7fb5cc1f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.796+0000 7fb5d16c8640 1 --2- 172.21.15.76:0/1255051807 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb5cc0ce9c0 0x7fb5cc1f77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.796+0000 7fb5d16c8640 1 -- 172.21.15.76:0/1255051807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5cc162560 msgr2=0x7fb5cc14e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.796+0000 7fb5d16c8640 1 --2- 172.21.15.76:0/1255051807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5cc162560 0x7fb5cc14e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.796+0000 7fb5d16c8640 1 -- 172.21.15.76:0/1255051807 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc1f8290 con 0x7fb5cc0ce5e0 2024-03-08T06:02:54.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.797+0000 7fb5d16c8640 1 --2- 172.21.15.76:0/1255051807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc0ce5e0 0x7fb5cc0b27a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800ccd0 tx=0x7fb5c800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:54.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.797+0000 7fb5c1ffb640 1 -- 172.21.15.76:0/1255051807 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8003990 con 0x7fb5cc0ce5e0 2024-03-08T06:02:54.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.797+0000 7fb5c1ffb640 1 -- 172.21.15.76:0/1255051807 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c8003b30 con 0x7fb5cc0ce5e0 2024-03-08T06:02:54.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.797+0000 7fb5d392c640 1 -- 172.21.15.76:0/1255051807 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5cc1f8570 con 0x7fb5cc0ce5e0 2024-03-08T06:02:54.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.797+0000 7fb5c1ffb640 1 -- 172.21.15.76:0/1255051807 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8004020 con 0x7fb5cc0ce5e0 2024-03-08T06:02:54.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.797+0000 7fb5d392c640 1 -- 172.21.15.76:0/1255051807 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5cc1e84f0 con 0x7fb5cc0ce5e0 2024-03-08T06:02:54.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.798+0000 7fb5d392c640 1 -- 172.21.15.76:0/1255051807 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5cc04f030 con 0x7fb5cc0ce5e0 2024-03-08T06:02:54.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.798+0000 7fb5c1ffb640 1 -- 172.21.15.76:0/1255051807 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5c80041c0 con 0x7fb5cc0ce5e0 2024-03-08T06:02:54.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.798+0000 7fb5c1ffb640 1 --2- 172.21.15.76:0/1255051807 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb5b006b4d0 0x7fb5b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:54.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.799+0000 7fb5c1ffb640 1 -- 172.21.15.76:0/1255051807 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb5c808a470 con 0x7fb5cc0ce5e0 2024-03-08T06:02:54.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.799+0000 7fb5d0ec7640 1 --2- 172.21.15.76:0/1255051807 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb5b006b4d0 0x7fb5b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:54.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.799+0000 7fb5d0ec7640 1 --2- 172.21.15.76:0/1255051807 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb5b006b4d0 0x7fb5b006d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc036000 tx=0x7fb5bc03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:54.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.803+0000 7fb5c1ffb640 1 -- 172.21.15.76:0/1255051807 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5c8014030 con 0x7fb5cc0ce5e0 2024-03-08T06:02:54.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.974+0000 7fb5d392c640 1 -- 172.21.15.76:0/1255051807 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5cc0d07a0 con 0x7fb5b006b4d0 2024-03-08T06:02:54.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.975+0000 7fb5c1ffb640 1 -- 172.21.15.76:0/1255051807 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb5cc0d07a0 con 0x7fb5b006b4d0 2024-03-08T06:02:54.975 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 5 to deep-scrub 2024-03-08T06:02:54.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.977+0000 7fb5d392c640 1 -- 172.21.15.76:0/1255051807 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb5b006b4d0 msgr2=0x7fb5b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.977+0000 7fb5d392c640 1 --2- 172.21.15.76:0/1255051807 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb5b006b4d0 0x7fb5b006d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc036000 tx=0x7fb5bc03b040 comp rx=0 tx=0).stop 2024-03-08T06:02:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.977+0000 7fb5d392c640 1 -- 172.21.15.76:0/1255051807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc0ce5e0 msgr2=0x7fb5cc0b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.977+0000 7fb5d392c640 1 --2- 172.21.15.76:0/1255051807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc0ce5e0 0x7fb5cc0b27a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800ccd0 tx=0x7fb5c800a5c0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.977+0000 7fb5d392c640 1 -- 172.21.15.76:0/1255051807 shutdown_connections 2024-03-08T06:02:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.977+0000 7fb5d392c640 1 --2- 172.21.15.76:0/1255051807 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fb5b006b4d0 0x7fb5b006d990 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.977+0000 7fb5d392c640 1 --2- 172.21.15.76:0/1255051807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5cc162560 0x7fb5cc14e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.977+0000 7fb5d392c640 1 --2- 172.21.15.76:0/1255051807 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb5cc0ce9c0 0x7fb5cc1f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.977+0000 7fb5d392c640 1 --2- 172.21.15.76:0/1255051807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc0ce5e0 0x7fb5cc0b27a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:54.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.977+0000 7fb5d392c640 1 -- 172.21.15.76:0/1255051807 >> 172.21.15.76:0/1255051807 conn(0x7fb5cc06ae60 msgr2=0x7fb5cc160940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:54.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.977+0000 7fb5d392c640 1 -- 172.21.15.76:0/1255051807 shutdown_connections 2024-03-08T06:02:54.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:54.977+0000 7fb5d392c640 1 -- 172.21.15.76:0/1255051807 wait complete. 2024-03-08T06:02:54.988 INFO:tasks.ceph:Scrubbing osd.6 2024-03-08T06:02:54.988 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-08T06:02:55.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.092+0000 7fab0c62e640 1 Processor -- start 2024-03-08T06:02:55.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.092+0000 7fab0c62e640 1 -- start start 2024-03-08T06:02:55.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.092+0000 7fab0c62e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fab0415db40 0x7fab041622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.093+0000 7fab0c62e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab0415df20 0x7fab040d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.093+0000 7fab0c62e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab041627f0 con 0x7fab040d0a40 2024-03-08T06:02:55.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.093+0000 7fab0c62e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab040d0ea0 con 0x7fab0415df20 2024-03-08T06:02:55.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.093+0000 7fab0c62e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab040cf020 con 0x7fab0415db40 2024-03-08T06:02:55.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.093+0000 7fab09bc9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab0415df20 0x7fab040d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.093+0000 7fab09bc9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab0415df20 0x7fab040d0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48718/0 (socket says 172.21.15.76:48718) 2024-03-08T06:02:55.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.093+0000 7fab09bc9640 1 -- 172.21.15.76:0/746820536 learned_addr learned my addr 172.21.15.76:0/746820536 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:55.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.093+0000 7fab0a3ca640 1 --2- 172.21.15.76:0/746820536 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fab0415db40 0x7fab041622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.093+0000 7fab09bc9640 1 -- 172.21.15.76:0/746820536 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fab0415db40 msgr2=0x7fab041622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:55.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.093+0000 7fab09bc9640 1 --2- 172.21.15.76:0/746820536 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fab0415db40 0x7fab041622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.093+0000 7fab09bc9640 1 -- 172.21.15.76:0/746820536 >> v1:172.21.15.81:6789/0 conn(0x7fab040d0a40 legacy=0x7fab0406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.093+0000 7fab09bc9640 1 -- 172.21.15.76:0/746820536 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab040cf300 con 0x7fab0415df20 2024-03-08T06:02:55.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.093+0000 7fab0a3ca640 1 --2- 172.21.15.76:0/746820536 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fab0415db40 0x7fab041622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:55.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.094+0000 7fab09bc9640 1 --2- 172.21.15.76:0/746820536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab0415df20 0x7fab040d0500 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fab040d1020 tx=0x7faaf402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b559a6134e4117 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.094+0000 7fab093c8640 1 -- 172.21.15.76:0/746820536 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf4034070 con 0x7fab0415df20 2024-03-08T06:02:55.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.094+0000 7fab093c8640 1 -- 172.21.15.76:0/746820536 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf4030840 con 0x7fab0415df20 2024-03-08T06:02:55.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.094+0000 7fab093c8640 1 -- 172.21.15.76:0/746820536 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf4030c10 con 0x7fab0415df20 2024-03-08T06:02:55.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.094+0000 7fab0c62e640 1 -- 172.21.15.76:0/746820536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab0415df20 msgr2=0x7fab040d0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.094+0000 7fab0c62e640 1 --2- 172.21.15.76:0/746820536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab0415df20 0x7fab040d0500 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fab040d1020 tx=0x7faaf402b980 comp rx=0 tx=0).stop 2024-03-08T06:02:55.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.094+0000 7fab0c62e640 1 -- 172.21.15.76:0/746820536 shutdown_connections 2024-03-08T06:02:55.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.095+0000 7fab0c62e640 1 --2- 172.21.15.76:0/746820536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab0415df20 0x7fab040d0500 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.095+0000 7fab0c62e640 1 --2- 172.21.15.76:0/746820536 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fab0415db40 0x7fab041622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.095+0000 7fab0c62e640 1 -- 172.21.15.76:0/746820536 >> 172.21.15.76:0/746820536 conn(0x7fab04066fc0 msgr2=0x7fab040b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:55.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.095+0000 7fab0c62e640 1 -- 172.21.15.76:0/746820536 shutdown_connections 2024-03-08T06:02:55.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.095+0000 7fab0c62e640 1 -- 172.21.15.76:0/746820536 wait complete. 2024-03-08T06:02:55.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.096+0000 7fab0c62e640 1 Processor -- start 2024-03-08T06:02:55.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.096+0000 7fab0c62e640 1 -- start start 2024-03-08T06:02:55.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.096+0000 7fab0c62e640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fab040d0a40 0x7fab04154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.096+0000 7fab0c62e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab0415db40 0x7fab04155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.096+0000 7fab0c62e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab0415df20 0x7fab041562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.096+0000 7fab0c62e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab040d23d0 con 0x7fab0415df20 2024-03-08T06:02:55.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.096+0000 7fab0c62e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab040d2170 con 0x7fab0415db40 2024-03-08T06:02:55.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.096+0000 7fab0c62e640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab040b6e10 con 0x7fab040d0a40 2024-03-08T06:02:55.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.096+0000 7fab0abcb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab0415df20 0x7fab041562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7fab0abcb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab0415df20 0x7fab041562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48720/0 (socket says 172.21.15.76:48720) 2024-03-08T06:02:55.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7fab0abcb640 1 -- 172.21.15.76:0/365274496 learned_addr learned my addr 172.21.15.76:0/365274496 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:55.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7fab0a3ca640 1 --2- 172.21.15.76:0/365274496 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fab040d0a40 0x7fab04154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7fab09bc9640 1 --2- 172.21.15.76:0/365274496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab0415db40 0x7fab04155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7fab0abcb640 1 -- 172.21.15.76:0/365274496 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fab040d0a40 msgr2=0x7fab04154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7fab0abcb640 1 --2- 172.21.15.76:0/365274496 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fab040d0a40 0x7fab04154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7fab0abcb640 1 -- 172.21.15.76:0/365274496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab0415db40 msgr2=0x7fab04155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7fab0abcb640 1 --2- 172.21.15.76:0/365274496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab0415db40 0x7fab04155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7fab0abcb640 1 -- 172.21.15.76:0/365274496 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab040c7d90 con 0x7fab0415df20 2024-03-08T06:02:55.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7fab0a3ca640 1 --2- 172.21.15.76:0/365274496 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fab040d0a40 0x7fab04154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:55.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7fab09bc9640 1 --2- 172.21.15.76:0/365274496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab0415db40 0x7fab04155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:02:55.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7fab0abcb640 1 --2- 172.21.15.76:0/365274496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab0415df20 0x7fab041562b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7faafc00c7a0 tx=0x7faafc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7faafaffd640 1 -- 172.21.15.76:0/365274496 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc003b70 con 0x7fab0415df20 2024-03-08T06:02:55.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7faafaffd640 1 -- 172.21.15.76:0/365274496 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faafc003d30 con 0x7fab0415df20 2024-03-08T06:02:55.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7fab0c62e640 1 -- 172.21.15.76:0/365274496 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab040c8030 con 0x7fab0415df20 2024-03-08T06:02:55.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7faafaffd640 1 -- 172.21.15.76:0/365274496 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc00aa90 con 0x7fab0415df20 2024-03-08T06:02:55.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.097+0000 7fab0c62e640 1 -- 172.21.15.76:0/365274496 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab040c85f0 con 0x7fab0415df20 2024-03-08T06:02:55.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.098+0000 7fab0c62e640 1 -- 172.21.15.76:0/365274496 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab0404f030 con 0x7fab0415df20 2024-03-08T06:02:55.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.099+0000 7faafaffd640 1 -- 172.21.15.76:0/365274496 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faafc00ac30 con 0x7fab0415df20 2024-03-08T06:02:55.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.099+0000 7faafaffd640 1 --2- 172.21.15.76:0/365274496 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faae806b500 0x7faae806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.099+0000 7faafaffd640 1 -- 172.21.15.76:0/365274496 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7faafc089fc0 con 0x7fab0415df20 2024-03-08T06:02:55.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.099+0000 7fab0a3ca640 1 --2- 172.21.15.76:0/365274496 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faae806b500 0x7faae806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.099+0000 7faafaffd640 1 --2- 172.21.15.76:0/365274496 >> [v2:172.21.15.81:6824/3040851989,v1:172.21.15.81:6825/3040851989] conn(0x7faae80719a0 0x7faae8073de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.099+0000 7faafaffd640 1 -- 172.21.15.76:0/365274496 --> [v2:172.21.15.81:6824/3040851989,v1:172.21.15.81:6825/3040851989] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faafc010e50 con 0x7faae80719a0 2024-03-08T06:02:55.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.099+0000 7faafaffd640 1 -- 172.21.15.76:0/365274496 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7faafc091050 con 0x7fab0415df20 2024-03-08T06:02:55.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.099+0000 7fab0a3ca640 1 --2- 172.21.15.76:0/365274496 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faae806b500 0x7faae806d9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fab040d4990 tx=0x7fab00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.099+0000 7fab09bc9640 1 --2- 172.21.15.76:0/365274496 >> [v2:172.21.15.81:6824/3040851989,v1:172.21.15.81:6825/3040851989] conn(0x7faae80719a0 0x7faae8073de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.100+0000 7fab09bc9640 1 --2- 172.21.15.76:0/365274496 >> [v2:172.21.15.81:6824/3040851989,v1:172.21.15.81:6825/3040851989] conn(0x7faae80719a0 0x7faae8073de0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.101+0000 7faafaffd640 1 -- 172.21.15.76:0/365274496 <== osd.6 v2:172.21.15.81:6824/3040851989 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7faafc010e50 con 0x7faae80719a0 2024-03-08T06:02:55.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.114+0000 7fab0c62e640 1 -- 172.21.15.76:0/365274496 --> [v2:172.21.15.81:6824/3040851989,v1:172.21.15.81:6825/3040851989] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fab040ced60 con 0x7faae80719a0 2024-03-08T06:02:55.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.114+0000 7faafaffd640 1 -- 172.21.15.76:0/365274496 <== osd.6 v2:172.21.15.81:6824/3040851989 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fab040ced60 con 0x7faae80719a0 2024-03-08T06:02:55.115 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-08T06:02:55.115 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T06:02:55.115 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-08T06:02:55.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.114+0000 7fab0c62e640 1 -- 172.21.15.76:0/365274496 >> [v2:172.21.15.81:6824/3040851989,v1:172.21.15.81:6825/3040851989] conn(0x7faae80719a0 msgr2=0x7faae8073de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.114+0000 7fab0c62e640 1 --2- 172.21.15.76:0/365274496 >> [v2:172.21.15.81:6824/3040851989,v1:172.21.15.81:6825/3040851989] conn(0x7faae80719a0 0x7faae8073de0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.114+0000 7fab0c62e640 1 -- 172.21.15.76:0/365274496 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faae806b500 msgr2=0x7faae806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.114+0000 7fab0c62e640 1 --2- 172.21.15.76:0/365274496 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7faae806b500 0x7faae806d9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fab040d4990 tx=0x7fab00008040 comp rx=0 tx=0).stop 2024-03-08T06:02:55.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.114+0000 7fab0c62e640 1 -- 172.21.15.76:0/365274496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab0415df20 msgr2=0x7fab041562b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.114+0000 7fab0c62e640 1 --2- 172.21.15.76:0/365274496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab0415df20 0x7fab041562b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7faafc00c7a0 tx=0x7faafc00cc60 comp rx=0 tx=0).stop 2024-03-08T06:02:55.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.114+0000 7fab0abcb640 1 -- 172.21.15.76:0/365274496 reap_dead start 2024-03-08T06:02:55.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.115+0000 7fab0c62e640 1 -- 172.21.15.76:0/365274496 shutdown_connections 2024-03-08T06:02:55.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.115+0000 7fab0c62e640 1 -- 172.21.15.76:0/365274496 >> 172.21.15.76:0/365274496 conn(0x7fab04066fc0 msgr2=0x7fab041592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:55.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.115+0000 7fab0c62e640 1 -- 172.21.15.76:0/365274496 shutdown_connections 2024-03-08T06:02:55.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.115+0000 7fab0c62e640 1 -- 172.21.15.76:0/365274496 wait complete. 2024-03-08T06:02:55.125 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-08T06:02:55.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.227+0000 7f7c66981640 1 Processor -- start 2024-03-08T06:02:55.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.227+0000 7f7c66981640 1 -- start start 2024-03-08T06:02:55.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.228+0000 7f7c66981640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7c6015dcb0 0x7f7c600ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.228+0000 7f7c66981640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7c600afde0 0x7f7c6006b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.228+0000 7f7c66981640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c600cd580 con 0x7f7c600af950 2024-03-08T06:02:55.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.228+0000 7f7c66981640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c600cd860 con 0x7f7c600afde0 2024-03-08T06:02:55.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.228+0000 7f7c66981640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c600cdb40 con 0x7f7c6015dcb0 2024-03-08T06:02:55.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.228+0000 7f7c64f1e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7c600afde0 0x7f7c6006b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.228+0000 7f7c5f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7c600af950 0x7f7c6006fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:58036/0 (socket says 172.21.15.76:58036) 2024-03-08T06:02:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.228+0000 7f7c5f7fe640 1 -- 172.21.15.76:0/3314290306 learned_addr learned my addr 172.21.15.76:0/3314290306 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.228+0000 7f7c5ffff640 1 --2- 172.21.15.76:0/3314290306 >> v2:172.21.15.81:3300/0 conn(0x7f7c6015dcb0 0x7f7c600ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.228+0000 7f7c64f1e640 1 -- 172.21.15.76:0/3314290306 >> v2:172.21.15.81:3300/0 conn(0x7f7c6015dcb0 msgr2=0x7f7c600ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.228+0000 7f7c64f1e640 1 --2- 172.21.15.76:0/3314290306 >> v2:172.21.15.81:3300/0 conn(0x7f7c6015dcb0 0x7f7c600ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.228+0000 7f7c64f1e640 1 -- 172.21.15.76:0/3314290306 >> v1:172.21.15.76:6789/0 conn(0x7f7c600af950 legacy=0x7f7c6006fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.228+0000 7f7c64f1e640 1 -- 172.21.15.76:0/3314290306 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c600cde20 con 0x7f7c600afde0 2024-03-08T06:02:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.228+0000 7f7c5effd640 1 -- 172.21.15.76:0/3314290306 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624623908 0 0) 0x7f7c600cd580 con 0x7f7c600af950 2024-03-08T06:02:55.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.229+0000 7f7c64f1e640 1 --2- 172.21.15.76:0/3314290306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c600afde0 0x7f7c6006b040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7c60073230 tx=0x7f7c5802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7022f8ec4fb595f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.229+0000 7f7c5effd640 1 -- 172.21.15.76:0/3314290306 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c58039070 con 0x7f7c600afde0 2024-03-08T06:02:55.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.229+0000 7f7c5effd640 1 -- 172.21.15.76:0/3314290306 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5802cc80 con 0x7f7c600afde0 2024-03-08T06:02:55.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.229+0000 7f7c5effd640 1 -- 172.21.15.76:0/3314290306 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c58034660 con 0x7f7c600afde0 2024-03-08T06:02:55.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.229+0000 7f7c66981640 1 -- 172.21.15.76:0/3314290306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c600afde0 msgr2=0x7f7c6006b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.229+0000 7f7c66981640 1 --2- 172.21.15.76:0/3314290306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c600afde0 0x7f7c6006b040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7c60073230 tx=0x7f7c5802bed0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.230+0000 7f7c66981640 1 -- 172.21.15.76:0/3314290306 shutdown_connections 2024-03-08T06:02:55.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.230+0000 7f7c66981640 1 --2- 172.21.15.76:0/3314290306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c600afde0 0x7f7c6006b040 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.230+0000 7f7c66981640 1 --2- 172.21.15.76:0/3314290306 >> v2:172.21.15.81:3300/0 conn(0x7f7c6015dcb0 0x7f7c600ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.230+0000 7f7c66981640 1 -- 172.21.15.76:0/3314290306 >> 172.21.15.76:0/3314290306 conn(0x7f7c600b2320 msgr2=0x7f7c600b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:55.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.230+0000 7f7c66981640 1 -- 172.21.15.76:0/3314290306 shutdown_connections 2024-03-08T06:02:55.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.230+0000 7f7c66981640 1 -- 172.21.15.76:0/3314290306 wait complete. 2024-03-08T06:02:55.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.231+0000 7f7c66981640 1 Processor -- start 2024-03-08T06:02:55.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.231+0000 7f7c66981640 1 -- start start 2024-03-08T06:02:55.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c66981640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c600af950 0x7f7c601e4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c66981640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c600afde0 0x7f7c600b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c66981640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7c6015dcb0 0x7f7c601f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c66981640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c600cd360 con 0x7f7c600af950 2024-03-08T06:02:55.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c66981640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c600cd1e0 con 0x7f7c600afde0 2024-03-08T06:02:55.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c66981640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6015e090 con 0x7f7c6015dcb0 2024-03-08T06:02:55.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c5ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c600af950 0x7f7c601e4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c5ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c600af950 0x7f7c601e4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48738/0 (socket says 172.21.15.76:48738) 2024-03-08T06:02:55.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c5ffff640 1 -- 172.21.15.76:0/528687500 learned_addr learned my addr 172.21.15.76:0/528687500 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:55.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c5ffff640 1 -- 172.21.15.76:0/528687500 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7c6015dcb0 msgr2=0x7f7c601f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c5ffff640 1 --2- 172.21.15.76:0/528687500 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7c6015dcb0 0x7f7c601f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c5ffff640 1 -- 172.21.15.76:0/528687500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c600afde0 msgr2=0x7f7c600b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c5ffff640 1 --2- 172.21.15.76:0/528687500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c600afde0 0x7f7c600b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c5ffff640 1 -- 172.21.15.76:0/528687500 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c600c7af0 con 0x7f7c600af950 2024-03-08T06:02:55.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c5ffff640 1 --2- 172.21.15.76:0/528687500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c600af950 0x7f7c601e4b30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000d800 tx=0x7f7c5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c5cff9640 1 -- 172.21.15.76:0/528687500 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000b6c0 con 0x7f7c600af950 2024-03-08T06:02:55.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c5cff9640 1 -- 172.21.15.76:0/528687500 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c50019070 con 0x7f7c600af950 2024-03-08T06:02:55.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c66981640 1 -- 172.21.15.76:0/528687500 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c600c7dd0 con 0x7f7c600af950 2024-03-08T06:02:55.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c5cff9640 1 -- 172.21.15.76:0/528687500 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000ba80 con 0x7f7c600af950 2024-03-08T06:02:55.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.232+0000 7f7c66981640 1 -- 172.21.15.76:0/528687500 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c600c8390 con 0x7f7c600af950 2024-03-08T06:02:55.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.233+0000 7f7c66981640 1 -- 172.21.15.76:0/528687500 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c600c7860 con 0x7f7c600af950 2024-03-08T06:02:55.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.234+0000 7f7c5cff9640 1 -- 172.21.15.76:0/528687500 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c5000bc20 con 0x7f7c600af950 2024-03-08T06:02:55.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.234+0000 7f7c5cff9640 1 --2- 172.21.15.76:0/528687500 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7c4c06b4d0 0x7f7c4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.234+0000 7f7c5f7fe640 1 --2- 172.21.15.76:0/528687500 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7c4c06b4d0 0x7f7c4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.234+0000 7f7c5cff9640 1 -- 172.21.15.76:0/528687500 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7c5008a4b0 con 0x7f7c600af950 2024-03-08T06:02:55.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.235+0000 7f7c5f7fe640 1 --2- 172.21.15.76:0/528687500 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7c4c06b4d0 0x7f7c4c06d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7c54007fd0 tx=0x7f7c54007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.239+0000 7f7c5cff9640 1 -- 172.21.15.76:0/528687500 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c50059530 con 0x7f7c600af950 2024-03-08T06:02:55.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.410+0000 7f7c66981640 1 -- 172.21.15.76:0/528687500 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c6004f030 con 0x7f7c4c06b4d0 2024-03-08T06:02:55.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.410+0000 7f7c5cff9640 1 -- 172.21.15.76:0/528687500 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7c6004f030 con 0x7f7c4c06b4d0 2024-03-08T06:02:55.411 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 6 to deep-scrub 2024-03-08T06:02:55.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.412+0000 7f7c66981640 1 -- 172.21.15.76:0/528687500 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7c4c06b4d0 msgr2=0x7f7c4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.412+0000 7f7c66981640 1 --2- 172.21.15.76:0/528687500 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7c4c06b4d0 0x7f7c4c06d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7c54007fd0 tx=0x7f7c54007960 comp rx=0 tx=0).stop 2024-03-08T06:02:55.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.412+0000 7f7c66981640 1 -- 172.21.15.76:0/528687500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c600af950 msgr2=0x7f7c601e4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.412+0000 7f7c66981640 1 --2- 172.21.15.76:0/528687500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c600af950 0x7f7c601e4b30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000d800 tx=0x7f7c5000dcc0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.412+0000 7f7c66981640 1 -- 172.21.15.76:0/528687500 shutdown_connections 2024-03-08T06:02:55.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.412+0000 7f7c66981640 1 --2- 172.21.15.76:0/528687500 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f7c4c06b4d0 0x7f7c4c06d990 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.412+0000 7f7c66981640 1 --2- 172.21.15.76:0/528687500 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7c6015dcb0 0x7f7c601f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.412+0000 7f7c66981640 1 --2- 172.21.15.76:0/528687500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c600afde0 0x7f7c600b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.412+0000 7f7c66981640 1 --2- 172.21.15.76:0/528687500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c600af950 0x7f7c601e4b30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.412+0000 7f7c66981640 1 -- 172.21.15.76:0/528687500 >> 172.21.15.76:0/528687500 conn(0x7f7c600b2320 msgr2=0x7f7c6006af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:55.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.412+0000 7f7c66981640 1 -- 172.21.15.76:0/528687500 shutdown_connections 2024-03-08T06:02:55.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.412+0000 7f7c66981640 1 -- 172.21.15.76:0/528687500 wait complete. 2024-03-08T06:02:55.423 INFO:tasks.ceph:Scrubbing osd.7 2024-03-08T06:02:55.423 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-08T06:02:55.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.526+0000 7fe3bd223640 1 Processor -- start 2024-03-08T06:02:55.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.526+0000 7fe3bd223640 1 -- start start 2024-03-08T06:02:55.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.527+0000 7fe3bd223640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe3b806c7a0 0x7fe3b806cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.527+0000 7fe3bd223640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b80cbf30 con 0x7fe3b81613b0 2024-03-08T06:02:55.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.527+0000 7fe3bd223640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b80b2bd0 con 0x7fe3b80cc6c0 2024-03-08T06:02:55.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.527+0000 7fe3bd223640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b806d140 con 0x7fe3b806c7a0 2024-03-08T06:02:55.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.527+0000 7fe3b6d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe3b81613b0 0x7fe3b806bfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:58048/0 (socket says 172.21.15.76:58048) 2024-03-08T06:02:55.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.527+0000 7fe3b6d9d640 1 -- 172.21.15.76:0/550100164 learned_addr learned my addr 172.21.15.76:0/550100164 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:55.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.527+0000 7fe3b659c640 1 --2- 172.21.15.76:0/550100164 >> v2:172.21.15.81:3300/0 conn(0x7fe3b806c7a0 0x7fe3b806cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b5d9b640 1 -- 172.21.15.76:0/550100164 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897227323 0 0) 0x7fe3b80cbf30 con 0x7fe3b81613b0 2024-03-08T06:02:55.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b5d9b640 1 -- 172.21.15.76:0/550100164 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3a0003520 con 0x7fe3b81613b0 2024-03-08T06:02:55.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b5d9b640 1 -- 172.21.15.76:0/550100164 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1891826742 0 0) 0x7fe3a0003520 con 0x7fe3b81613b0 2024-03-08T06:02:55.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b5d9b640 1 -- 172.21.15.76:0/550100164 >> v2:172.21.15.81:3300/0 conn(0x7fe3b806c7a0 msgr2=0x7fe3b806cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:55.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b5d9b640 1 --2- 172.21.15.76:0/550100164 >> v2:172.21.15.81:3300/0 conn(0x7fe3b806c7a0 0x7fe3b806cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b5d9b640 1 -- 172.21.15.76:0/550100164 >> v1:172.21.15.81:6789/0 conn(0x7fe3b80cc6c0 legacy=0x7fe3b80ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b5d9b640 1 -- 172.21.15.76:0/550100164 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b8099eb0 con 0x7fe3b81613b0 2024-03-08T06:02:55.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b5d9b640 1 -- 172.21.15.76:0/550100164 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1289212351 0 0) 0x7fe3a80019f0 con 0x7fe3b81613b0 2024-03-08T06:02:55.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b5d9b640 1 -- 172.21.15.76:0/550100164 >> v1:172.21.15.76:6789/0 conn(0x7fe3b81613b0 legacy=0x7fe3b806bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b5d9b640 1 --2- 172.21.15.76:0/550100164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a00039b0 0x7fe3a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b5d9b640 1 --2- 172.21.15.76:0/550100164 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a0004800 0x7fe3a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b5d9b640 1 --2- 172.21.15.76:0/550100164 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3a00051c0 0x7fe3a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b5d9b640 1 -- 172.21.15.76:0/550100164 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a000c130 con 0x7fe3a00039b0 2024-03-08T06:02:55.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b5d9b640 1 -- 172.21.15.76:0/550100164 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a000c2e0 con 0x7fe3a0004800 2024-03-08T06:02:55.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b5d9b640 1 -- 172.21.15.76:0/550100164 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a000c4e0 con 0x7fe3a00051c0 2024-03-08T06:02:55.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b6d9d640 1 --2- 172.21.15.76:0/550100164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a00039b0 0x7fe3a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b659c640 1 --2- 172.21.15.76:0/550100164 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a0004800 0x7fe3a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b759e640 1 --2- 172.21.15.76:0/550100164 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3a00051c0 0x7fe3a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b6d9d640 1 -- 172.21.15.76:0/550100164 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3a00051c0 msgr2=0x7fe3a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:55.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b6d9d640 1 --2- 172.21.15.76:0/550100164 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3a00051c0 0x7fe3a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b6d9d640 1 -- 172.21.15.76:0/550100164 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a0004800 msgr2=0x7fe3a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:55.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b6d9d640 1 --2- 172.21.15.76:0/550100164 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a0004800 0x7fe3a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b6d9d640 1 -- 172.21.15.76:0/550100164 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a0003520 con 0x7fe3a00039b0 2024-03-08T06:02:55.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.528+0000 7fe3b759e640 1 -- 172.21.15.76:0/550100164 reap_dead start 2024-03-08T06:02:55.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.529+0000 7fe3b6d9d640 1 --2- 172.21.15.76:0/550100164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a00039b0 0x7fe3a0003dd0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800a850 tx=0x7fe3a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25c463b069fe652 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.529+0000 7fe3b5d9b640 1 -- 172.21.15.76:0/550100164 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a800a4a0 con 0x7fe3a00039b0 2024-03-08T06:02:55.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.529+0000 7fe3b5d9b640 1 -- 172.21.15.76:0/550100164 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a8034650 con 0x7fe3a00039b0 2024-03-08T06:02:55.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.529+0000 7fe3b5d9b640 1 -- 172.21.15.76:0/550100164 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8034a80 con 0x7fe3a00039b0 2024-03-08T06:02:55.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.530+0000 7fe3bd223640 1 -- 172.21.15.76:0/550100164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a00039b0 msgr2=0x7fe3a0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.530+0000 7fe3bd223640 1 --2- 172.21.15.76:0/550100164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a00039b0 0x7fe3a0003dd0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800a850 tx=0x7fe3a0005830 comp rx=0 tx=0).stop 2024-03-08T06:02:55.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.530+0000 7fe3bd223640 1 -- 172.21.15.76:0/550100164 shutdown_connections 2024-03-08T06:02:55.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.530+0000 7fe3bd223640 1 --2- 172.21.15.76:0/550100164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a00039b0 0x7fe3a0003dd0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.530+0000 7fe3bd223640 1 -- 172.21.15.76:0/550100164 >> 172.21.15.76:0/550100164 conn(0x7fe3b80ce350 msgr2=0x7fe3b80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:55.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.530+0000 7fe3bd223640 1 -- 172.21.15.76:0/550100164 shutdown_connections 2024-03-08T06:02:55.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.530+0000 7fe3bd223640 1 -- 172.21.15.76:0/550100164 wait complete. 2024-03-08T06:02:55.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.531+0000 7fe3bd223640 1 Processor -- start 2024-03-08T06:02:55.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.531+0000 7fe3bd223640 1 -- start start 2024-03-08T06:02:55.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.532+0000 7fe3bd223640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a00039b0 0x7fe3b80b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.532+0000 7fe3bd223640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3b81f44a0 0x7fe3b80b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.532+0000 7fe3bd223640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3b80ae520 0x7fe3b80d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.532+0000 7fe3bd223640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b8161790 con 0x7fe3b81f44a0 2024-03-08T06:02:55.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.532+0000 7fe3bd223640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80cc1a0 con 0x7fe3a00039b0 2024-03-08T06:02:55.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.532+0000 7fe3bd223640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b809a470 con 0x7fe3b80ae520 2024-03-08T06:02:55.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.532+0000 7fe3b659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3b81f44a0 0x7fe3b80b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.532+0000 7fe3b659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3b81f44a0 0x7fe3b80b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48746/0 (socket says 172.21.15.76:48746) 2024-03-08T06:02:55.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.532+0000 7fe3b659c640 1 -- 172.21.15.76:0/316861161 learned_addr learned my addr 172.21.15.76:0/316861161 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:55.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.532+0000 7fe3b6d9d640 1 --2- 172.21.15.76:0/316861161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a00039b0 0x7fe3b80b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.532+0000 7fe3b759e640 1 --2- 172.21.15.76:0/316861161 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3b80ae520 0x7fe3b80d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.532+0000 7fe3b659c640 1 -- 172.21.15.76:0/316861161 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3b80ae520 msgr2=0x7fe3b80d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.532+0000 7fe3b659c640 1 --2- 172.21.15.76:0/316861161 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe3b80ae520 0x7fe3b80d0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.532+0000 7fe3b659c640 1 -- 172.21.15.76:0/316861161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a00039b0 msgr2=0x7fe3b80b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.532+0000 7fe3b659c640 1 --2- 172.21.15.76:0/316861161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a00039b0 0x7fe3b80b2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.532+0000 7fe3b659c640 1 -- 172.21.15.76:0/316861161 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b80b0b40 con 0x7fe3b81f44a0 2024-03-08T06:02:55.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.533+0000 7fe3b659c640 1 --2- 172.21.15.76:0/316861161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3b81f44a0 0x7fe3b80b0600 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac011960 tx=0x7fe3ac011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.533+0000 7fe39f7fe640 1 -- 172.21.15.76:0/316861161 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00faf0 con 0x7fe3b81f44a0 2024-03-08T06:02:55.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.533+0000 7fe39f7fe640 1 -- 172.21.15.76:0/316861161 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac00fc90 con 0x7fe3b81f44a0 2024-03-08T06:02:55.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.533+0000 7fe3bd223640 1 -- 172.21.15.76:0/316861161 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b81f4920 con 0x7fe3b81f44a0 2024-03-08T06:02:55.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.533+0000 7fe39f7fe640 1 -- 172.21.15.76:0/316861161 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac014440 con 0x7fe3b81f44a0 2024-03-08T06:02:55.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.533+0000 7fe3bd223640 1 -- 172.21.15.76:0/316861161 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b81f4ed0 con 0x7fe3b81f44a0 2024-03-08T06:02:55.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.534+0000 7fe3bd223640 1 -- 172.21.15.76:0/316861161 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe3b804f030 con 0x7fe3b81f44a0 2024-03-08T06:02:55.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.534+0000 7fe39f7fe640 1 -- 172.21.15.76:0/316861161 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3ac012040 con 0x7fe3b81f44a0 2024-03-08T06:02:55.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.534+0000 7fe39f7fe640 1 --2- 172.21.15.76:0/316861161 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe3a406b500 0x7fe3a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.534+0000 7fe39f7fe640 1 -- 172.21.15.76:0/316861161 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe3ac08a4b0 con 0x7fe3b81f44a0 2024-03-08T06:02:55.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.535+0000 7fe3b6d9d640 1 --2- 172.21.15.76:0/316861161 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe3a406b500 0x7fe3a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.535+0000 7fe39f7fe640 1 --2- 172.21.15.76:0/316861161 >> [v2:172.21.15.81:6816/298940182,v1:172.21.15.81:6817/298940182] conn(0x7fe3a40719a0 0x7fe3a4073de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.535+0000 7fe39f7fe640 1 -- 172.21.15.76:0/316861161 --> [v2:172.21.15.81:6816/298940182,v1:172.21.15.81:6817/298940182] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe3ac025ec0 con 0x7fe3a40719a0 2024-03-08T06:02:55.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.535+0000 7fe39f7fe640 1 -- 172.21.15.76:0/316861161 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3ac091050 con 0x7fe3b81f44a0 2024-03-08T06:02:55.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.535+0000 7fe3b759e640 1 --2- 172.21.15.76:0/316861161 >> [v2:172.21.15.81:6816/298940182,v1:172.21.15.81:6817/298940182] conn(0x7fe3a40719a0 0x7fe3a4073de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.535+0000 7fe3b6d9d640 1 --2- 172.21.15.76:0/316861161 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe3a406b500 0x7fe3a406d9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8034000 tx=0x7fe3a802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.535+0000 7fe3b759e640 1 --2- 172.21.15.76:0/316861161 >> [v2:172.21.15.81:6816/298940182,v1:172.21.15.81:6817/298940182] conn(0x7fe3a40719a0 0x7fe3a4073de0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.536+0000 7fe39f7fe640 1 -- 172.21.15.76:0/316861161 <== osd.7 v2:172.21.15.81:6816/298940182 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe3ac025ec0 con 0x7fe3a40719a0 2024-03-08T06:02:55.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.549+0000 7fe3bd223640 1 -- 172.21.15.76:0/316861161 --> [v2:172.21.15.81:6816/298940182,v1:172.21.15.81:6817/298940182] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe3b80aec20 con 0x7fe3a40719a0 2024-03-08T06:02:55.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.549+0000 7fe39f7fe640 1 -- 172.21.15.76:0/316861161 <== osd.7 v2:172.21.15.81:6816/298940182 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe3b80aec20 con 0x7fe3a40719a0 2024-03-08T06:02:55.550 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-08T06:02:55.550 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T06:02:55.550 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-08T06:02:55.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.549+0000 7fe3bd223640 1 -- 172.21.15.76:0/316861161 >> [v2:172.21.15.81:6816/298940182,v1:172.21.15.81:6817/298940182] conn(0x7fe3a40719a0 msgr2=0x7fe3a4073de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.549+0000 7fe3bd223640 1 --2- 172.21.15.76:0/316861161 >> [v2:172.21.15.81:6816/298940182,v1:172.21.15.81:6817/298940182] conn(0x7fe3a40719a0 0x7fe3a4073de0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.549+0000 7fe3bd223640 1 -- 172.21.15.76:0/316861161 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe3a406b500 msgr2=0x7fe3a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.549+0000 7fe3bd223640 1 --2- 172.21.15.76:0/316861161 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7fe3a406b500 0x7fe3a406d9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8034000 tx=0x7fe3a802f000 comp rx=0 tx=0).stop 2024-03-08T06:02:55.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.549+0000 7fe3bd223640 1 -- 172.21.15.76:0/316861161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3b81f44a0 msgr2=0x7fe3b80b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.549+0000 7fe3bd223640 1 --2- 172.21.15.76:0/316861161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3b81f44a0 0x7fe3b80b0600 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac011960 tx=0x7fe3ac011e20 comp rx=0 tx=0).stop 2024-03-08T06:02:55.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.549+0000 7fe3b759e640 1 -- 172.21.15.76:0/316861161 reap_dead start 2024-03-08T06:02:55.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.549+0000 7fe3bd223640 1 -- 172.21.15.76:0/316861161 shutdown_connections 2024-03-08T06:02:55.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.549+0000 7fe3bd223640 1 -- 172.21.15.76:0/316861161 >> 172.21.15.76:0/316861161 conn(0x7fe3b80ce350 msgr2=0x7fe3b8070210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:55.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.549+0000 7fe3bd223640 1 -- 172.21.15.76:0/316861161 shutdown_connections 2024-03-08T06:02:55.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.549+0000 7fe3bd223640 1 -- 172.21.15.76:0/316861161 wait complete. 2024-03-08T06:02:55.560 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-08T06:02:55.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.661+0000 7f0dd4895640 1 Processor -- start 2024-03-08T06:02:55.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.661+0000 7f0dd4895640 1 -- start start 2024-03-08T06:02:55.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.662+0000 7f0dd4895640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dd015dbc0 0x7f0dd00d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.662+0000 7f0dd4895640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0dd00b5cf0 0x7f0dd00b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.662+0000 7f0dd4895640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd00d27d0 con 0x7f0dd00b9840 2024-03-08T06:02:55.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.662+0000 7f0dd4895640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd00d2ab0 con 0x7f0dd00b5cf0 2024-03-08T06:02:55.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.662+0000 7f0dd4895640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd00d2d90 con 0x7f0dd015dbc0 2024-03-08T06:02:55.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.662+0000 7f0dced9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0dd00b5cf0 0x7f0dd00b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.662+0000 7f0dcdd9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0dd00b9840 0x7f0dd00b55e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:58050/0 (socket says 172.21.15.76:58050) 2024-03-08T06:02:55.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.662+0000 7f0dced9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0dd00b5cf0 0x7f0dd00b6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48752/0 (socket says 172.21.15.76:48752) 2024-03-08T06:02:55.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.662+0000 7f0dcdd9b640 1 -- 172.21.15.76:0/1870755496 learned_addr learned my addr 172.21.15.76:0/1870755496 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:55.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.663+0000 7f0dce59c640 1 --2- 172.21.15.76:0/1870755496 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dd015dbc0 0x7f0dd00d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.663+0000 7f0dced9d640 1 -- 172.21.15.76:0/1870755496 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dd015dbc0 msgr2=0x7f0dd00d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:55.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.663+0000 7f0dced9d640 1 --2- 172.21.15.76:0/1870755496 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dd015dbc0 0x7f0dd00d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.663+0000 7f0dced9d640 1 -- 172.21.15.76:0/1870755496 >> v1:172.21.15.76:6789/0 conn(0x7f0dd00b9840 legacy=0x7f0dd00b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.663+0000 7f0dcd59a640 1 -- 172.21.15.76:0/1870755496 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710296610 0 0) 0x7f0dd00d27d0 con 0x7f0dd00b9840 2024-03-08T06:02:55.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.663+0000 7f0dced9d640 1 -- 172.21.15.76:0/1870755496 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd00d3070 con 0x7f0dd00b5cf0 2024-03-08T06:02:55.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.663+0000 7f0dced9d640 1 --2- 172.21.15.76:0/1870755496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dd00b5cf0 0x7f0dd00b6c90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80097a0 tx=0x7f0dc802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe77cad0db8ae9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.664+0000 7f0dcd59a640 1 -- 172.21.15.76:0/1870755496 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8035020 con 0x7f0dd00b5cf0 2024-03-08T06:02:55.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.664+0000 7f0dcd59a640 1 -- 172.21.15.76:0/1870755496 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc8031830 con 0x7f0dd00b5cf0 2024-03-08T06:02:55.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.664+0000 7f0dcd59a640 1 -- 172.21.15.76:0/1870755496 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8031c00 con 0x7f0dd00b5cf0 2024-03-08T06:02:55.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.664+0000 7f0dd4895640 1 -- 172.21.15.76:0/1870755496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dd00b5cf0 msgr2=0x7f0dd00b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.664+0000 7f0dd4895640 1 --2- 172.21.15.76:0/1870755496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dd00b5cf0 0x7f0dd00b6c90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80097a0 tx=0x7f0dc802b8d0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.664+0000 7f0dd4895640 1 -- 172.21.15.76:0/1870755496 shutdown_connections 2024-03-08T06:02:55.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.664+0000 7f0dd4895640 1 --2- 172.21.15.76:0/1870755496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dd00b5cf0 0x7f0dd00b6c90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.664+0000 7f0dd4895640 1 --2- 172.21.15.76:0/1870755496 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dd015dbc0 0x7f0dd00d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.664+0000 7f0dd4895640 1 -- 172.21.15.76:0/1870755496 >> 172.21.15.76:0/1870755496 conn(0x7f0dd00670d0 msgr2=0x7f0dd00d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:55.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.664+0000 7f0dd4895640 1 -- 172.21.15.76:0/1870755496 shutdown_connections 2024-03-08T06:02:55.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.664+0000 7f0dd4895640 1 -- 172.21.15.76:0/1870755496 wait complete. 2024-03-08T06:02:55.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.666+0000 7f0dd4895640 1 Processor -- start 2024-03-08T06:02:55.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.666+0000 7f0dd4895640 1 -- start start 2024-03-08T06:02:55.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.666+0000 7f0dd4895640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dd00b5cf0 0x7f0dd00cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.666+0000 7f0dd4895640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dd00b9840 0x7f0dd0159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.666+0000 7f0dd4895640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd015dbc0 0x7f0dd00cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.666+0000 7f0dd4895640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd00d25b0 con 0x7f0dd00b5cf0 2024-03-08T06:02:55.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.666+0000 7f0dd4895640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd00d2430 con 0x7f0dd015dbc0 2024-03-08T06:02:55.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.666+0000 7f0dd4895640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd015dfa0 con 0x7f0dd00b9840 2024-03-08T06:02:55.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.666+0000 7f0dce59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dd00b5cf0 0x7f0dd00cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.666+0000 7f0dce59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dd00b5cf0 0x7f0dd00cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48766/0 (socket says 172.21.15.76:48766) 2024-03-08T06:02:55.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.666+0000 7f0dce59c640 1 -- 172.21.15.76:0/1127821010 learned_addr learned my addr 172.21.15.76:0/1127821010 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:55.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.667+0000 7f0dcdd9b640 1 --2- 172.21.15.76:0/1127821010 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dd00b9840 0x7f0dd0159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.667+0000 7f0dced9d640 1 --2- 172.21.15.76:0/1127821010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd015dbc0 0x7f0dd00cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.667+0000 7f0dce59c640 1 -- 172.21.15.76:0/1127821010 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dd00b9840 msgr2=0x7f0dd0159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.667+0000 7f0dce59c640 1 --2- 172.21.15.76:0/1127821010 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dd00b9840 0x7f0dd0159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.667+0000 7f0dce59c640 1 -- 172.21.15.76:0/1127821010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd015dbc0 msgr2=0x7f0dd00cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.667+0000 7f0dce59c640 1 --2- 172.21.15.76:0/1127821010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd015dbc0 0x7f0dd00cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.667+0000 7f0dce59c640 1 -- 172.21.15.76:0/1127821010 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd01583a0 con 0x7f0dd00b5cf0 2024-03-08T06:02:55.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.667+0000 7f0dce59c640 1 --2- 172.21.15.76:0/1127821010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dd00b5cf0 0x7f0dd00cd420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0007e90 tx=0x7f0dc000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.667+0000 7f0dbeffd640 1 -- 172.21.15.76:0/1127821010 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0014620 con 0x7f0dd00b5cf0 2024-03-08T06:02:55.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.667+0000 7f0dbeffd640 1 -- 172.21.15.76:0/1127821010 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc0019070 con 0x7f0dd00b5cf0 2024-03-08T06:02:55.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.667+0000 7f0dbeffd640 1 -- 172.21.15.76:0/1127821010 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0014a20 con 0x7f0dd00b5cf0 2024-03-08T06:02:55.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.667+0000 7f0dd4895640 1 -- 172.21.15.76:0/1127821010 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd0158680 con 0x7f0dd00b5cf0 2024-03-08T06:02:55.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.667+0000 7f0dd4895640 1 -- 172.21.15.76:0/1127821010 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd0158c40 con 0x7f0dd00b5cf0 2024-03-08T06:02:55.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.668+0000 7f0dd4895640 1 -- 172.21.15.76:0/1127821010 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd004f030 con 0x7f0dd00b5cf0 2024-03-08T06:02:55.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.669+0000 7f0dbeffd640 1 -- 172.21.15.76:0/1127821010 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0dc0014bc0 con 0x7f0dd00b5cf0 2024-03-08T06:02:55.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.669+0000 7f0dbeffd640 1 --2- 172.21.15.76:0/1127821010 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0db006b4d0 0x7f0db006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.669+0000 7f0dbeffd640 1 -- 172.21.15.76:0/1127821010 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0dc008a930 con 0x7f0dd00b5cf0 2024-03-08T06:02:55.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.669+0000 7f0dcdd9b640 1 --2- 172.21.15.76:0/1127821010 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0db006b4d0 0x7f0db006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.669+0000 7f0dcdd9b640 1 --2- 172.21.15.76:0/1127821010 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0db006b4d0 0x7f0db006d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00cf380 tx=0x7f0dc4007b50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.673+0000 7f0dbeffd640 1 -- 172.21.15.76:0/1127821010 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0dc0059920 con 0x7f0dd00b5cf0 2024-03-08T06:02:55.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.845+0000 7f0dd4895640 1 -- 172.21.15.76:0/1127821010 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dd00c9f60 con 0x7f0db006b4d0 2024-03-08T06:02:55.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.845+0000 7f0dbeffd640 1 -- 172.21.15.76:0/1127821010 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0dd00c9f60 con 0x7f0db006b4d0 2024-03-08T06:02:55.845 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 7 to deep-scrub 2024-03-08T06:02:55.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.847+0000 7f0dd4895640 1 -- 172.21.15.76:0/1127821010 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0db006b4d0 msgr2=0x7f0db006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.847+0000 7f0dd4895640 1 --2- 172.21.15.76:0/1127821010 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0db006b4d0 0x7f0db006d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00cf380 tx=0x7f0dc4007b50 comp rx=0 tx=0).stop 2024-03-08T06:02:55.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.847+0000 7f0dd4895640 1 -- 172.21.15.76:0/1127821010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dd00b5cf0 msgr2=0x7f0dd00cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.847+0000 7f0dd4895640 1 --2- 172.21.15.76:0/1127821010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dd00b5cf0 0x7f0dd00cd420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0007e90 tx=0x7f0dc000e4b0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.847+0000 7f0dd4895640 1 -- 172.21.15.76:0/1127821010 shutdown_connections 2024-03-08T06:02:55.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.847+0000 7f0dd4895640 1 --2- 172.21.15.76:0/1127821010 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f0db006b4d0 0x7f0db006d990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.847+0000 7f0dd4895640 1 --2- 172.21.15.76:0/1127821010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd015dbc0 0x7f0dd00cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.847+0000 7f0dd4895640 1 --2- 172.21.15.76:0/1127821010 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0dd00b9840 0x7f0dd0159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.847+0000 7f0dd4895640 1 --2- 172.21.15.76:0/1127821010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dd00b5cf0 0x7f0dd00cd420 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.847+0000 7f0dd4895640 1 -- 172.21.15.76:0/1127821010 >> 172.21.15.76:0/1127821010 conn(0x7f0dd00670d0 msgr2=0x7f0dd00c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:55.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.847+0000 7f0dd4895640 1 -- 172.21.15.76:0/1127821010 shutdown_connections 2024-03-08T06:02:55.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.847+0000 7f0dd4895640 1 -- 172.21.15.76:0/1127821010 wait complete. 2024-03-08T06:02:55.858 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:02:55.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.957+0000 7f38cc5d7640 1 Processor -- start 2024-03-08T06:02:55.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.957+0000 7f38cc5d7640 1 -- start start 2024-03-08T06:02:55.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.958+0000 7f38cc5d7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f38c415a9f0 0x7f38c4071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.958+0000 7f38cc5d7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c40722e0 con 0x7f38c40c98e0 2024-03-08T06:02:55.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.958+0000 7f38cc5d7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c40ae8d0 con 0x7f38c40cec20 2024-03-08T06:02:55.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.958+0000 7f38cc5d7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c40ce790 con 0x7f38c415a9f0 2024-03-08T06:02:55.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.958+0000 7f38ca373640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f38c415a9f0 0x7f38c4071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.958+0000 7f38c9b72640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f38c40c98e0 0x7f38c40c9d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:58058/0 (socket says 172.21.15.76:58058) 2024-03-08T06:02:55.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.958+0000 7f38c9b72640 1 -- 172.21.15.76:0/1511061547 learned_addr learned my addr 172.21.15.76:0/1511061547 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:02:55.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.959+0000 7f38ca373640 1 -- 172.21.15.76:0/1511061547 >> v1:172.21.15.81:6789/0 conn(0x7f38c40cec20 legacy=0x7f38c406e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:02:55.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.959+0000 7f38ca373640 1 -- 172.21.15.76:0/1511061547 >> v1:172.21.15.76:6789/0 conn(0x7f38c40c98e0 legacy=0x7f38c40c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.959+0000 7f38ca373640 1 -- 172.21.15.76:0/1511061547 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c40aebb0 con 0x7f38c415a9f0 2024-03-08T06:02:55.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.959+0000 7f38c9371640 1 -- 172.21.15.76:0/1511061547 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521926625 0 0) 0x7f38c40722e0 con 0x7f38c40c98e0 2024-03-08T06:02:55.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.959+0000 7f38ca373640 1 --2- 172.21.15.76:0/1511061547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c415a9f0 0x7f38c4071da0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f38c00097a0 tx=0x7f38c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19608b8b3d511482 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.960+0000 7f38c9371640 1 -- 172.21.15.76:0/1511061547 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0035020 con 0x7f38c415a9f0 2024-03-08T06:02:55.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.960+0000 7f38c9371640 1 -- 172.21.15.76:0/1511061547 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c0031830 con 0x7f38c415a9f0 2024-03-08T06:02:55.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.960+0000 7f38c9371640 1 -- 172.21.15.76:0/1511061547 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0031c00 con 0x7f38c415a9f0 2024-03-08T06:02:55.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.960+0000 7f38cc5d7640 1 -- 172.21.15.76:0/1511061547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c415a9f0 msgr2=0x7f38c4071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.960+0000 7f38cc5d7640 1 --2- 172.21.15.76:0/1511061547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c415a9f0 0x7f38c4071da0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f38c00097a0 tx=0x7f38c002b8d0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.960+0000 7f38cc5d7640 1 -- 172.21.15.76:0/1511061547 shutdown_connections 2024-03-08T06:02:55.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.960+0000 7f38cc5d7640 1 --2- 172.21.15.76:0/1511061547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c415a9f0 0x7f38c4071da0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.960+0000 7f38cc5d7640 1 -- 172.21.15.76:0/1511061547 >> 172.21.15.76:0/1511061547 conn(0x7f38c40cf740 msgr2=0x7f38c40cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:55.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.960+0000 7f38cc5d7640 1 -- 172.21.15.76:0/1511061547 shutdown_connections 2024-03-08T06:02:55.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.960+0000 7f38cc5d7640 1 -- 172.21.15.76:0/1511061547 wait complete. 2024-03-08T06:02:55.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.961+0000 7f38cc5d7640 1 Processor -- start 2024-03-08T06:02:55.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.961+0000 7f38cc5d7640 1 -- start start 2024-03-08T06:02:55.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38cc5d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c40c98e0 0x7f38c414cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38cc5d7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38c40cec20 0x7f38c4159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38cc5d7640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f38c415a9f0 0x7f38c415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38cc5d7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40ca4a0 con 0x7f38c40c98e0 2024-03-08T06:02:55.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38cc5d7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40ce420 con 0x7f38c40cec20 2024-03-08T06:02:55.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38cc5d7640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40ca620 con 0x7f38c415a9f0 2024-03-08T06:02:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38ca373640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c40c98e0 0x7f38c414cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38ca373640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c40c98e0 0x7f38c414cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48784/0 (socket says 172.21.15.76:48784) 2024-03-08T06:02:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38ca373640 1 -- 172.21.15.76:0/1119713278 learned_addr learned my addr 172.21.15.76:0/1119713278 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:02:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38cab74640 1 --2- 172.21.15.76:0/1119713278 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f38c415a9f0 0x7f38c415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38c9b72640 1 --2- 172.21.15.76:0/1119713278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38c40cec20 0x7f38c4159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38ca373640 1 -- 172.21.15.76:0/1119713278 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f38c415a9f0 msgr2=0x7f38c415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38ca373640 1 --2- 172.21.15.76:0/1119713278 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f38c415a9f0 0x7f38c415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38ca373640 1 -- 172.21.15.76:0/1119713278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38c40cec20 msgr2=0x7f38c4159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38ca373640 1 --2- 172.21.15.76:0/1119713278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38c40cec20 0x7f38c4159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38ca373640 1 -- 172.21.15.76:0/1119713278 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c414c150 con 0x7f38c40c98e0 2024-03-08T06:02:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.962+0000 7f38c9b72640 1 --2- 172.21.15.76:0/1119713278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38c40cec20 0x7f38c4159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:02:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.963+0000 7f38ca373640 1 --2- 172.21.15.76:0/1119713278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c40c98e0 0x7f38c414cea0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f38c00097a0 tx=0x7f38c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.963+0000 7f38baffd640 1 -- 172.21.15.76:0/1119713278 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0004230 con 0x7f38c40c98e0 2024-03-08T06:02:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.963+0000 7f38baffd640 1 -- 172.21.15.76:0/1119713278 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c00043f0 con 0x7f38c40c98e0 2024-03-08T06:02:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.963+0000 7f38cc5d7640 1 -- 172.21.15.76:0/1119713278 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c415a220 con 0x7f38c40c98e0 2024-03-08T06:02:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.963+0000 7f38baffd640 1 -- 172.21.15.76:0/1119713278 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0003850 con 0x7f38c40c98e0 2024-03-08T06:02:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.963+0000 7f38cc5d7640 1 -- 172.21.15.76:0/1119713278 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c414c6a0 con 0x7f38c40c98e0 2024-03-08T06:02:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.964+0000 7f38cc5d7640 1 -- 172.21.15.76:0/1119713278 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c404f030 con 0x7f38c40c98e0 2024-03-08T06:02:55.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.964+0000 7f38baffd640 1 -- 172.21.15.76:0/1119713278 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38c0038020 con 0x7f38c40c98e0 2024-03-08T06:02:55.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.964+0000 7f38baffd640 1 --2- 172.21.15.76:0/1119713278 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f38a806b4d0 0x7f38a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:02:55.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.965+0000 7f38baffd640 1 -- 172.21.15.76:0/1119713278 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f38c00abb20 con 0x7f38c40c98e0 2024-03-08T06:02:55.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.965+0000 7f38c9b72640 1 --2- 172.21.15.76:0/1119713278 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f38a806b4d0 0x7f38a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:02:55.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.965+0000 7f38c9b72640 1 --2- 172.21.15.76:0/1119713278 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f38a806b4d0 0x7f38a806d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f38b4007fd0 tx=0x7f38b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:02:55.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:55.969+0000 7f38baffd640 1 -- 172.21.15.76:0/1119713278 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38c0040070 con 0x7f38c40c98e0 2024-03-08T06:02:56.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:56.138+0000 7f38cc5d7640 1 -- 172.21.15.76:0/1119713278 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38c4155cc0 con 0x7f38a806b4d0 2024-03-08T06:02:56.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:56.139+0000 7f38baffd640 1 -- 172.21.15.76:0/1119713278 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13456 (secure 0 0 0) 0x7f38c4155cc0 con 0x7f38a806b4d0 2024-03-08T06:02:56.139 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:02:56.140 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-08T06:02:56.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:56.141+0000 7f38cc5d7640 1 -- 172.21.15.76:0/1119713278 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f38a806b4d0 msgr2=0x7f38a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:56.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:56.141+0000 7f38cc5d7640 1 --2- 172.21.15.76:0/1119713278 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f38a806b4d0 0x7f38a806d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f38b4007fd0 tx=0x7f38b4009040 comp rx=0 tx=0).stop 2024-03-08T06:02:56.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:56.141+0000 7f38cc5d7640 1 -- 172.21.15.76:0/1119713278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c40c98e0 msgr2=0x7f38c414cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:02:56.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:56.141+0000 7f38cc5d7640 1 --2- 172.21.15.76:0/1119713278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c40c98e0 0x7f38c414cea0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f38c00097a0 tx=0x7f38c00046c0 comp rx=0 tx=0).stop 2024-03-08T06:02:56.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:56.141+0000 7f38cc5d7640 1 -- 172.21.15.76:0/1119713278 shutdown_connections 2024-03-08T06:02:56.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:56.141+0000 7f38cc5d7640 1 --2- 172.21.15.76:0/1119713278 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f38a806b4d0 0x7f38a806d990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:56.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:56.141+0000 7f38cc5d7640 1 --2- 172.21.15.76:0/1119713278 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f38c415a9f0 0x7f38c415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:56.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:56.141+0000 7f38cc5d7640 1 --2- 172.21.15.76:0/1119713278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38c40cec20 0x7f38c4159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:56.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:56.141+0000 7f38cc5d7640 1 --2- 172.21.15.76:0/1119713278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c40c98e0 0x7f38c414cea0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:02:56.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:56.141+0000 7f38cc5d7640 1 -- 172.21.15.76:0/1119713278 >> 172.21.15.76:0/1119713278 conn(0x7f38c40cf740 msgr2=0x7f38c40cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:02:56.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:56.141+0000 7f38cc5d7640 1 -- 172.21.15.76:0/1119713278 shutdown_connections 2024-03-08T06:02:56.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:02:56.141+0000 7f38cc5d7640 1 -- 172.21.15.76:0/1119713278 wait complete. 2024-03-08T06:02:56.152 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-03-08T06:02:55.951332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":250612,"kb_used_data":5320,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754724108,"statfs":{"total":773094113280,"available":772837486592,"internally_reserved":0,"allocated":5447680,"data_stored":3051224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2890,"num_objects":-1000,"num_object_clones":0,"num_object_copies":-2000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1000,"num_write_kb":-1000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-14,"num_bytes_recovered":-42,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.861840"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":216,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-08T06:02:49.368893+0000","last_change":"2024-03-08T06:00:26.178419+0000","last_active":"2024-03-08T06:02:49.368893+0000","last_peered":"2024-03-08T06:02:49.368893+0000","last_clean":"2024-03-08T06:02:49.368893+0000","last_became_active":"2024-03-08T06:00:26.175826+0000","last_became_peered":"2024-03-08T06:00:26.175826+0000","last_unstale":"2024-03-08T06:02:49.368893+0000","last_undegraded":"2024-03-08T06:02:49.368893+0000","last_fullsized":"2024-03-08T06:02:49.368893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:00:24.029492+0000","last_clean_scrub_stamp":"2024-03-08T06:00:24.029492+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:36:00.682517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36440,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335400,"statfs":{"total":96636764160,"available":96599449600,"internally_reserved":0,"allocated":1126400,"data_stored":824179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31700,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35864,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335976,"statfs":{"total":96636764160,"available":96600039424,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31704,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":824179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:02:56.154 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-08T06:00:24.029492+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=8, tm_hour=6, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=68, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=8, tm_hour=6, tm_min=2, tm_sec=51, tm_wday=4, tm_yday=68, tm_isdst=0) 2024-03-08T06:02:56.154 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-08T06:03:06.070 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:03:06.073 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:03:16.155 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:03:16.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.265+0000 7f5ffb0ed640 1 Processor -- start 2024-03-08T06:03:16.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.265+0000 7f5ffb0ed640 1 -- start start 2024-03-08T06:03:16.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.265+0000 7f5ffb0ed640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5ff415dde0 0x7f5ff4162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.265+0000 7f5ffb0ed640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5ff40d0b90 0x7f5ff40d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.265+0000 7f5ffb0ed640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff4162790 con 0x7f5ff40b6d00 2024-03-08T06:03:16.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.265+0000 7f5ffb0ed640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff40cf180 con 0x7f5ff40d0b90 2024-03-08T06:03:16.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.265+0000 7f5ffb0ed640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff40d1ed0 con 0x7f5ff415dde0 2024-03-08T06:03:16.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.266+0000 7f5ff968a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5ff40d0b90 0x7f5ff40d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.266+0000 7f5ff968a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5ff40d0b90 0x7f5ff40d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56610/0 (socket says 172.21.15.76:56610) 2024-03-08T06:03:16.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.266+0000 7f5ff968a640 1 -- 172.21.15.76:0/3578078589 learned_addr learned my addr 172.21.15.76:0/3578078589 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:03:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.266+0000 7f5ff8e89640 1 --2- 172.21.15.76:0/3578078589 >> v2:172.21.15.81:3300/0 conn(0x7f5ff415dde0 0x7f5ff4162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.266+0000 7f5ff968a640 1 -- 172.21.15.76:0/3578078589 >> v2:172.21.15.81:3300/0 conn(0x7f5ff415dde0 msgr2=0x7f5ff4162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.266+0000 7f5ff968a640 1 --2- 172.21.15.76:0/3578078589 >> v2:172.21.15.81:3300/0 conn(0x7f5ff415dde0 0x7f5ff4162250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.266+0000 7f5ff968a640 1 -- 172.21.15.76:0/3578078589 >> v1:172.21.15.81:6789/0 conn(0x7f5ff40b6d00 legacy=0x7f5ff40d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.266+0000 7f5ff968a640 1 -- 172.21.15.76:0/3578078589 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff40d21b0 con 0x7f5ff40d0b90 2024-03-08T06:03:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.267+0000 7f5ff968a640 1 --2- 172.21.15.76:0/3578078589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff40d0b90 0x7f5ff40d29b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4162960 tx=0x7f5fec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed885a5d6fe5b1e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.267+0000 7f5ff37fe640 1 -- 172.21.15.76:0/3578078589 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec02ca70 con 0x7f5ff40d0b90 2024-03-08T06:03:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.267+0000 7f5ff37fe640 1 -- 172.21.15.76:0/3578078589 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec02cc10 con 0x7f5ff40d0b90 2024-03-08T06:03:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.267+0000 7f5ff37fe640 1 -- 172.21.15.76:0/3578078589 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0343f0 con 0x7f5ff40d0b90 2024-03-08T06:03:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.267+0000 7f5ffb0ed640 1 -- 172.21.15.76:0/3578078589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff40d0b90 msgr2=0x7f5ff40d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.267+0000 7f5ffb0ed640 1 --2- 172.21.15.76:0/3578078589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff40d0b90 0x7f5ff40d29b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4162960 tx=0x7f5fec02bed0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.267+0000 7f5ffb0ed640 1 -- 172.21.15.76:0/3578078589 shutdown_connections 2024-03-08T06:03:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.267+0000 7f5ffb0ed640 1 --2- 172.21.15.76:0/3578078589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff40d0b90 0x7f5ff40d29b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.267+0000 7f5ffb0ed640 1 --2- 172.21.15.76:0/3578078589 >> v2:172.21.15.81:3300/0 conn(0x7f5ff415dde0 0x7f5ff4162250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.267+0000 7f5ffb0ed640 1 -- 172.21.15.76:0/3578078589 >> 172.21.15.76:0/3578078589 conn(0x7f5ff40b56e0 msgr2=0x7f5ff40b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.267+0000 7f5ffb0ed640 1 -- 172.21.15.76:0/3578078589 shutdown_connections 2024-03-08T06:03:16.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.267+0000 7f5ffb0ed640 1 -- 172.21.15.76:0/3578078589 wait complete. 2024-03-08T06:03:16.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.269+0000 7f5ffb0ed640 1 Processor -- start 2024-03-08T06:03:16.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.269+0000 7f5ffb0ed640 1 -- start start 2024-03-08T06:03:16.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ffb0ed640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5ff40b6d00 0x7f5ff40c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ffb0ed640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ff40d0b90 0x7f5ff4152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ffb0ed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff415dde0 0x7f5ff40b8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ffb0ed640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40d2ef0 con 0x7f5ff415dde0 2024-03-08T06:03:16.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ffb0ed640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40cee20 con 0x7f5ff40d0b90 2024-03-08T06:03:16.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ffb0ed640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40d3070 con 0x7f5ff40b6d00 2024-03-08T06:03:16.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ff968a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff415dde0 0x7f5ff40b8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ff968a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff415dde0 0x7f5ff40b8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56626/0 (socket says 172.21.15.76:56626) 2024-03-08T06:03:16.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ff968a640 1 -- 172.21.15.76:0/2224046370 learned_addr learned my addr 172.21.15.76:0/2224046370 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:03:16.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ff8e89640 1 --2- 172.21.15.76:0/2224046370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5ff40b6d00 0x7f5ff40c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ff3fff640 1 --2- 172.21.15.76:0/2224046370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ff40d0b90 0x7f5ff4152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ff968a640 1 -- 172.21.15.76:0/2224046370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5ff40b6d00 msgr2=0x7f5ff40c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:16.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ff968a640 1 --2- 172.21.15.76:0/2224046370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5ff40b6d00 0x7f5ff40c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ff968a640 1 -- 172.21.15.76:0/2224046370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ff40d0b90 msgr2=0x7f5ff4152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:16.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ff968a640 1 --2- 172.21.15.76:0/2224046370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ff40d0b90 0x7f5ff4152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ff968a640 1 -- 172.21.15.76:0/2224046370 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff415fa50 con 0x7f5ff415dde0 2024-03-08T06:03:16.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ff968a640 1 --2- 172.21.15.76:0/2224046370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff415dde0 0x7f5ff40b8e20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009f10 tx=0x7f5fec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:16.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ff17fa640 1 -- 172.21.15.76:0/2224046370 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec043020 con 0x7f5ff415dde0 2024-03-08T06:03:16.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ff17fa640 1 -- 172.21.15.76:0/2224046370 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec03e450 con 0x7f5ff415dde0 2024-03-08T06:03:16.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ffb0ed640 1 -- 172.21.15.76:0/2224046370 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff415fd30 con 0x7f5ff415dde0 2024-03-08T06:03:16.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ff17fa640 1 -- 172.21.15.76:0/2224046370 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec009510 con 0x7f5ff415dde0 2024-03-08T06:03:16.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.270+0000 7f5ffb0ed640 1 -- 172.21.15.76:0/2224046370 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff41602f0 con 0x7f5ff415dde0 2024-03-08T06:03:16.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.272+0000 7f5ffb0ed640 1 -- 172.21.15.76:0/2224046370 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff404f030 con 0x7f5ff415dde0 2024-03-08T06:03:16.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.272+0000 7f5ff17fa640 1 -- 172.21.15.76:0/2224046370 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fec046020 con 0x7f5ff415dde0 2024-03-08T06:03:16.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.272+0000 7f5ff17fa640 1 --2- 172.21.15.76:0/2224046370 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5fd806b4d0 0x7f5fd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.272+0000 7f5ff17fa640 1 -- 172.21.15.76:0/2224046370 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5fec0ac790 con 0x7f5ff415dde0 2024-03-08T06:03:16.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.272+0000 7f5ff8e89640 1 --2- 172.21.15.76:0/2224046370 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5fd806b4d0 0x7f5fd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.273+0000 7f5ff8e89640 1 --2- 172.21.15.76:0/2224046370 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5fd806b4d0 0x7f5fd806d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8006fd0 tx=0x7f5fe8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:16.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.277+0000 7f5ff17fa640 1 -- 172.21.15.76:0/2224046370 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5fec04d030 con 0x7f5ff415dde0 2024-03-08T06:03:16.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.442+0000 7f5ffb0ed640 1 -- 172.21.15.76:0/2224046370 --> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ff40d1870 con 0x7f5fd806b4d0 2024-03-08T06:03:16.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.443+0000 7f5ff17fa640 1 -- 172.21.15.76:0/2224046370 <== mgr.4107 v2:172.21.15.76:6832/754497070 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13359 (secure 0 0 0) 0x7f5ff40d1870 con 0x7f5fd806b4d0 2024-03-08T06:03:16.444 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:03:16.445 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-08T06:03:16.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.445+0000 7f5ffb0ed640 1 -- 172.21.15.76:0/2224046370 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5fd806b4d0 msgr2=0x7f5fd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:16.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.445+0000 7f5ffb0ed640 1 --2- 172.21.15.76:0/2224046370 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5fd806b4d0 0x7f5fd806d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8006fd0 tx=0x7f5fe8008040 comp rx=0 tx=0).stop 2024-03-08T06:03:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.445+0000 7f5ffb0ed640 1 -- 172.21.15.76:0/2224046370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff415dde0 msgr2=0x7f5ff40b8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.445+0000 7f5ffb0ed640 1 --2- 172.21.15.76:0/2224046370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff415dde0 0x7f5ff40b8e20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009f10 tx=0x7f5fec004a40 comp rx=0 tx=0).stop 2024-03-08T06:03:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.445+0000 7f5ffb0ed640 1 -- 172.21.15.76:0/2224046370 shutdown_connections 2024-03-08T06:03:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.445+0000 7f5ffb0ed640 1 --2- 172.21.15.76:0/2224046370 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f5fd806b4d0 0x7f5fd806d990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.445+0000 7f5ffb0ed640 1 --2- 172.21.15.76:0/2224046370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff415dde0 0x7f5ff40b8e20 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.445+0000 7f5ffb0ed640 1 --2- 172.21.15.76:0/2224046370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ff40d0b90 0x7f5ff4152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.445+0000 7f5ffb0ed640 1 --2- 172.21.15.76:0/2224046370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5ff40b6d00 0x7f5ff40c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.445+0000 7f5ffb0ed640 1 -- 172.21.15.76:0/2224046370 >> 172.21.15.76:0/2224046370 conn(0x7f5ff40b56e0 msgr2=0x7f5ff40b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.445+0000 7f5ffb0ed640 1 -- 172.21.15.76:0/2224046370 shutdown_connections 2024-03-08T06:03:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.445+0000 7f5ffb0ed640 1 -- 172.21.15.76:0/2224046370 wait complete. 2024-03-08T06:03:16.456 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-03-08T06:03:15.955232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":250600,"kb_used_data":5320,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754724120,"statfs":{"total":773094113280,"available":772837498880,"internally_reserved":0,"allocated":5447680,"data_stored":3051224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002262"},"pg_stats":[{"pgid":"1.0","version":"45'119","reported_seq":227,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-08T06:02:56.124353+0000","last_change":"2024-03-08T06:02:56.124404+0000","last_active":"2024-03-08T06:02:56.124353+0000","last_peered":"2024-03-08T06:02:56.124353+0000","last_clean":"2024-03-08T06:02:56.124353+0000","last_became_active":"2024-03-08T06:00:26.175826+0000","last_became_peered":"2024-03-08T06:00:26.175826+0000","last_unstale":"2024-03-08T06:02:56.124353+0000","last_undegraded":"2024-03-08T06:02:56.124353+0000","last_fullsized":"2024-03-08T06:02:56.124353+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'119","last_scrub_stamp":"2024-03-08T06:02:56.124264+0000","last_deep_scrub":"45'119","last_deep_scrub_stamp":"2024-03-08T06:02:56.124264+0000","last_clean_scrub_stamp":"2024-03-08T06:02:56.124264+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:22:30.837728+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335404,"statfs":{"total":96636764160,"available":96599453696,"internally_reserved":0,"allocated":1126400,"data_stored":824179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31700,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35864,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335976,"statfs":{"total":96636764160,"available":96600039424,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31704,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1126400,"data_stored":824179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:03:16.457 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-08T06:03:16.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.557+0000 7f4e34465640 1 Processor -- start 2024-03-08T06:03:16.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.557+0000 7f4e34465640 1 -- start start 2024-03-08T06:03:16.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.558+0000 7f4e34465640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e2c0b9840 0x7f4e2c0b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.558+0000 7f4e34465640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4e2c0b5b20 0x7f4e2c0b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.558+0000 7f4e34465640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e2c0d27d0 con 0x7f4e2c15dbc0 2024-03-08T06:03:16.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.558+0000 7f4e34465640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e2c0b5f90 con 0x7f4e2c0b5b20 2024-03-08T06:03:16.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.558+0000 7f4e34465640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c0d2ab0 con 0x7f4e2c0b9840 2024-03-08T06:03:16.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.558+0000 7f4e31a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e2c0b9840 0x7f4e2c0b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.558+0000 7f4e31a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e2c0b9840 0x7f4e2c0b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.76:45198/0 (socket says 172.21.15.76:45198) 2024-03-08T06:03:16.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.558+0000 7f4e31a00640 1 -- 172.21.15.76:0/1881151336 learned_addr learned my addr 172.21.15.76:0/1881151336 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:03:16.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.558+0000 7f4e32a02640 1 --2- 172.21.15.76:0/1881151336 >> v2:172.21.15.81:3300/0 conn(0x7f4e2c0b5b20 0x7f4e2c0b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.558+0000 7f4e31a00640 1 -- 172.21.15.76:0/1881151336 >> v2:172.21.15.81:3300/0 conn(0x7f4e2c0b5b20 msgr2=0x7f4e2c0b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:16.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.558+0000 7f4e31a00640 1 --2- 172.21.15.76:0/1881151336 >> v2:172.21.15.81:3300/0 conn(0x7f4e2c0b5b20 0x7f4e2c0b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.558+0000 7f4e311ff640 1 -- 172.21.15.76:0/1881151336 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550061422 0 0) 0x7f4e2c0d27d0 con 0x7f4e2c15dbc0 2024-03-08T06:03:16.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.558+0000 7f4e31a00640 1 -- 172.21.15.76:0/1881151336 >> v1:172.21.15.81:6789/0 conn(0x7f4e2c15dbc0 legacy=0x7f4e2c0d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:16.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.558+0000 7f4e31a00640 1 -- 172.21.15.76:0/1881151336 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2c0d2d90 con 0x7f4e2c0b9840 2024-03-08T06:03:16.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.559+0000 7f4e31a00640 1 --2- 172.21.15.76:0/1881151336 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e2c0b9840 0x7f4e2c0b55e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009870 tx=0x7f4e1c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70ec8d5c30b8acab server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:16.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.559+0000 7f4e311ff640 1 -- 172.21.15.76:0/1881151336 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c034070 con 0x7f4e2c0b9840 2024-03-08T06:03:16.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.559+0000 7f4e311ff640 1 -- 172.21.15.76:0/1881151336 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1c030840 con 0x7f4e2c0b9840 2024-03-08T06:03:16.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.559+0000 7f4e311ff640 1 -- 172.21.15.76:0/1881151336 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c030c10 con 0x7f4e2c0b9840 2024-03-08T06:03:16.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.560+0000 7f4e34465640 1 -- 172.21.15.76:0/1881151336 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e2c0b9840 msgr2=0x7f4e2c0b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:16.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.560+0000 7f4e34465640 1 --2- 172.21.15.76:0/1881151336 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e2c0b9840 0x7f4e2c0b55e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009870 tx=0x7f4e1c02b980 comp rx=0 tx=0).stop 2024-03-08T06:03:16.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.560+0000 7f4e34465640 1 -- 172.21.15.76:0/1881151336 shutdown_connections 2024-03-08T06:03:16.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.560+0000 7f4e34465640 1 --2- 172.21.15.76:0/1881151336 >> v2:172.21.15.81:3300/0 conn(0x7f4e2c0b5b20 0x7f4e2c0b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.560+0000 7f4e34465640 1 --2- 172.21.15.76:0/1881151336 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e2c0b9840 0x7f4e2c0b55e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.560+0000 7f4e34465640 1 -- 172.21.15.76:0/1881151336 >> 172.21.15.76:0/1881151336 conn(0x7f4e2c0670d0 msgr2=0x7f4e2c0d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:16.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.560+0000 7f4e34465640 1 -- 172.21.15.76:0/1881151336 shutdown_connections 2024-03-08T06:03:16.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.560+0000 7f4e34465640 1 -- 172.21.15.76:0/1881151336 wait complete. 2024-03-08T06:03:16.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.561+0000 7f4e34465640 1 Processor -- start 2024-03-08T06:03:16.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.561+0000 7f4e34465640 1 -- start start 2024-03-08T06:03:16.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.562+0000 7f4e34465640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e2c0b5b20 0x7f4e2c152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.562+0000 7f4e34465640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e2c0b9840 0x7f4e2c15a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.562+0000 7f4e34465640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e2c15dbc0 0x7f4e2c15c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.562+0000 7f4e34465640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c15dfa0 con 0x7f4e2c0b9840 2024-03-08T06:03:16.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.562+0000 7f4e34465640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c0d3870 con 0x7f4e2c0b5b20 2024-03-08T06:03:16.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.562+0000 7f4e34465640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c15e120 con 0x7f4e2c15dbc0 2024-03-08T06:03:16.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.562+0000 7f4e31a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e2c0b9840 0x7f4e2c15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.562+0000 7f4e31a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e2c0b9840 0x7f4e2c15a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56634/0 (socket says 172.21.15.76:56634) 2024-03-08T06:03:16.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.562+0000 7f4e31a00640 1 -- 172.21.15.76:0/3822185858 learned_addr learned my addr 172.21.15.76:0/3822185858 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:03:16.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.563+0000 7f4e32a02640 1 --2- 172.21.15.76:0/3822185858 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e2c15dbc0 0x7f4e2c15c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.563+0000 7f4e32201640 1 --2- 172.21.15.76:0/3822185858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e2c0b5b20 0x7f4e2c152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.563+0000 7f4e31a00640 1 -- 172.21.15.76:0/3822185858 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e2c15dbc0 msgr2=0x7f4e2c15c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:16.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.563+0000 7f4e31a00640 1 --2- 172.21.15.76:0/3822185858 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e2c15dbc0 0x7f4e2c15c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.563+0000 7f4e31a00640 1 -- 172.21.15.76:0/3822185858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e2c0b5b20 msgr2=0x7f4e2c152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:16.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.563+0000 7f4e31a00640 1 --2- 172.21.15.76:0/3822185858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e2c0b5b20 0x7f4e2c152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.563+0000 7f4e31a00640 1 -- 172.21.15.76:0/3822185858 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2c15aad0 con 0x7f4e2c0b9840 2024-03-08T06:03:16.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.563+0000 7f4e32a02640 1 --2- 172.21.15.76:0/3822185858 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e2c15dbc0 0x7f4e2c15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:03:16.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.563+0000 7f4e31a00640 1 --2- 172.21.15.76:0/3822185858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e2c0b9840 0x7f4e2c15a080 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0090b0 tx=0x7f4e1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:16.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.563+0000 7f4e22ffd640 1 -- 172.21.15.76:0/3822185858 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c034040 con 0x7f4e2c0b9840 2024-03-08T06:03:16.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.563+0000 7f4e22ffd640 1 -- 172.21.15.76:0/3822185858 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1c030640 con 0x7f4e2c0b9840 2024-03-08T06:03:16.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.563+0000 7f4e22ffd640 1 -- 172.21.15.76:0/3822185858 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c039d40 con 0x7f4e2c0b9840 2024-03-08T06:03:16.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.563+0000 7f4e34465640 1 -- 172.21.15.76:0/3822185858 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2c15adb0 con 0x7f4e2c0b9840 2024-03-08T06:03:16.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.563+0000 7f4e34465640 1 -- 172.21.15.76:0/3822185858 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2c15b370 con 0x7f4e2c0b9840 2024-03-08T06:03:16.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.564+0000 7f4e34465640 1 -- 172.21.15.76:0/3822185858 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e2c04f030 con 0x7f4e2c0b9840 2024-03-08T06:03:16.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.564+0000 7f4e22ffd640 1 -- 172.21.15.76:0/3822185858 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e1c04b060 con 0x7f4e2c0b9840 2024-03-08T06:03:16.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.565+0000 7f4e22ffd640 1 --2- 172.21.15.76:0/3822185858 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4e1006b4d0 0x7f4e1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.565+0000 7f4e22ffd640 1 -- 172.21.15.76:0/3822185858 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4e1c0abc70 con 0x7f4e2c0b9840 2024-03-08T06:03:16.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.565+0000 7f4e32201640 1 --2- 172.21.15.76:0/3822185858 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4e1006b4d0 0x7f4e1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.565+0000 7f4e32201640 1 --2- 172.21.15.76:0/3822185858 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4e1006b4d0 0x7f4e1006d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4e24000f60 tx=0x7f4e24009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:16.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.569+0000 7f4e22ffd640 1 -- 172.21.15.76:0/3822185858 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e1c04ce90 con 0x7f4e2c0b9840 2024-03-08T06:03:16.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.740+0000 7f4e34465640 1 -- 172.21.15.76:0/3822185858 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f4e2c0c8790 con 0x7f4e2c0b9840 2024-03-08T06:03:16.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.743+0000 7f4e22ffd640 1 -- 172.21.15.76:0/3822185858 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f4e1c07ae10 con 0x7f4e2c0b9840 2024-03-08T06:03:16.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.744+0000 7f4e22ffd640 1 -- 172.21.15.76:0/3822185858 <== mon.0 v2:172.21.15.76:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4e1c07a710 con 0x7f4e2c0b9840 2024-03-08T06:03:16.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.745+0000 7f4e34465640 1 -- 172.21.15.76:0/3822185858 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4e1006b4d0 msgr2=0x7f4e1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:16.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.746+0000 7f4e34465640 1 --2- 172.21.15.76:0/3822185858 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4e1006b4d0 0x7f4e1006d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4e24000f60 tx=0x7f4e24009040 comp rx=0 tx=0).stop 2024-03-08T06:03:16.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.746+0000 7f4e34465640 1 -- 172.21.15.76:0/3822185858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e2c0b9840 msgr2=0x7f4e2c15a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:16.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.746+0000 7f4e34465640 1 --2- 172.21.15.76:0/3822185858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e2c0b9840 0x7f4e2c15a080 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0090b0 tx=0x7f4e1c0046c0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.746+0000 7f4e34465640 1 -- 172.21.15.76:0/3822185858 shutdown_connections 2024-03-08T06:03:16.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.746+0000 7f4e34465640 1 --2- 172.21.15.76:0/3822185858 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f4e1006b4d0 0x7f4e1006d990 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.746+0000 7f4e34465640 1 --2- 172.21.15.76:0/3822185858 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e2c15dbc0 0x7f4e2c15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.746+0000 7f4e34465640 1 --2- 172.21.15.76:0/3822185858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e2c0b9840 0x7f4e2c15a080 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.746+0000 7f4e34465640 1 --2- 172.21.15.76:0/3822185858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e2c0b5b20 0x7f4e2c152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.746+0000 7f4e34465640 1 -- 172.21.15.76:0/3822185858 >> 172.21.15.76:0/3822185858 conn(0x7f4e2c0670d0 msgr2=0x7f4e2c156060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:16.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.746+0000 7f4e34465640 1 -- 172.21.15.76:0/3822185858 shutdown_connections 2024-03-08T06:03:16.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.746+0000 7f4e34465640 1 -- 172.21.15.76:0/3822185858 wait complete. 2024-03-08T06:03:16.757 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-08T06:03:16.757 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:03:16.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.857+0000 7f21865b5640 1 Processor -- start 2024-03-08T06:03:16.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.857+0000 7f21865b5640 1 -- start start 2024-03-08T06:03:16.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.858+0000 7f21865b5640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f218015dde0 0x7f2180162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.858+0000 7f21865b5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21800b6d00 0x7f21800d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.858+0000 7f21865b5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2180162790 con 0x7f21800cec10 2024-03-08T06:03:16.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.858+0000 7f21865b5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21800cf180 con 0x7f21800b6d00 2024-03-08T06:03:16.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.858+0000 7f21865b5640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21800d1ed0 con 0x7f218015dde0 2024-03-08T06:03:16.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.858+0000 7f217f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21800b6d00 0x7f21800d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.858+0000 7f2184b52640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f21800cec10 0x7f218006c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52854/0 (socket says 172.21.15.76:52854) 2024-03-08T06:03:16.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.858+0000 7f2184b52640 1 -- 172.21.15.76:0/1330291532 learned_addr learned my addr 172.21.15.76:0/1330291532 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-03-08T06:03:16.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.859+0000 7f217ffff640 1 --2- 172.21.15.76:0/1330291532 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f218015dde0 0x7f2180162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.859+0000 7f217f7fe640 1 -- 172.21.15.76:0/1330291532 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f218015dde0 msgr2=0x7f2180162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:16.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.859+0000 7f217f7fe640 1 --2- 172.21.15.76:0/1330291532 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f218015dde0 0x7f2180162250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.859+0000 7f217f7fe640 1 -- 172.21.15.76:0/1330291532 >> v1:172.21.15.76:6789/0 conn(0x7f21800cec10 legacy=0x7f218006c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:16.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.859+0000 7f217effd640 1 -- 172.21.15.76:0/1330291532 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457817318 0 0) 0x7f2180162790 con 0x7f21800cec10 2024-03-08T06:03:16.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.859+0000 7f217f7fe640 1 -- 172.21.15.76:0/1330291532 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800d21b0 con 0x7f21800b6d00 2024-03-08T06:03:16.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.859+0000 7f217f7fe640 1 --2- 172.21.15.76:0/1330291532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21800b6d00 0x7f21800d0480 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f21740097a0 tx=0x7f217402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcdd56705b8e81 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:16.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.860+0000 7f217effd640 1 -- 172.21.15.76:0/1330291532 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21740315a0 con 0x7f21800b6d00 2024-03-08T06:03:16.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.860+0000 7f217effd640 1 -- 172.21.15.76:0/1330291532 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2174037070 con 0x7f21800b6d00 2024-03-08T06:03:16.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.860+0000 7f217effd640 1 -- 172.21.15.76:0/1330291532 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174031980 con 0x7f21800b6d00 2024-03-08T06:03:16.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.860+0000 7f21865b5640 1 -- 172.21.15.76:0/1330291532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21800b6d00 msgr2=0x7f21800d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:16.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.860+0000 7f21865b5640 1 --2- 172.21.15.76:0/1330291532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21800b6d00 0x7f21800d0480 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f21740097a0 tx=0x7f217402b840 comp rx=0 tx=0).stop 2024-03-08T06:03:16.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.860+0000 7f21865b5640 1 -- 172.21.15.76:0/1330291532 shutdown_connections 2024-03-08T06:03:16.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.860+0000 7f21865b5640 1 --2- 172.21.15.76:0/1330291532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21800b6d00 0x7f21800d0480 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.860+0000 7f21865b5640 1 --2- 172.21.15.76:0/1330291532 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f218015dde0 0x7f2180162250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.860+0000 7f21865b5640 1 -- 172.21.15.76:0/1330291532 >> 172.21.15.76:0/1330291532 conn(0x7f21800b56e0 msgr2=0x7f21800b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:16.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.860+0000 7f21865b5640 1 -- 172.21.15.76:0/1330291532 shutdown_connections 2024-03-08T06:03:16.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.860+0000 7f21865b5640 1 -- 172.21.15.76:0/1330291532 wait complete. 2024-03-08T06:03:16.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.861+0000 7f21865b5640 1 Processor -- start 2024-03-08T06:03:16.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.861+0000 7f21865b5640 1 -- start start 2024-03-08T06:03:16.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.862+0000 7f21865b5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21800b6d00 0x7f218014f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.862+0000 7f21865b5640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f21800cec10 0x7f21800b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.862+0000 7f21865b5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f218015dde0 0x7f21800b9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.862+0000 7f21865b5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21800d0dd0 con 0x7f218015dde0 2024-03-08T06:03:16.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.862+0000 7f21865b5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21800d0c50 con 0x7f21800b6d00 2024-03-08T06:03:16.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.862+0000 7f21865b5640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218006c8a0 con 0x7f21800cec10 2024-03-08T06:03:16.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.862+0000 7f2184b52640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f218015dde0 0x7f21800b9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.862+0000 7f2184b52640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f218015dde0 0x7f21800b9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56648/0 (socket says 172.21.15.76:56648) 2024-03-08T06:03:16.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.862+0000 7f2184b52640 1 -- 172.21.15.76:0/298443780 learned_addr learned my addr 172.21.15.76:0/298443780 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-08T06:03:16.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.862+0000 7f217f7fe640 1 --2- 172.21.15.76:0/298443780 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f21800cec10 0x7f21800b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.862+0000 7f217ffff640 1 --2- 172.21.15.76:0/298443780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21800b6d00 0x7f218014f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.863+0000 7f2184b52640 1 -- 172.21.15.76:0/298443780 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f21800cec10 msgr2=0x7f21800b8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:16.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.863+0000 7f2184b52640 1 --2- 172.21.15.76:0/298443780 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f21800cec10 0x7f21800b8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.863+0000 7f2184b52640 1 -- 172.21.15.76:0/298443780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21800b6d00 msgr2=0x7f218014f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:16.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.863+0000 7f2184b52640 1 --2- 172.21.15.76:0/298443780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21800b6d00 0x7f218014f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:16.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.863+0000 7f2184b52640 1 -- 172.21.15.76:0/298443780 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21801533a0 con 0x7f218015dde0 2024-03-08T06:03:16.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.863+0000 7f2184b52640 1 --2- 172.21.15.76:0/298443780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f218015dde0 0x7f21800b9160 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f217800cb30 tx=0x7f217800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:16.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.863+0000 7f217cff9640 1 -- 172.21.15.76:0/298443780 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178017070 con 0x7f218015dde0 2024-03-08T06:03:16.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.863+0000 7f217cff9640 1 -- 172.21.15.76:0/298443780 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f217800acc0 con 0x7f218015dde0 2024-03-08T06:03:16.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.863+0000 7f21865b5640 1 -- 172.21.15.76:0/298443780 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218015e340 con 0x7f218015dde0 2024-03-08T06:03:16.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.863+0000 7f217cff9640 1 -- 172.21.15.76:0/298443780 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178012660 con 0x7f218015dde0 2024-03-08T06:03:16.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.863+0000 7f21865b5640 1 -- 172.21.15.76:0/298443780 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218015e900 con 0x7f218015dde0 2024-03-08T06:03:16.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.864+0000 7f21865b5640 1 -- 172.21.15.76:0/298443780 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f218004f030 con 0x7f218015dde0 2024-03-08T06:03:16.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.864+0000 7f217cff9640 1 -- 172.21.15.76:0/298443780 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f217801a070 con 0x7f218015dde0 2024-03-08T06:03:16.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.865+0000 7f217cff9640 1 --2- 172.21.15.76:0/298443780 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f216c06b4d0 0x7f216c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:16.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.865+0000 7f217cff9640 1 -- 172.21.15.76:0/298443780 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2178089ed0 con 0x7f218015dde0 2024-03-08T06:03:16.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.865+0000 7f217ffff640 1 --2- 172.21.15.76:0/298443780 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f216c06b4d0 0x7f216c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:16.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.865+0000 7f217ffff640 1 --2- 172.21.15.76:0/298443780 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f216c06b4d0 0x7f216c06d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f21700079a0 tx=0x7f2170008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:16.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:16.869+0000 7f217cff9640 1 -- 172.21.15.76:0/298443780 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21801533a0 con 0x7f218015dde0 2024-03-08T06:03:17.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:17.078+0000 7f21865b5640 1 -- 172.21.15.76:0/298443780 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21800cd0f0 con 0x7f218015dde0 2024-03-08T06:03:17.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:17.078+0000 7f217cff9640 1 -- 172.21.15.76:0/298443780 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+3693 (secure 0 0 0) 0x7f21780587e0 con 0x7f218015dde0 2024-03-08T06:03:17.079 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-08T06:03:17.079 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":17,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/606148582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6833","nonce":606148582},{"type":"v1","addr":"172.21.15.81:6835","nonce":606148582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5332,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6834/1130946465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1130946465},{"type":"v1","addr":"172.21.15.81:6834","nonce":1130946465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/278988477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":278988477},{"type":"v1","addr":"172.21.15.76:6836","nonce":278988477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2526423312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":2526423312},{"type":"v1","addr":"172.21.15.76:6837","nonce":2526423312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2024-03-08T06:03:17.080 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 17 2024-03-08T06:03:17.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:17.080+0000 7f21865b5640 1 -- 172.21.15.76:0/298443780 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f216c06b4d0 msgr2=0x7f216c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:17.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:17.080+0000 7f21865b5640 1 --2- 172.21.15.76:0/298443780 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f216c06b4d0 0x7f216c06d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f21700079a0 tx=0x7f2170008040 comp rx=0 tx=0).stop 2024-03-08T06:03:17.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:17.080+0000 7f21865b5640 1 -- 172.21.15.76:0/298443780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f218015dde0 msgr2=0x7f21800b9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:17.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:17.080+0000 7f21865b5640 1 --2- 172.21.15.76:0/298443780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f218015dde0 0x7f21800b9160 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f217800cb30 tx=0x7f217800a060 comp rx=0 tx=0).stop 2024-03-08T06:03:17.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:17.080+0000 7f21865b5640 1 -- 172.21.15.76:0/298443780 shutdown_connections 2024-03-08T06:03:17.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:17.080+0000 7f21865b5640 1 --2- 172.21.15.76:0/298443780 >> [v2:172.21.15.76:6832/754497070,v1:172.21.15.76:6833/754497070] conn(0x7f216c06b4d0 0x7f216c06d990 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:17.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:17.080+0000 7f21865b5640 1 --2- 172.21.15.76:0/298443780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f218015dde0 0x7f21800b9160 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:17.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:17.080+0000 7f21865b5640 1 --2- 172.21.15.76:0/298443780 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f21800cec10 0x7f21800b8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:17.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:17.080+0000 7f21865b5640 1 --2- 172.21.15.76:0/298443780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21800b6d00 0x7f218014f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:17.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:17.080+0000 7f21865b5640 1 -- 172.21.15.76:0/298443780 >> 172.21.15.76:0/298443780 conn(0x7f21800b56e0 msgr2=0x7f21800b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:17.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:17.080+0000 7f21865b5640 1 -- 172.21.15.76:0/298443780 shutdown_connections 2024-03-08T06:03:17.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-08T06:03:17.080+0000 7f21865b5640 1 -- 172.21.15.76:0/298443780 wait complete. 2024-03-08T06:03:17.092 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-08T06:03:17.092 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-08T06:03:17.092 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-08T06:03:17.093 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.101 INFO:tasks.ceph.mds.a:Stopped 2024-03-08T06:03:17.102 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-08T06:03:17.102 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.152 INFO:tasks.ceph.mds.c:Stopped 2024-03-08T06:03:17.152 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-08T06:03:17.153 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.165 INFO:tasks.ceph.mds.b:Stopped 2024-03-08T06:03:17.165 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-08T06:03:17.165 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.217 INFO:tasks.ceph.mds.d:Stopped 2024-03-08T06:03:17.217 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-08T06:03:17.217 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-08T06:03:17.217 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.243 INFO:tasks.ceph.osd.0:Stopped 2024-03-08T06:03:17.244 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-08T06:03:17.244 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.308 INFO:tasks.ceph.osd.1:Stopped 2024-03-08T06:03:17.308 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-08T06:03:17.308 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.383 INFO:tasks.ceph.osd.2:Stopped 2024-03-08T06:03:17.383 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-08T06:03:17.383 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.451 INFO:tasks.ceph.osd.3:Stopped 2024-03-08T06:03:17.452 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-08T06:03:17.452 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.490 INFO:tasks.ceph.osd.4:Stopped 2024-03-08T06:03:17.491 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-08T06:03:17.491 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.561 INFO:tasks.ceph.osd.5:Stopped 2024-03-08T06:03:17.561 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-08T06:03:17.562 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.640 INFO:tasks.ceph.osd.6:Stopped 2024-03-08T06:03:17.640 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-08T06:03:17.640 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.714 INFO:tasks.ceph.osd.7:Stopped 2024-03-08T06:03:17.714 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-08T06:03:17.715 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-08T06:03:17.715 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.740 INFO:tasks.ceph.mgr.y:Stopped 2024-03-08T06:03:17.741 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-08T06:03:17.741 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.781 INFO:tasks.ceph.mgr.x:Stopped 2024-03-08T06:03:17.782 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-08T06:03:17.782 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-08T06:03:17.782 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.793 INFO:tasks.ceph.mon.a:Stopped 2024-03-08T06:03:17.794 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-08T06:03:17.794 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.833 INFO:tasks.ceph.mon.b:Stopped 2024-03-08T06:03:17.834 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-08T06:03:17.834 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:03:17.887 INFO:tasks.ceph.mon.c:Stopped 2024-03-08T06:03:17.887 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-08T06:03:17.887 DEBUG:teuthology.orchestra.run.smithi076:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'missing required features' | head -n 1 2024-03-08T06:03:17.939 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi076.front.sepia.ceph.com 2024-03-08T06:03:17.939 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-08T06:03:18.139 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi076.front.sepia.ceph.com 2024-03-08T06:03:18.139 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-08T06:03:18.220 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi076.front.sepia.ceph.com 2024-03-08T06:03:18.220 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-08T06:03:18.285 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi076.front.sepia.ceph.com 2024-03-08T06:03:18.286 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-08T06:03:18.353 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi081.front.sepia.ceph.com 2024-03-08T06:03:18.353 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-08T06:03:18.476 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi081.front.sepia.ceph.com 2024-03-08T06:03:18.476 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-08T06:03:18.587 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi081.front.sepia.ceph.com 2024-03-08T06:03:18.587 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-08T06:03:18.637 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi081.front.sepia.ceph.com 2024-03-08T06:03:18.638 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-08T06:03:18.687 INFO:tasks.ceph:Archiving mon data... 2024-03-08T06:03:18.689 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417/data/mon.a.tgz 2024-03-08T06:03:18.689 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2024-03-08T06:03:18.705 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.0Vbd01zzFZ 2024-03-08T06:03:18.706 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.0Vbd01zzFZ 2024-03-08T06:03:18.840 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.0Vbd01zzFZ 2024-03-08T06:03:18.926 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.0Vbd01zzFZ is 306KB 2024-03-08T06:03:19.024 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.0Vbd01zzFZ 2024-03-08T06:03:19.039 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417/data/mon.b.tgz 2024-03-08T06:03:19.039 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-03-08T06:03:19.055 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.5zBucVBEeX 2024-03-08T06:03:19.055 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.5zBucVBEeX 2024-03-08T06:03:19.206 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.5zBucVBEeX 2024-03-08T06:03:19.290 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.5zBucVBEeX is 405KB 2024-03-08T06:03:19.387 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.5zBucVBEeX 2024-03-08T06:03:19.401 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417/data/mon.c.tgz 2024-03-08T06:03:19.401 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-03-08T06:03:19.455 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.Y9vIzgVrad 2024-03-08T06:03:19.456 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Y9vIzgVrad 2024-03-08T06:03:19.596 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.Y9vIzgVrad 2024-03-08T06:03:19.680 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.Y9vIzgVrad is 405KB 2024-03-08T06:03:19.779 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.Y9vIzgVrad 2024-03-08T06:03:19.794 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-08T06:03:19.794 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-08T06:03:19.796 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-08T06:03:19.879 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-08T06:03:19.915 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-08T06:03:19.915 INFO:tasks.ceph:Archiving crash dumps... 2024-03-08T06:03:19.916 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/crash to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417/remote/smithi076/crash 2024-03-08T06:03:19.917 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-08T06:03:19.947 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/crash to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417/remote/smithi081/crash 2024-03-08T06:03:19.948 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-08T06:03:19.981 INFO:tasks.ceph:Shutting down logrotate 2024-03-08T06:03:19.981 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:03:19.992 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:03:20.081 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-08T06:03:20.108 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-08T06:03:20.135 INFO:tasks.ceph:Compressing logs... 2024-03-08T06:03:20.135 DEBUG:teuthology.orchestra.run.smithi076:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-08T06:03:20.151 DEBUG:teuthology.orchestra.run.smithi081:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-08T06:03:20.223 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-08T06:03:20.223 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37796.log 2024-03-08T06:03:20.224 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44379.log 2024-03-08T06:03:20.224 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.37796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37836.log 2024-03-08T06:03:20.224 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.1.44379.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37796.log.gz 2024-03-08T06:03:20.224 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.37676.log 2024-03-08T06:03:20.224 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.1.44379.log.gz 2024-03-08T06:03:20.224 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-08T06:03:20.224 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37836.log: /var/log/ceph/ceph-client.admin.37676.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-08T06:03:20.225 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37836.log.gz 2024-03-08T06:03:20.225 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37676.log.gz 2024-03-08T06:03:20.225 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-08T06:03:20.225 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37756.log 2024-03-08T06:03:20.225 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-08T06:03:20.225 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37756.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-08T06:03:20.225 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-osd.7.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37756.log.gz 2024-03-08T06:03:20.225 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37716.log 2024-03-08T06:03:20.226 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37596.log 2024-03-08T06:03:20.226 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37716.log: gzip -5 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-08T06:03:20.226 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.3.43777.log 2024-03-08T06:03:20.226 INFO:teuthology.orchestra.run.smithi081.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37716.log.gz 2024-03-08T06:03:20.226 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37596.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37596.log.gz 2024-03-08T06:03:20.226 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-08T06:03:20.226 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.3.43777.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.3.43777.log.gz 2024-03-08T06:03:20.226 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph.log 2024-03-08T06:03:20.227 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-08T06:03:20.227 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43388.log 2024-03-08T06:03:20.227 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-mon.b.log: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-08T06:03:20.227 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-08T06:03:20.227 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.1.43388.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43388.log.gz 2024-03-08T06:03:20.227 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37556.log 2024-03-08T06:03:20.227 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-mds.d.log: 92.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-08T06:03:20.231 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37636.log 2024-03-08T06:03:20.232 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37556.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37556.log.gz 2024-03-08T06:03:20.243 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-08T06:03:20.243 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37636.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37636.log.gz 2024-03-08T06:03:20.248 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-mon.c.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-08T06:03:20.297 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50702.log 2024-03-08T06:03:20.297 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51500.log 2024-03-08T06:03:20.297 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52520.log 2024-03-08T06:03:20.297 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43765.log 2024-03-08T06:03:20.297 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50702.log.gz 2024-03-08T06:03:20.298 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51500.log.gz 2024-03-08T06:03:20.298 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52520.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-08T06:03:20.298 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43765.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52520.log.gz 2024-03-08T06:03:20.298 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52060.log 2024-03-08T06:03:20.298 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49379.log 2024-03-08T06:03:20.298 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43765.log.gz 2024-03-08T06:03:20.298 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49925.log 2024-03-08T06:03:20.298 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52060.log.gz 2024-03-08T06:03:20.298 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55988.log 2024-03-08T06:03:20.299 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49925.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49379.log.gz 2024-03-08T06:03:20.299 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50372.log 2024-03-08T06:03:20.299 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55988.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49925.log.gz -5 2024-03-08T06:03:20.299 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47824.log 2024-03-08T06:03:20.299 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4%/var/log/ceph/ceph-client.admin.50372.log: -- replaced with /var/log/ceph/ceph-client.admin.55988.log.gz 2024-03-08T06:03:20.299 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54114.log 2024-03-08T06:03:20.299 INFO:teuthology.orchestra.run.smithi076.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50372.log.gz 2024-03-08T06:03:20.299 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49589.log 2024-03-08T06:03:20.300 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55286.log 2024-03-08T06:03:20.300 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54114.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47824.log.gz 2024-03-08T06:03:20.300 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47698.log 2024-03-08T06:03:20.300 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54114.log.gz 2024-03-08T06:03:20.300 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49589.log.gz 2024-03-08T06:03:20.300 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47260.log 2024-03-08T06:03:20.300 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47698.log: 87.6%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.55286.log.gz 2024-03-08T06:03:20.300 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51164.log 2024-03-08T06:03:20.301 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47260.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47698.log.gz 2024-03-08T06:03:20.301 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50093.log 2024-03-08T06:03:20.301 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47260.log.gz 2024-03-08T06:03:20.301 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43461.log 2024-03-08T06:03:20.301 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46573.log 2024-03-08T06:03:20.301 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51164.log.gz 2024-03-08T06:03:20.301 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46936.log 2024-03-08T06:03:20.301 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50093.log.gz 2024-03-08T06:03:20.301 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43461.log.gz 2024-03-08T06:03:20.302 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50414.log 2024-03-08T06:03:20.302 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46573.log.gz 2024-03-08T06:03:20.302 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56028.log 2024-03-08T06:03:20.302 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46936.log: /var/log/ceph/ceph-client.admin.50414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54466.log 2024-03-08T06:03:20.302 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46936.log.gz 2024-03-08T06:03:20.302 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50414.log.gz 2024-03-08T06:03:20.302 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.56028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46937.log 2024-03-08T06:03:20.302 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48989.log 2024-03-08T06:03:20.302 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56028.log.gz 2024-03-08T06:03:20.303 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46937.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54466.log.gz 2024-03-08T06:03:20.303 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49799.log 2024-03-08T06:03:20.303 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46937.log.gz 2024-03-08T06:03:20.303 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55202.log 2024-03-08T06:03:20.303 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48989.log: /var/log/ceph/ceph-client.admin.49799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54382.log 2024-03-08T06:03:20.303 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48989.log.gz 2024-03-08T06:03:20.303 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49799.log.gz 2024-03-08T06:03:20.303 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49673.log 2024-03-08T06:03:20.303 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44293.log 2024-03-08T06:03:20.304 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55202.log.gz 2024-03-08T06:03:20.304 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49673.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54382.log.gz 2024-03-08T06:03:20.304 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37435.log 2024-03-08T06:03:20.304 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49673.log.gz 2024-03-08T06:03:20.304 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55076.log 2024-03-08T06:03:20.304 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44293.log.gz 2024-03-08T06:03:20.304 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45403.log 2024-03-08T06:03:20.304 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52704.log 2024-03-08T06:03:20.305 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37435.log.gz 2024-03-08T06:03:20.305 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55076.log.gz 2024-03-08T06:03:20.305 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46247.log 2024-03-08T06:03:20.305 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45403.log.gz 2024-03-08T06:03:20.305 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49841.log 2024-03-08T06:03:20.305 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45649.log 2024-03-08T06:03:20.305 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52704.log.gz 2024-03-08T06:03:20.305 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49841.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46247.log.gz 2024-03-08T06:03:20.306 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51584.log 2024-03-08T06:03:20.306 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49841.log.gz 2024-03-08T06:03:20.306 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52914.log 2024-03-08T06:03:20.306 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51584.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45649.log.gz 2024-03-08T06:03:20.306 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49124.log 2024-03-08T06:03:20.306 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52914.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51584.log.gz 2024-03-08T06:03:20.306 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50492.log 2024-03-08T06:03:20.306 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52914.log.gz 2024-03-08T06:03:20.307 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52352.log 2024-03-08T06:03:20.307 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50492.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49124.log.gz 2024-03-08T06:03:20.307 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45943.log 2024-03-08T06:03:20.307 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50492.log.gz 2024-03-08T06:03:20.307 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51854.log 2024-03-08T06:03:20.307 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45943.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52352.log.gz 2024-03-08T06:03:20.307 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46584.log 2024-03-08T06:03:20.307 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51854.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45943.log.gz 2024-03-08T06:03:20.307 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47176.log 2024-03-08T06:03:20.308 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5%/var/log/ceph/ceph-client.admin.46584.log: -- replaced with /var/log/ceph/ceph-client.admin.51854.log.gz 2024-03-08T06:03:20.308 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37538.log 2024-03-08T06:03:20.308 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5%/var/log/ceph/ceph-client.admin.47176.log: -- replaced with /var/log/ceph/ceph-client.admin.46584.log.gz 2024-03-08T06:03:20.308 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46243.log 2024-03-08T06:03:20.308 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46250.log 2024-03-08T06:03:20.308 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47176.log.gz 2024-03-08T06:03:20.308 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46243.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37538.log.gz 2024-03-08T06:03:20.308 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47218.log 2024-03-08T06:03:20.309 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46243.log.gz 2024-03-08T06:03:20.309 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51290.log 2024-03-08T06:03:20.309 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47218.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46250.log.gz 2024-03-08T06:03:20.309 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49757.log 2024-03-08T06:03:20.309 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51290.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47218.log.gz 2024-03-08T06:03:20.309 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54886.log 2024-03-08T06:03:20.309 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51290.log.gz 2024-03-08T06:03:20.309 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50618.log 2024-03-08T06:03:20.310 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54886.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49757.log.gz 2024-03-08T06:03:20.310 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55370.log 2024-03-08T06:03:20.310 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50618.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54886.log.gz 2024-03-08T06:03:20.310 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49295.log 2024-03-08T06:03:20.310 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50618.log.gz 2024-03-08T06:03:20.310 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52872.log 2024-03-08T06:03:20.310 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48030.log 2024-03-08T06:03:20.310 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55370.log.gz 2024-03-08T06:03:20.311 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52872.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49295.log.gz 2024-03-08T06:03:20.311 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55580.log 2024-03-08T06:03:20.311 INFO:teuthology.orchestra.run.smithi076.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52872.log.gz 2024-03-08T06:03:20.311 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54198.log 2024-03-08T06:03:20.311 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55580.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48030.log.gz 2024-03-08T06:03:20.311 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43638.log 2024-03-08T06:03:20.311 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55580.log.gz 2024-03-08T06:03:20.311 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47488.log 2024-03-08T06:03:20.312 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43638.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54198.log.gz 2024-03-08T06:03:20.312 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.48775.log 2024-03-08T06:03:20.312 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47488.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43638.log.gz 2024-03-08T06:03:20.312 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53952.log 2024-03-08T06:03:20.312 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.2.48775.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47488.log.gz 2024-03-08T06:03:20.312 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.51728.log -- replaced with /var/log/ceph/ceph-client.2.48775.log.gz 2024-03-08T06:03:20.312 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-03-08T06:03:20.312 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51122.log 2024-03-08T06:03:20.312 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48905.log 2024-03-08T06:03:20.313 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53952.log.gz 2024-03-08T06:03:20.313 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51122.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51728.log.gz 2024-03-08T06:03:20.313 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45261.log 2024-03-08T06:03:20.313 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51122.log.gz 2024-03-08T06:03:20.313 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47134.log 2024-03-08T06:03:20.313 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45261.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48905.log.gz 2024-03-08T06:03:20.313 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55664.log 2024-03-08T06:03:20.313 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45261.log.gz/var/log/ceph/ceph-client.admin.47134.log: 2024-03-08T06:03:20.313 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45219.log 2024-03-08T06:03:20.314 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47134.log.gz 2024-03-08T06:03:20.314 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50009.log 2024-03-08T06:03:20.314 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37181.log 2024-03-08T06:03:20.314 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55664.log.gz 2024-03-08T06:03:20.314 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50009.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45219.log.gz 2024-03-08T06:03:20.314 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51206.log 2024-03-08T06:03:20.314 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50009.log.gz 2024-03-08T06:03:20.314 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46245.log 2024-03-08T06:03:20.315 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51206.log: gzip 89.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.37181.log.gz --verbose 2024-03-08T06:03:20.315 INFO:teuthology.orchestra.run.smithi076.stderr: -- /var/log/ceph/ceph-client.admin.44039.log 2024-03-08T06:03:20.315 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46245.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51206.log.gz 2024-03-08T06:03:20.315 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43807.log 2024-03-08T06:03:20.315 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46245.log.gz 2024-03-08T06:03:20.315 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46248.log 2024-03-08T06:03:20.315 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49421.log 2024-03-08T06:03:20.315 INFO:teuthology.orchestra.run.smithi076.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44039.log.gz 2024-03-08T06:03:20.315 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46248.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43807.log.gz 2024-03-08T06:03:20.316 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54030.log 2024-03-08T06:03:20.316 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49421.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46248.log.gz 2024-03-08T06:03:20.316 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45985.log 2024-03-08T06:03:20.316 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49421.log.gz 2024-03-08T06:03:20.316 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45901.log 2024-03-08T06:03:20.316 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45985.log: gzip -5 88.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.54030.log.gz /var/log/ceph/ceph-client.admin.49032.log 2024-03-08T06:03:20.316 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-03-08T06:03:20.316 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45985.log.gz 2024-03-08T06:03:20.316 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46578.log 2024-03-08T06:03:20.317 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45901.log.gz 2024-03-08T06:03:20.317 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49883.log 2024-03-08T06:03:20.317 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46578.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.49032.log.gz -5 2024-03-08T06:03:20.317 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47302.log 2024-03-08T06:03:20.317 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46578.log.gz 2024-03-08T06:03:20.317 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52436.log 2024-03-08T06:03:20.317 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47302.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49883.log.gz 2024-03-08T06:03:20.317 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-08T06:03:20.317 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47302.log.gz 2024-03-08T06:03:20.318 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47344.log 2024-03-08T06:03:20.318 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52436.log.gz 2024-03-08T06:03:20.318 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51770.log 2024-03-08T06:03:20.318 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52746.log 2024-03-08T06:03:20.318 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51770.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47344.log.gz 2024-03-08T06:03:20.318 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44335.log 2024-03-08T06:03:20.318 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51770.log.gz 2024-03-08T06:03:20.318 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50051.log 2024-03-08T06:03:20.319 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52746.log: /var/log/ceph/ceph-client.admin.44335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46246.log 2024-03-08T06:03:20.319 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52746.log.gz 2024-03-08T06:03:20.319 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-08T06:03:20.319 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44335.log.gz 2024-03-08T06:03:20.319 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49967.log 2024-03-08T06:03:20.319 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46246.log: -- /var/log/ceph/ceph-client.admin.51980.log 2024-03-08T06:03:20.319 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50051.log.gz 2024-03-08T06:03:20.319 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51458.log 2024-03-08T06:03:20.319 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46246.log.gz 2024-03-08T06:03:20.320 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49967.log.gz 2024-03-08T06:03:20.320 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55118.log 2024-03-08T06:03:20.320 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51458.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51980.log.gz 2024-03-08T06:03:20.320 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51626.log 2024-03-08T06:03:20.320 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51458.log.gz 2024-03-08T06:03:20.320 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47428.log 2024-03-08T06:03:20.320 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55118.log.gz 2024-03-08T06:03:20.320 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46930.log 2024-03-08T06:03:20.321 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55874.log 2024-03-08T06:03:20.321 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51626.log.gz 2024-03-08T06:03:20.321 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46930.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47428.log.gz 2024-03-08T06:03:20.321 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46934.log 2024-03-08T06:03:20.321 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46930.log.gz 2024-03-08T06:03:20.321 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49253.log 2024-03-08T06:03:20.321 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47614.log 2024-03-08T06:03:20.321 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55874.log.gz 2024-03-08T06:03:20.322 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49253.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46934.log.gz 2024-03-08T06:03:20.322 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46586.log 2024-03-08T06:03:20.322 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49253.log.gz 2024-03-08T06:03:20.322 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48475.log 2024-03-08T06:03:20.322 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49547.log 2024-03-08T06:03:20.322 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47614.log.gz 2024-03-08T06:03:20.322 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46586.log.gz 2024-03-08T06:03:20.322 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44251.log 2024-03-08T06:03:20.322 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48475.log.gz 2024-03-08T06:03:20.323 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52310.log 2024-03-08T06:03:20.323 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44125.log 2024-03-08T06:03:20.323 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49547.log.gz 2024-03-08T06:03:20.323 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44251.log.gz 2024-03-08T06:03:20.323 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51416.log 2024-03-08T06:03:20.323 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52310.log.gz 2024-03-08T06:03:20.323 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46027.log 2024-03-08T06:03:20.323 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50246.log 2024-03-08T06:03:20.323 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44125.log.gz 2024-03-08T06:03:20.324 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51416.log.gz 2024-03-08T06:03:20.324 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55748.log 2024-03-08T06:03:20.324 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46027.log.gz 2024-03-08T06:03:20.324 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50918.log 2024-03-08T06:03:20.324 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55748.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50246.log.gz 2024-03-08T06:03:20.324 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44419.log 2024-03-08T06:03:20.324 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55748.log.gz 2024-03-08T06:03:20.324 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45487.log 2024-03-08T06:03:20.324 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50918.log.gz 2024-03-08T06:03:20.325 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44419.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-08T06:03:20.325 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54508.log 2024-03-08T06:03:20.325 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45487.log.gz 2024-03-08T06:03:20.325 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45859.log 2024-03-08T06:03:20.325 INFO:teuthology.orchestra.run.smithi076.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44419.log.gz 2024-03-08T06:03:20.325 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46935.log 2024-03-08T06:03:20.325 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54508.log: /var/log/ceph/ceph-client.admin.45859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45361.log 2024-03-08T06:03:20.325 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54508.log.gz 2024-03-08T06:03:20.325 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45859.log.gz 2024-03-08T06:03:20.325 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53497.log 2024-03-08T06:03:20.326 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52956.log 2024-03-08T06:03:20.326 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45361.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46935.log.gz 2024-03-08T06:03:20.326 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55160.log 2024-03-08T06:03:20.326 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45361.log.gz 2024-03-08T06:03:20.326 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54844.log 2024-03-08T06:03:20.326 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52956.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.53497.log.gz 2024-03-08T06:03:20.326 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52562.log 2024-03-08T06:03:20.326 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52956.log.gz 2024-03-08T06:03:20.326 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50792.log 2024-03-08T06:03:20.327 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54844.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55160.log.gz 2024-03-08T06:03:20.327 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52562.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.54844.log.gz 2024-03-08T06:03:20.327 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52394.log 2024-03-08T06:03:20.327 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52562.log.gz 2024-03-08T06:03:20.327 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45775.log 2024-03-08T06:03:20.327 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50792.log: /var/log/ceph/ceph-client.admin.52394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43557.log 2024-03-08T06:03:20.327 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50792.log.gz 2024-03-08T06:03:20.327 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52394.log.gz 2024-03-08T06:03:20.327 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54634.log 2024-03-08T06:03:20.328 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45775.log.gz 2024-03-08T06:03:20.328 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45529.log 2024-03-08T06:03:20.328 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54760.log 2024-03-08T06:03:20.328 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54634.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43557.log.gz 2024-03-08T06:03:20.328 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55328.log 2024-03-08T06:03:20.328 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54634.log.gz 2024-03-08T06:03:20.328 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46249.log 2024-03-08T06:03:20.328 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45529.log.gz 2024-03-08T06:03:20.329 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54760.log: /var/log/ceph/ceph-client.admin.55328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47740.log 2024-03-08T06:03:20.329 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54760.log.gz 2024-03-08T06:03:20.329 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55328.log.gz 2024-03-08T06:03:20.329 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54550.log 2024-03-08T06:03:20.329 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47386.log 2024-03-08T06:03:20.329 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46249.log.gz 2024-03-08T06:03:20.329 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54550.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47740.log.gz 2024-03-08T06:03:20.329 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51080.log 2024-03-08T06:03:20.329 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46579.log 2024-03-08T06:03:20.330 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47386.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54550.log.gz 2024-03-08T06:03:20.330 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51080.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47386.log.gz 2024-03-08T06:03:20.330 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52788.log 2024-03-08T06:03:20.330 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51080.log.gz 2024-03-08T06:03:20.330 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55832.log 2024-03-08T06:03:20.330 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46579.log: /var/log/ceph/ceph-client.admin.52788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52661.log 2024-03-08T06:03:20.330 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46579.log.gz 2024-03-08T06:03:20.331 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55832.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52788.log.gz 2024-03-08T06:03:20.331 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47530.log 2024-03-08T06:03:20.331 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52661.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55832.log.gz 2024-03-08T06:03:20.331 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43849.log 2024-03-08T06:03:20.331 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52661.log.gz 2024-03-08T06:03:20.331 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49631.log 2024-03-08T06:03:20.331 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47530.log.gz 2024-03-08T06:03:20.331 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54340.log 2024-03-08T06:03:20.331 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43849.log: /var/log/ceph/ceph-client.admin.49631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53647.log 2024-03-08T06:03:20.332 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43849.log.gz 2024-03-08T06:03:20.332 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54340.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49631.log.gz 2024-03-08T06:03:20.332 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47572.log 2024-03-08T06:03:20.332 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51938.log 2024-03-08T06:03:20.332 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53647.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54340.log.gz 2024-03-08T06:03:20.332 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52140.log 2024-03-08T06:03:20.332 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53647.log.gz 2024-03-08T06:03:20.332 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51938.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47572.log.gz 2024-03-08T06:03:20.333 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-08T06:03:20.333 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46587.log 2024-03-08T06:03:20.333 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51938.log.gz 2024-03-08T06:03:20.333 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45303.log 2024-03-08T06:03:20.333 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52140.log.gz 2024-03-08T06:03:20.333 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55538.log 2024-03-08T06:03:20.333 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46587.log.gz 2024-03-08T06:03:20.333 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49505.log 2024-03-08T06:03:20.334 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55538.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45303.log.gz 2024-03-08T06:03:20.334 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45691.log 2024-03-08T06:03:20.334 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49505.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55538.log.gz 2024-03-08T06:03:20.334 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54256.log 2024-03-08T06:03:20.334 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49505.log.gz 2024-03-08T06:03:20.334 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51248.log 2024-03-08T06:03:20.334 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51668.log 2024-03-08T06:03:20.334 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54256.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45691.log.gz 2024-03-08T06:03:20.335 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55496.log 2024-03-08T06:03:20.335 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54256.log.gz 2024-03-08T06:03:20.335 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51668.log: gzip -5 89.0% --verbose -- /var/log/ceph/ceph-client.admin.49715.log 2024-03-08T06:03:20.335 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51248.log.gz 2024-03-08T06:03:20.335 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55496.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51668.log.gz 2024-03-08T06:03:20.335 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53377.log 2024-03-08T06:03:20.335 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.55496.log.gz --verbose 2024-03-08T06:03:20.335 INFO:teuthology.orchestra.run.smithi076.stderr: -- /var/log/ceph/ceph-client.admin.50834.log 2024-03-08T06:03:20.335 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49715.log: /var/log/ceph/ceph-client.0.53377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53190.log 2024-03-08T06:03:20.336 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49715.log.gz 2024-03-08T06:03:20.336 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.0.53377.log.gz 2024-03-08T06:03:20.336 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46583.log 2024-03-08T06:03:20.336 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50834.log.gz 2024-03-08T06:03:20.336 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54298.log 2024-03-08T06:03:20.336 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47656.log 2024-03-08T06:03:20.336 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46583.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53190.log.gz 2024-03-08T06:03:20.336 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53868.log 2024-03-08T06:03:20.336 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46583.log.gz 2024-03-08T06:03:20.337 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47656.log: gzip -5 --verbose -- 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54298.log.gz 2024-03-08T06:03:20.337 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-client.admin.54676.log 2024-03-08T06:03:20.337 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47656.log.gz 2024-03-08T06:03:20.337 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50876.log 2024-03-08T06:03:20.337 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53868.log.gz 2024-03-08T06:03:20.337 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53910.log 2024-03-08T06:03:20.337 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50876.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54676.log.gz 2024-03-08T06:03:20.337 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50660.log 2024-03-08T06:03:20.338 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50876.log.gz 2024-03-08T06:03:20.338 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54802.log 2024-03-08T06:03:20.338 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46251.log 2024-03-08T06:03:20.338 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50660.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53910.log.gz 2024-03-08T06:03:20.338 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54802.log: gzip -5 87.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.50660.log.gz /var/log/ceph/ceph-client.admin.46585.log 2024-03-08T06:03:20.338 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-03-08T06:03:20.338 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46251.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54802.log.gz 2024-03-08T06:03:20.338 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31151.log 2024-03-08T06:03:20.338 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46251.log.gz 2024-03-08T06:03:20.339 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43369.log 2024-03-08T06:03:20.339 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph.tmp-client.admin.31151.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46585.log.gz 2024-03-08T06:03:20.339 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31151.log.gz 2024-03-08T06:03:20.339 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54718.log 2024-03-08T06:03:20.339 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53040.log 2024-03-08T06:03:20.339 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43369.log.gz 2024-03-08T06:03:20.339 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43891.log 2024-03-08T06:03:20.339 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53040.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54718.log.gz 2024-03-08T06:03:20.339 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-08T06:03:20.340 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43891.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53040.log.gz 2024-03-08T06:03:20.340 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49211.log 2024-03-08T06:03:20.340 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2%/var/log/ceph/ceph-mgr.y.log: -- replaced with /var/log/ceph/ceph-client.admin.43891.log.gz 2024-03-08T06:03:20.340 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52224.log 2024-03-08T06:03:20.340 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45571.log 2024-03-08T06:03:20.340 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49211.log.gz 2024-03-08T06:03:20.340 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53082.log 2024-03-08T06:03:20.340 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52224.log.gz 2024-03-08T06:03:20.341 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55622.log 2024-03-08T06:03:20.341 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47866.log 2024-03-08T06:03:20.341 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53082.log: -- replaced with /var/log/ceph/ceph-client.admin.45571.log.gz 2024-03-08T06:03:20.341 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46931.log 2024-03-08T06:03:20.341 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53082.log.gz 2024-03-08T06:03:20.341 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- 89.1%/var/log/ceph/ceph-client.admin.47866.log: -- replaced with /var/log/ceph/ceph-client.admin.55622.log.gz 2024-03-08T06:03:20.341 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-client.admin.44485.log 2024-03-08T06:03:20.341 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49074.log 2024-03-08T06:03:20.341 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46931.log.gz/var/log/ceph/ceph-client.admin.44485.log: 2024-03-08T06:03:20.342 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54424.log 2024-03-08T06:03:20.342 INFO:teuthology.orchestra.run.smithi076.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47866.log.gz 2024-03-08T06:03:20.342 INFO:teuthology.orchestra.run.smithi076.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44485.log.gz 2024-03-08T06:03:20.342 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55454.log 2024-03-08T06:03:20.342 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49074.log: gzip/var/log/ceph/ceph-client.admin.54424.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47782.log 2024-03-08T06:03:20.342 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9%/var/log/ceph/ceph-client.admin.55454.log: -- replaced with /var/log/ceph/ceph-client.admin.49074.log.gz 2024-03-08T06:03:20.342 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54424.log.gz 2024-03-08T06:03:20.342 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53455.log 2024-03-08T06:03:20.342 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55454.log.gz 2024-03-08T06:03:20.343 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52998.log 2024-03-08T06:03:20.343 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47782.log: /var/log/ceph/ceph-client.admin.53455.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47782.log.gz 2024-03-08T06:03:20.343 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45817.log 2024-03-08T06:03:20.343 INFO:teuthology.orchestra.run.smithi076.stderr:gzip/var/log/ceph/ceph-client.admin.52998.log: -5 --verbose -- 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53455.log.gz 2024-03-08T06:03:20.343 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-client.admin.49337.log 2024-03-08T06:03:20.343 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45817.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52998.log.gz 2024-03-08T06:03:20.343 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45817.log.gz 2024-03-08T06:03:20.343 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-osd.0.log 2024-03-08T06:03:20.343 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44167.log 2024-03-08T06:03:20.343 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49337.log.gz 2024-03-08T06:03:20.344 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55790.log 2024-03-08T06:03:20.344 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50330.log 2024-03-08T06:03:20.344 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55790.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44167.log.gz 2024-03-08T06:03:20.344 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55244.log 2024-03-08T06:03:20.344 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55790.log.gzgzip 2024-03-08T06:03:20.344 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44081.log 2024-03-08T06:03:20.344 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49463.log 2024-03-08T06:03:20.344 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55244.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50330.log.gz 2024-03-08T06:03:20.344 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55706.log 2024-03-08T06:03:20.345 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55244.log.gz 2024-03-08T06:03:20.345 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44081.log.gz 2024-03-08T06:03:20.345 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53784.log 2024-03-08T06:03:20.345 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49463.log.gz 2024-03-08T06:03:20.345 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50960.log 2024-03-08T06:03:20.345 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55706.log.gz 2024-03-08T06:03:20.345 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54156.log 2024-03-08T06:03:20.345 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6%/var/log/ceph/ceph-client.admin.50960.log: -- replaced with /var/log/ceph/ceph-client.admin.53784.log.gz 2024-03-08T06:03:20.345 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51896.log 2024-03-08T06:03:20.346 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50960.log.gz/var/log/ceph/ceph-client.admin.54156.log: 2024-03-08T06:03:20.346 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55412.log 2024-03-08T06:03:20.346 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54156.log.gz 2024-03-08T06:03:20.346 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51812.log 2024-03-08T06:03:20.346 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51896.log.gz 2024-03-08T06:03:20.346 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49168.log 2024-03-08T06:03:20.346 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55412.log: /var/log/ceph/ceph-client.admin.51812.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55412.log.gz 2024-03-08T06:03:20.346 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51812.log.gz 2024-03-08T06:03:20.347 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-08T06:03:20.347 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44209.log 2024-03-08T06:03:20.347 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49168.log: gzip/var/log/ceph/ceph.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54072.log 2024-03-08T06:03:20.347 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49168.log.gz 2024-03-08T06:03:20.347 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44209.log: gzip -5 --verbose -- 85.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-08T06:03:20.347 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-osd.2.log 2024-03-08T06:03:20.347 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51002.log 2024-03-08T06:03:20.347 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44209.log.gz 2024-03-08T06:03:20.347 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54072.log.gz 2024-03-08T06:03:20.348 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53826.log 2024-03-08T06:03:20.348 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.51002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46069.log 2024-03-08T06:03:20.348 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53826.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51002.log.gz 2024-03-08T06:03:20.348 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53826.log.gz 2024-03-08T06:03:20.348 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-08T06:03:20.348 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45135.log 2024-03-08T06:03:20.348 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mds.c.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46069.log.gz 2024-03-08T06:03:20.348 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48947.log 2024-03-08T06:03:20.348 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45135.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-08T06:03:20.349 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45135.log.gz 2024-03-08T06:03:20.349 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43723.log 2024-03-08T06:03:20.349 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50534.log 2024-03-08T06:03:20.349 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48947.log.gz 2024-03-08T06:03:20.349 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43723.log: gzip -5 --verbose 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43723.log.gz 2024-03-08T06:03:20.349 INFO:teuthology.orchestra.run.smithi076.stderr: -- /var/log/ceph/ceph-client.admin.52182.log 2024-03-08T06:03:20.349 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.50534.log: -- /var/log/ceph/ceph-client.admin.45177.log 2024-03-08T06:03:20.349 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50534.log.gz 2024-03-08T06:03:20.349 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44377.log 2024-03-08T06:03:20.350 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45177.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52182.log.gz 2024-03-08T06:03:20.350 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45177.log.gz 2024-03-08T06:03:20.350 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52478.log 2024-03-08T06:03:20.350 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48344.log 2024-03-08T06:03:20.350 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52478.log: /var/log/ceph/ceph-client.admin.44377.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44377.log.gz 2024-03-08T06:03:20.350 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52478.log.gz 2024-03-08T06:03:20.350 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51374.log 2024-03-08T06:03:20.350 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50576.log 2024-03-08T06:03:20.351 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.0.48344.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48344.log.gz 2024-03-08T06:03:20.351 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45093.log 2024-03-08T06:03:20.351 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51374.log.gz 2024-03-08T06:03:20.351 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50576.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50576.log.gz 2024-03-08T06:03:20.351 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50204.log 2024-03-08T06:03:20.351 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53124.log 2024-03-08T06:03:20.351 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45093.log: /var/log/ceph/ceph-client.admin.50204.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45093.log.gz 2024-03-08T06:03:20.351 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46938.log 2024-03-08T06:03:20.351 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50204.log.gz 2024-03-08T06:03:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53124.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.53124.log.gz 2024-03-08T06:03:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45733.log 2024-03-08T06:03:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50288.log 2024-03-08T06:03:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46938.log: /var/log/ceph/ceph-client.admin.45733.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46938.log.gz 2024-03-08T06:03:20.352 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45733.log.gz 2024-03-08T06:03:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51542.log 2024-03-08T06:03:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43935.log 2024-03-08T06:03:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50288.log: /var/log/ceph/ceph-client.admin.51542.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50288.log.gz 2024-03-08T06:03:20.353 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51542.log.gz 2024-03-08T06:03:20.353 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55916.log 2024-03-08T06:03:20.353 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52830.log 2024-03-08T06:03:20.353 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55916.log: /var/log/ceph/ceph-client.admin.43935.log: 87.4% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43935.log.gz 2024-03-08T06:03:20.353 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55916.log.gz 2024-03-08T06:03:20.353 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43680.log 2024-03-08T06:03:20.353 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45445.log 2024-03-08T06:03:20.353 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52830.log: /var/log/ceph/ceph-client.admin.43680.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52830.log.gz 2024-03-08T06:03:20.353 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43680.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52606.log 2024-03-08T06:03:20.354 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-03-08T06:03:20.354 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43979.log 2024-03-08T06:03:20.354 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45445.log: /var/log/ceph/ceph-client.admin.52606.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45445.log.gz 2024-03-08T06:03:20.354 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56070.log 2024-03-08T06:03:20.354 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52606.log.gz 2024-03-08T06:03:20.354 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44901.log 2024-03-08T06:03:20.354 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.56070.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43979.log.gz 2024-03-08T06:03:20.354 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56070.log.gz 2024-03-08T06:03:20.355 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51332.log 2024-03-08T06:03:20.355 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-08T06:03:20.355 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51332.log: /var/log/ceph/ceph-client.admin.44901.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51332.log.gz 2024-03-08T06:03:20.355 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46798.log 2024-03-08T06:03:20.355 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mds.a.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.44901.log.gz 2024-03-08T06:03:20.355 INFO:teuthology.orchestra.run.smithi076.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-08T06:03:20.355 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52268.log 2024-03-08T06:03:20.355 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54592.log 2024-03-08T06:03:20.356 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52268.log: /var/log/ceph/ceph-client.admin.46798.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46798.log.gz 2024-03-08T06:03:20.356 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52268.log.gz 2024-03-08T06:03:20.356 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54592.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54592.log.gz 2024-03-08T06:03:20.356 INFO:teuthology.orchestra.run.smithi076.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-08T06:03:20.356 INFO:teuthology.orchestra.run.smithi081.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-08T06:03:20.356 INFO:teuthology.orchestra.run.smithi081.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-08T06:03:20.396 INFO:teuthology.orchestra.run.smithi076.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-08T06:03:20.896 INFO:teuthology.orchestra.run.smithi076.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-08T06:03:20.910 INFO:teuthology.orchestra.run.smithi081.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-08T06:03:20.925 INFO:teuthology.orchestra.run.smithi081.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-08T06:03:20.968 INFO:teuthology.orchestra.run.smithi076.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-08T06:03:21.010 INFO:teuthology.orchestra.run.smithi076.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-08T06:03:21.225 INFO:teuthology.orchestra.run.smithi081.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-08T06:03:21.226 INFO:teuthology.orchestra.run.smithi081.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-08T06:03:21.410 INFO:teuthology.orchestra.run.smithi076.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-08T06:03:21.412 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-03-08T06:03:21.412 INFO:teuthology.orchestra.run.smithi076.stderr:real 0m1.250s 2024-03-08T06:03:21.412 INFO:teuthology.orchestra.run.smithi076.stderr:user 0m3.642s 2024-03-08T06:03:21.413 INFO:teuthology.orchestra.run.smithi076.stderr:sys 0m0.279s 2024-03-08T06:03:21.500 INFO:teuthology.orchestra.run.smithi081.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-08T06:03:21.502 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-03-08T06:03:21.502 INFO:teuthology.orchestra.run.smithi081.stderr:real 0m1.310s 2024-03-08T06:03:21.503 INFO:teuthology.orchestra.run.smithi081.stderr:user 0m4.854s 2024-03-08T06:03:21.503 INFO:teuthology.orchestra.run.smithi081.stderr:sys 0m0.182s 2024-03-08T06:03:21.503 INFO:tasks.ceph:Archiving logs... 2024-03-08T06:03:21.503 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/log/ceph to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417/remote/smithi076/log 2024-03-08T06:03:21.504 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-08T06:03:22.232 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/log/ceph to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417/remote/smithi081/log 2024-03-08T06:03:22.232 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-08T06:03:22.875 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-08T06:03:22.884 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-08T06:03:22.885 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-08T06:03:22.887 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-08T06:03:22.966 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-08T06:03:22.975 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-08T06:03:22.975 DEBUG:teuthology.orchestra.run.smithi076:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T06:03:22.978 DEBUG:teuthology.orchestra.run.smithi081:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T06:03:22.991 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-03-08T06:03:22.992 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-03-08T06:03:22.995 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T06:03:22.995 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-03-08T06:03:22.995 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 7 +1713us[+1713us] +/- 109ms 2024-03-08T06:03:22.996 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 -1424us[-1424us] +/- 69ms 2024-03-08T06:03:22.996 INFO:teuthology.orchestra.run.smithi076.stdout:^* hv03.front.sepia.ceph.com 3 6 377 8 +371us[ +372us] +/- 66ms 2024-03-08T06:03:22.996 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T06:03:22.997 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T06:03:22.998 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-03-08T06:03:22.998 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 8 +1708us[+1717us] +/- 109ms 2024-03-08T06:03:22.998 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 -1419us[-1409us] +/- 69ms 2024-03-08T06:03:22.998 INFO:teuthology.orchestra.run.smithi081.stdout:^* hv03.front.sepia.ceph.com 3 6 377 4 +367us[ +376us] +/- 66ms 2024-03-08T06:03:22.998 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T06:03:22.999 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-08T06:03:23.007 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-08T06:03:23.008 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-08T06:03:23.019 DEBUG:teuthology.orchestra.run.smithi076:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-08T06:03:23.115 DEBUG:teuthology.orchestra.run.smithi081:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-08T06:03:23.184 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-08T06:03:23.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:03:23.215 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-08T06:03:23.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:03:23.246 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-08T06:03:23.256 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-08T06:03:23.274 INFO:teuthology.task.internal:Duration was 586.569962 seconds 2024-03-08T06:03:23.274 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-08T06:03:23.283 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-08T06:03:23.284 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-08T06:03:23.286 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-08T06:03:23.328 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T06:03:23.330 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T06:03:23.496 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-08T06:03:23.496 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi076.front.sepia.ceph.com 2024-03-08T06:03:23.496 DEBUG:teuthology.orchestra.run.smithi076:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-08T06:03:23.515 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi081.front.sepia.ceph.com 2024-03-08T06:03:23.515 DEBUG:teuthology.orchestra.run.smithi081:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-08T06:03:23.551 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-08T06:03:23.551 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-08T06:03:23.557 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-08T06:03:23.619 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-08T06:03:23.620 DEBUG:teuthology.orchestra.run.smithi076:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-08T06:03:23.626 DEBUG:teuthology.orchestra.run.smithi081:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-08T06:03:23.759 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-08T06:03:23.769 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-08T06:03:23.769 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-08T06:03:23.796 DEBUG:teuthology.orchestra.run.smithi081:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-08T06:03:23.823 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-08T06:03:23.834 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-08T06:03:23.839 DEBUG:teuthology.orchestra.run.smithi081:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-08T06:03:23.862 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = core 2024-03-08T06:03:23.888 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = core 2024-03-08T06:03:23.904 DEBUG:teuthology.orchestra.run.smithi076:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-08T06:03:23.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:03:23.948 DEBUG:teuthology.orchestra.run.smithi081:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-08T06:03:23.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:03:23.962 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-08T06:03:23.974 INFO:teuthology.task.internal:Transferring archived files... 2024-03-08T06:03:23.974 DEBUG:teuthology.misc:Transferring archived files from smithi076:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417/remote/smithi076 2024-03-08T06:03:23.974 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-08T06:03:24.024 DEBUG:teuthology.misc:Transferring archived files from smithi081:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417/remote/smithi081 2024-03-08T06:03:24.024 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-08T06:03:24.058 INFO:teuthology.task.internal:Removing archive directory... 2024-03-08T06:03:24.058 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-08T06:03:24.060 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-08T06:03:24.108 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-08T06:03:24.118 INFO:teuthology.task.internal:Not uploading archives. 2024-03-08T06:03:24.118 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-08T06:03:24.133 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-08T06:03:24.134 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-08T06:03:24.136 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-08T06:03:24.148 INFO:teuthology.orchestra.run.smithi076.stdout: 265301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 8 06:03 /home/ubuntu/cephtest 2024-03-08T06:03:24.162 INFO:teuthology.orchestra.run.smithi081.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 8 06:03 /home/ubuntu/cephtest 2024-03-08T06:03:24.163 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-08T06:03:24.174 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-08T06:03:24.200 INFO:teuthology.nuke:Checking targets against current locks 2024-03-08T06:03:24.228 DEBUG:teuthology.nuke:shortname: smithi076 2024-03-08T06:03:24.228 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-08T06:03:24.253 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 05:44:35.857957', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKhnkFqUvrG7IuDf2BSBP5sxy4mKIsOWPSx8+7GZs+T1PXg7Ik6V+ReoC7jNDuzTxr44jT+2K9nPAyuWL4OFTgk='} 2024-03-08T06:03:24.256 DEBUG:teuthology.nuke:shortname: smithi081 2024-03-08T06:03:24.256 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-08T06:03:24.281 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585417', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 05:44:35.859438', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBRX6/RIgJsEJ5Pga1/gJkuha9CjhD24Ar+FSY6norAtgGSq5IKJS2vFEpHLUJ/WwLDDmn/t3NCnOcTMIfvYZwc='} 2024-03-08T06:03:24.311 INFO:teuthology.orchestra.console.smithi076:Power off 2024-03-08T06:03:24.312 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-08T06:03:24.340 DEBUG:teuthology.orchestra.console.smithi076:power off output: Chassis Power Control: Down/Off 2024-03-08T06:03:24.340 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T06:03:24.358 INFO:teuthology.orchestra.console.smithi081:Power off 2024-03-08T06:03:24.358 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-08T06:03:24.367 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2024-03-08T06:03:24.380 DEBUG:teuthology.orchestra.console.smithi081:power off output: Chassis Power Control: Down/Off 2024-03-08T06:03:24.380 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T06:03:24.402 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on 2024-03-08T06:03:28.369 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T06:03:28.403 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T06:03:28.483 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2024-03-08T06:03:28.515 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on 2024-03-08T06:03:32.486 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T06:03:32.517 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T06:03:32.600 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is off 2024-03-08T06:03:32.627 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is off 2024-03-08T06:03:32.701 INFO:teuthology.orchestra.console.smithi076:Power off completed 2024-03-08T06:03:32.729 INFO:teuthology.orchestra.console.smithi081:Power off completed 2024-03-08T06:03:32.835 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin} duration: 586.5699617862701 failure_reason: 'Test failure: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters)' flavor: default owner: scheduled_jcollin@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=08b5231ef43244be9c46188226d0f8e9 status: fail success: false 2024-03-08T06:03:32.835 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-08T06:03:32.923 INFO:teuthology.run:FAIL